{"id":51318689,"url":"https://github.com/netdata/plugin-ipc","last_synced_at":"2026-07-01T10:31:46.491Z","repository":{"id":343107021,"uuid":"1176084855","full_name":"netdata/plugin-ipc","owner":"netdata","description":null,"archived":false,"fork":false,"pushed_at":"2026-06-29T13:06:43.000Z","size":15964,"stargazers_count":1,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-29T15:09:04.923Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/netdata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-03-08T15:45:04.000Z","updated_at":"2026-06-29T13:07:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"756fa306-6ec5-4752-9caa-8aeeaee45bb9","html_url":"https://github.com/netdata/plugin-ipc","commit_stats":null,"previous_names":["netdata/plugin-ipc"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/netdata/plugin-ipc","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netdata%2Fplugin-ipc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netdata%2Fplugin-ipc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netdata%2Fplugin-ipc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netdata%2Fplugin-ipc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/netdata","download_url":"https://codeload.github.com/netdata/plugin-ipc/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netdata%2Fplugin-ipc/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35003452,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-01T02:00:05.325Z","response_time":130,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-01T10:31:45.755Z","updated_at":"2026-07-01T10:31:46.485Z","avatar_url":"https://github.com/netdata.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# plugin-ipc\n\nCross-language IPC library for Netdata plugins and helper services.\n\nThis repository contains the C library, Rust crate, and Go package for the\nsame wire contracts and typed APIs. The goal is simple:\n\n- one specification\n- one interoperable protocol stack\n- one typed service model\n- one local snapshot/cache helper layer\n- three implementations: C, Rust, Go\n\nThis README is a summary of the current verified state of the repository.\nThe authoritative specifications live under [docs/](docs/README.md).\n\n## Non-Negotiable Contract\n\n- These rules apply to every implementation language and every service unless\n  a future spec explicitly changes the global contract.\n- C, Rust, and Go must implement the same wire contract and typed behavior.\n- NetIPC does not do backward-compatible, forward-compatible, or best-effort\n  decoding across method, layout, status, echoed-key, or generation drift.\n- Netdata plugins and NetIPC providers/clients must match the documented\n  contract exactly. Any mismatch is rejected.\n- Mixed-generation stitched lookup responses are not supported.\n- Level 2 lookup callers pass typed semantic keys; they do not split requests\n  around transport payload budgets and do not stitch responses manually.\n- Lookup `PAYLOAD_EXCEEDED` is an internal Level 2 retry signal. The client\n  retries only the affected suffix and returns one logical response.\n- Lookup `OVERSIZED_ITEM` is a final per-item outcome. One oversized valid item\n  must not invalidate the rest of the logical lookup batch.\n- Payload budgets and logical lookup ceilings are initialization policy.\n  Zero-valued fields use documented defaults; consumers can override them for\n  small IoT systems or large-memory HPC deployments.\n- Named defaults are defaults, not protocol hard limits. Explicit client/server\n  initialization config is the deployment authority.\n\n## Service Model\n\nThe public contract is service-oriented, not plugin-oriented:\n\n- clients connect to a specific service kind\n- clients do not care which plugin/process serves that service\n- one service endpoint serves exactly one request kind\n- sessions are long-lived and reused for many requests\n\nExamples of service kinds:\n\n- `cgroups-snapshot`\n- `cgroups-lookup`\n- `apps-lookup`\n- `ip-to-asn`\n- `pid-traffic`\n\nOperational implications:\n\n- startup order is not guaranteed\n- a client may start before its provider exists\n- services may disappear and reappear during runtime\n- enrichments are optional by design\n- clients refresh / reconnect from their normal loop and must tolerate service absence\n\n## What This Repository Implements\n\n- **Level 1 transport**\n  - connection lifecycle\n  - handshake and profile negotiation\n  - framing, chunking, batching, pipelining\n  - baseline and shared-memory transports\n- **Codec**\n  - wire encode/decode\n  - typed views\n  - response builders\n  - validation rules\n- **Level 2 typed API**\n  - typed client calls\n  - managed typed servers\n  - retry / reconnect behavior\n  - internal reusable buffers\n- **Level 3 snapshot API**\n  - refresh\n  - local cache construction\n  - fast hash lookup\n  - cache preservation on failure\n\nThe design is layered:\n\n- **Level 1** and **Codec** are parallel building blocks\n- **Level 2** composes Level 1 + Codec\n- **Level 3** builds on Level 2\n\nSee:\n\n- [docs/README.md](docs/README.md)\n- [docs/level2-typed-api.md](docs/level2-typed-api.md)\n- [docs/level3-snapshot-api.md](docs/level3-snapshot-api.md)\n- [docs/getting-started.md](docs/getting-started.md)\n- [docs/netipc-integrator-skill.md](docs/netipc-integrator-skill.md)\n\n## Platforms And Transports\n\n| Platform | Baseline transport | Negotiated fast path | Languages |\n|---|---|---|---|\n| POSIX / Linux | Unix domain `SOCK_SEQPACKET` | POSIX shared memory | C, Rust, Go |\n| Windows | Named Pipes | Windows shared memory | C, Rust, Go |\n\nImportant facts:\n\n- the same wire contracts are implemented in all three languages\n- cross-language interoperability is mandatory\n- Go stays pure Go, without `cgo`\n- Level 2 and Level 3 are transport-agnostic from the caller perspective\n\n## API Levels\n\n### Level 1: Transport\n\nLevel 1 works with framed byte messages.\n\nIt owns:\n\n- send / receive\n- message IDs\n- batch directories\n- chunk continuation\n- profile negotiation\n- transport-specific session details\n\nRelevant specs:\n\n- [docs/level1-transport.md](docs/level1-transport.md)\n- [docs/level1-wire-envelope.md](docs/level1-wire-envelope.md)\n- [docs/level1-posix-uds.md](docs/level1-posix-uds.md)\n- [docs/level1-posix-shm.md](docs/level1-posix-shm.md)\n- [docs/level1-windows-np.md](docs/level1-windows-np.md)\n- [docs/level1-windows-shm.md](docs/level1-windows-shm.md)\n\n### Codec\n\nCodec is pure wire-format logic.\n\nIt owns:\n\n- encode / decode\n- typed views over payload bytes\n- response builders\n- validation of field layout and bounds\n\nIt does **not** own:\n\n- sockets\n- pipes\n- shared memory mappings\n- retries\n- cache policy\n\nRelevant specs:\n\n- [docs/codec.md](docs/codec.md)\n- [docs/codec-cgroups-snapshot.md](docs/codec-cgroups-snapshot.md)\n- [docs/codec-cgroups-lookup.md](docs/codec-cgroups-lookup.md)\n- [docs/codec-apps-lookup.md](docs/codec-apps-lookup.md)\n\n### Level 2: Typed API\n\nLevel 2 is the public convenience layer.\n\nThe public contract is:\n\n- clients issue typed calls\n- servers export one typed service kind\n- one server endpoint serves one request kind only\n- callers do not manage transport scratch buffers\n- callers do not manipulate raw payload bytes\n\nRelevant specs:\n\n- [docs/level2-typed-api.md](docs/level2-typed-api.md)\n- [docs/getting-started.md](docs/getting-started.md)\n\n### Level 3: Snapshot / Cache\n\nLevel 3 provides:\n\n- typed snapshot refresh\n- local materialization\n- O(1)-style hash lookup on the hot path\n- cache retention across refresh failures\n\nRelevant spec:\n\n- [docs/level3-snapshot-api.md](docs/level3-snapshot-api.md)\n\n## Interoperability\n\nThis repository is intentionally built around interoperability, not\nsingle-language wrappers.\n\nWhat is covered:\n\n- C client -\u003e C / Rust / Go server\n- Rust client -\u003e C / Rust / Go server\n- Go client -\u003e C / Rust / Go server\n- baseline transport matrices on POSIX and Windows\n- shared-memory matrices on POSIX and Windows\n- typed Level 2 services\n- snapshot refresh and local lookup flows\n- benchmark matrices across all directed pairs\n\nThe interop results are validated by the test suite and by the checked-in\nbenchmark reports:\n\n- [benchmarks-posix.md](benchmarks-posix.md)\n- [benchmarks-windows.md](benchmarks-windows.md)\n\n## Performance Snapshot\n\nThe repository includes checked-in benchmark reports with complete,\nfail-closed matrices:\n\n- POSIX report: [benchmarks-posix.md](benchmarks-posix.md)\n  - generated `2026-05-25`\n  - machine: local benchmark host\n  - complete matrix rows: `297`\n- Windows report: [benchmarks-windows.md](benchmarks-windows.md)\n  - generated `2026-05-25`\n  - machine: local Windows benchmark host\n  - complete matrix rows: `201`\n\nHeadline numbers from the current checked-in reports:\n\n- **POSIX baseline UDS ping-pong**\n  - `142.1k` to `169.5k` req/s across the 3x3 language matrix\n- **POSIX SHM ping-pong**\n  - `2.32M` to `3.22M` req/s\n- **POSIX UDS batch ping-pong**\n  - `20.18M` to `30.13M` req/s\n- **POSIX SHM batch ping-pong**\n  - `28.71M` to `48.95M` req/s\n- **POSIX UDS pipeline (depth=16)**\n  - `440.4k` to `620.6k` req/s\n- **POSIX UDS pipeline+batch (depth=16)**\n  - `41.61M` to `76.45M` req/s\n- **POSIX snapshot refresh**\n  - baseline: `131.2k` to `156.3k` req/s\n  - SHM: `1.10M` to `1.52M` req/s\n- **POSIX lookup methods**\n  - `cgroups-lookup`: `31.8k` to `1.10M` req/s across max-rate scenarios\n  - `apps-lookup`: `35.7k` to `1.34M` req/s across max-rate scenarios\n- **POSIX local cache lookup**\n  - C: `173.47M` req/s\n  - Go: `138.48M` req/s\n  - Rust: `147.68M` req/s\n\n- **Windows Named Pipe ping-pong**\n  - `18.2k` to `21.0k` req/s\n- **Windows SHM ping-pong**\n  - `2.10M` to `2.72M` req/s\n- **Windows Named Pipe batch ping-pong**\n  - `7.01M` to `8.55M` req/s\n- **Windows SHM batch ping-pong**\n  - `36.49M` to `58.77M` req/s\n- **Windows Named Pipe pipeline (depth=16)**\n  - `245.4k` to `270.5k` req/s\n- **Windows Named Pipe pipeline+batch (depth=16)**\n  - `28.98M` to `41.27M` req/s\n- **Windows snapshot refresh**\n  - Named Pipe: `16.1k` to `21.0k` req/s\n  - SHM: `857.8k` to `1.26M` req/s\n- **Windows local cache lookup**\n  - C: `130.65M` req/s\n  - Go: `107.47M` req/s\n  - Rust: `164.31M` req/s\n\nThe full reports include:\n\n- per-pair throughput\n- latency percentiles\n- client and server CPU\n- complete scenario validation summaries\n- performance floor checks\n\n## Reliability, Testing, And Coverage\n\nThe repo is not asking the reader to trust the design on words alone.\nThe current validation story includes:\n\n- CMake-based build and `ctest` workflows\n- unit tests\n- cross-language interop tests\n- typed service tests\n- transport tests\n- shared-memory tests\n- coverage scripts for C, Go, and Rust\n- benchmark generators that reject incomplete matrices\n\n### Current verified state\n\nLinux / POSIX:\n\n- build: passing\n- `ctest`: `46/46` passing\n- C coverage: `94.1%`\n- Go coverage: `95.8%`\n- Rust coverage: `98.57%`\n  - measured with `cargo-llvm-cov`\n  - Linux run now excludes Windows-tagged Rust files from the Linux total\n  - Unix Rust service tests now live in a separate `cgroups_unix_tests.rs` file\n  - Unix Rust transport tests now live in separate `posix_tests.rs` and `shm_tests.rs` files\n  - the small Rust protocol test modules stay inline for now because externalizing them lowered the headline total without enough runtime-signal benefit\n\nWindows (`win11`):\n\n- build: passing\n- `ctest`: `28/28` passing\n- C coverage: `93.2%`\n- Go coverage: `95.4%`\n- Rust coverage: `92.08%`\n\nImportant honesty point:\n\n- core build, transport, service, interop, and benchmark validation is strong\n  on both Linux and Windows\n- Linux still has broader chaos / hardening / stress breadth than Windows\n- so the platforms are in **good functional parity**, but not yet in\n  **full validation parity**\n\nCoverage details:\n\n- [WINDOWS-COVERAGE.md](WINDOWS-COVERAGE.md)\n- [COVERAGE-EXCLUSIONS.md](COVERAGE-EXCLUSIONS.md)\n\n## Specifications And Trust Model\n\nThe specs are authoritative.\n\nRule:\n\n- when code and spec disagree, the spec wins unless explicitly revised\n\nStart here:\n\n- [docs/README.md](docs/README.md)\n\nRecommended reading order:\n\n1. [docs/README.md](docs/README.md)\n2. [docs/getting-started.md](docs/getting-started.md)\n3. [docs/level2-typed-api.md](docs/level2-typed-api.md)\n4. [docs/level3-snapshot-api.md](docs/level3-snapshot-api.md)\n5. the relevant Level 1 transport spec for your platform\n\n## Building And Running\n\n### Linux / POSIX\n\n```bash\ncmake -S . -B build -DCMAKE_BUILD_TYPE=RelWithDebInfo\ncmake --build build -j4\nctest --test-dir build --output-on-failure -j4\n```\n\nCoverage:\n\n```bash\nbash tests/run-coverage-c.sh\nbash tests/run-coverage-go.sh\nbash tests/run-coverage-rust.sh\n```\n\n### Windows (`win11`)\n\nUse a `mingw64` shell with native Windows `cargo` and `go` ahead of any MSYS\ntoolchain copies in `PATH`. This remains the native Windows sign-off path.\n\n```bash\ncmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo\ncmake --build build -j4\nctest --test-dir build --output-on-failure -j4\n```\n\nMSYS transition validation:\n\n```bash\nbash tests/run-windows-msys-validation.sh\n```\n\nThat lane builds the C artifacts with `/usr/bin/gcc`, runs the targeted\nWindows functional/interoperability slice, repeats `test_win_shm`, and compares\na bounded benchmark subset against the native `mingw64` lane with explicit\nthroughput floors per scenario. Policy-failed benchmark rows are rerun as paired\nnative+MSYS rows before final failure, with prior attempts saved next to the\nfinal `policy.csv`. It does not replace native Windows sign-off.\n\nCoverage:\n\n```bash\nbash tests/run-coverage-c-windows.sh\nbash tests/run-coverage-go-windows.sh\nbash tests/run-coverage-rust-windows.sh\n```\n\nVerifier:\n\n```bash\nbash tests/run-verifier-windows.sh\n```\n\nFor practical Windows workflow details, see:\n\n- [WINDOWS-COVERAGE.md](WINDOWS-COVERAGE.md)\n- [TODO-pending-from-rewrite.md](TODO-pending-from-rewrite.md)\n\n## Repository Layout\n\n```text\n.\n├── bench/\n│   └── drivers/\n├── docs/\n├── src/\n│   ├── crates/netipc/\n│   ├── go/pkg/netipc/\n│   └── libnetdata/netipc/\n└── tests/\n    ├── fixtures/\n    └── run-*.sh\n```\n\nMain implementation roots:\n\n- C: [src/libnetdata/netipc/](src/libnetdata/netipc/)\n- Rust: [src/crates/netipc/](src/crates/netipc/)\n- Go: [src/go/pkg/netipc/](src/go/pkg/netipc/)\n\n## Current Limits\n\nThis is the honest current state:\n\n- coverage thresholds are enforced, but they are **not** at `100%`\n- Linux and Windows are functionally close, but Windows still has less\n  chaos / hardening / stress breadth\n- some documented exclusions still require special infrastructure such as:\n  - allocation-failure injection\n  - kernel / OS failure injection\n  - race-window orchestration\n\nThose limits are tracked explicitly instead of being hidden:\n\n- [COVERAGE-EXCLUSIONS.md](COVERAGE-EXCLUSIONS.md)\n- [TODO-pending-from-rewrite.md](TODO-pending-from-rewrite.md)\n\n## Summary\n\nIf you need the shortest trustworthy summary:\n\n- the protocol stack is specified, implemented, and measured in C, Rust, and Go\n- interop across the three languages is a core requirement, not an afterthought\n- both POSIX and Windows have validated baseline and SHM benchmark matrices\n- the public service API is typed\n- the local snapshot/cache layer is implemented and benchmarked\n- the repo has real tests, real coverage, and real benchmark artifacts checked in\n\nFor design details, start with [docs/README.md](docs/README.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetdata%2Fplugin-ipc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetdata%2Fplugin-ipc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetdata%2Fplugin-ipc/lists"}