{"id":49336244,"url":"https://github.com/datanoisetv/shannon","last_synced_at":"2026-04-27T01:02:00.093Z","repository":{"id":353466353,"uuid":"1219544017","full_name":"DatanoiseTV/shannon","owner":"DatanoiseTV","description":"Zero-instrumentation L7 observability for Linux via eBPF. See plaintext HTTP/HTTPS, gRPC, Postgres, MySQL, MongoDB, Redis, Kafka, Cassandra from any process — including through TLS — without keys, sidecars, or code changes. Pure Rust (aya), kernel 5.8+.","archived":false,"fork":false,"pushed_at":"2026-04-24T03:27:52.000Z","size":44627,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-24T03:29:15.992Z","etag":null,"topics":["apm","aya","bpf","cassandra","ebpf","ebpf-tools","grpc","http","kafka","linux","mongodb","mysql","networking","observability","postgresql","redis","rust","service-mesh","tls","tracing"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DatanoiseTV.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"docs/roadmap.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-24T01:33:14.000Z","updated_at":"2026-04-24T03:27:55.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/DatanoiseTV/shannon","commit_stats":null,"previous_names":["datanoisetv/shannon"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/DatanoiseTV/shannon","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DatanoiseTV%2Fshannon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DatanoiseTV%2Fshannon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DatanoiseTV%2Fshannon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DatanoiseTV%2Fshannon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DatanoiseTV","download_url":"https://codeload.github.com/DatanoiseTV/shannon/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DatanoiseTV%2Fshannon/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32318417,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T23:26:28.701Z","status":"ssl_error","status_checked_at":"2026-04-26T23:26:25.802Z","response_time":129,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apm","aya","bpf","cassandra","ebpf","ebpf-tools","grpc","http","kafka","linux","mongodb","mysql","networking","observability","postgresql","redis","rust","service-mesh","tls","tracing"],"created_at":"2026-04-27T01:01:59.214Z","updated_at":"2026-04-27T01:02:00.077Z","avatar_url":"https://github.com/DatanoiseTV.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# shannon\n\n\u003e Named after Claude Shannon. Turns a stream of bytes on a wire into meaning you can read.\n\n`shannon` is a zero-instrumentation L7 observability tool for Linux. Drop a single\nbinary on a host (or a Kubernetes node) and see, in real time, every HTTP, gRPC,\nPostgres, Redis, MySQL and DNS request the machine is making — including requests\nover TLS, without configuring any keys.\n\nIt does this with eBPF: kernel probes on TCP, uprobes on `libssl` (and other TLS\nruntimes), and protocol parsers in userspace. No sidecars. No application changes.\nNo private keys. No kernel modules.\n\n## Status\n\nPre-alpha. Kernel ≥ 5.8 with BTF is required. Tested on Debian 13 / kernel 6.12.\nAPI and event schema are **not** stable.\n\n## Quick start\n\n```bash\n# interactive TUI (default)\nsudo shannon\n\n# just postgres queries from pid 1234\nsudo shannon trace -p 1234 --protocol postgres\n\n# top-like view, sorted by p99 (also: rps, p50, bytes, errors)\nsudo shannon top --sort p99\n\n# scope by process name (glob) or by Kubernetes pod\nsudo shannon trace --comm 'nginx*'\nsudo shannon record --pod billing-7d8c -o billing.jsonl.zst\n\n# attach uprobes (libssl + libsqlite3) to a static binary. Covers Go apps\n# bundling their own TLS, custom Rust builds, appliance firmware in a\n# single ELF. Best-effort: missing symbols silently skip.\nsudo shannon trace --attach-bin /opt/my-app/bin/server\n\n# service-map: who talks to whom, by protocol\nsudo shannon map                                          # ANSI-redrawn table\nsudo shannon map --format tui                             # ratatui: scroll, sort, quit\nsudo shannon map --format dot | dot -Tsvg \u003e map.svg\nsudo shannon map --format json \u003e edges.ndjson\n\n# trace + dump every X.509 seen on the wire, flag self-signed / weak\n# sig-alg / short RSA / expired / long-validity anomalies inline\nsudo shannon trace --dump-certs /tmp/shannon-certs\n\n# pin those certs: dump once, review, keep the .der files you trust,\n# then re-run with --cert-pin — anything new trips \"not in allowlist\".\nsudo shannon trace --cert-pin /tmp/shannon-certs\n\n# pre-flight: is this box supported?\nshannon doctor\n\n# record everything to a file (zstd-compressed JSONL)\nsudo shannon record -o capture.jsonl.zst --rotate 100M\n\n# play it back later (pipe to jq, feed the TUI, or summarise)\nsudo shannon trace --replay capture.jsonl.zst\nshannon analyze capture.jsonl.zst\n\n# build an API catalog from observed HTTP traffic and export OpenAPI 3.0\nsudo shannon trace --catalog catalog.jsonl --openapi spec.yaml\n\n# infer a .proto schema from raw protobuf messages a service is sending\nshannon proto-infer --samples /tmp/grpc-bodies -o inferred.proto\n\n# ask a local LLM (Ollama / LM Studio / vLLM) about an existing capture\nshannon ask --catalog catalog.jsonl \"what processes talked to s3?\"\n\n# shell completions\nshannon completions zsh \u003e ~/.zfunc/_shannon\n```\n\n## Install\n\n### From a release `.deb` (Debian / Ubuntu, x86_64 + aarch64)\n\nEach tagged release ships per-arch `.deb`s and SHA-256 sums:\n\n```bash\nver=0.1.0\narch=$(dpkg --print-architecture)         # amd64 or arm64\ncurl -LO https://github.com/DatanoiseTV/shannon/releases/download/v${ver}/shannon_${ver}-1_${arch}.deb\nsudo dpkg -i shannon_${ver}-1_${arch}.deb\nsudo shannon doctor                       # confirm BTF + kprobe targets\nsudo systemctl enable --now shannon       # optional: continuous background recorder\n```\n\nThe package installs `/usr/bin/shannon`, a hardened systemd unit at\n`/usr/lib/systemd/system/shannon.service` (disabled by default — opt\nin if you want a recorder service), `/etc/default/shannon` for tunables,\nand the docs under `/usr/share/doc/shannon/`.\n\n### Kubernetes (DaemonSet)\n\n```bash\nkubectl apply -f https://raw.githubusercontent.com/DatanoiseTV/shannon/main/deploy/k8s/daemonset.yaml\n```\n\nCreates a `shannon` namespace, runs one recorder per node with\n`hostPID + hostNetwork`, writes rotating captures under each node's\n`/var/log/shannon/`, and exposes `:9750/metrics` for Prometheus.\nSee [`deploy/k8s/README.md`](deploy/k8s/README.md) for hardened\nconfig (drop `privileged`, explicit caps) and a `PodMonitor`.\n\n### Container image\n\n```bash\ndocker run --rm --privileged --pid=host --network=host \\\n  -v /sys/fs/bpf:/sys/fs/bpf \\\n  -v /sys/kernel/btf:/sys/kernel/btf:ro \\\n  ghcr.io/datanoisetv/shannon:latest doctor\n```\n\nMulti-arch (amd64 + arm64), tagged on every release plus `:latest`\non stable tags and `:main` on every push to main.\n\n### From source\n\n```bash\ngit clone https://github.com/DatanoiseTV/shannon\ncd shannon\nrustup toolchain install nightly --component rust-src\ncargo install bpf-linker\ncargo xtask build --release\nsudo ./target/release/shannon doctor\n```\n\n### Cross-building from a non-Linux workstation\n\nshannon is Linux-only (eBPF). If you develop on macOS, `scripts/push-build.sh`\npushes the tree to a Linux host and runs cargo there:\n\n```bash\n# defaults: rsync to syso@10.243.243.8:~/shannon, sources ~/.cargo/env\nSHANNON_HOST=you@yourhost SHANNON_PATH=shannon scripts/push-build.sh build\nscripts/push-build.sh test parsers       # any cargo-test filter\nscripts/push-build.sh cargo fmt --check  # any cargo subcommand + args\nscripts/push-build.sh smoke              # build + run the UDP smoke test\nscripts/push-build.sh shell              # rsync + drop into an ssh session\n```\n\nSee [docs/architecture.md](docs/architecture.md) for how it works,\n[SECURITY.md](SECURITY.md) for what it can and cannot see, and\n[DEMO.md](DEMO.md) for a scripted end-to-end run with verbatim output\nfrom each parser (DNS, HTTP, TLS, Redis, MQTT, CoAP, SQLite uprobes).\n\n## What it decodes today\n\n**52 wire-format parsers** plus five overlay classifiers (gRPC,\nGraphQL, NTLM, LLM-API, Socket.IO) span web, databases, messaging,\nmail, directory, telephony, SMS / carrier, media-streaming,\nremote-access, operational-technology, IoT / constrained devices,\nfile-sharing, VPN, AAA / network-management, and legacy chat.\n\n### Web + APIs\n\n| Protocol | Notes |\n|---|---|\n| HTTP/1.x | Requests, responses, headers, bodies, chunked + range reassembly |\n| HTTP/2 | HPACK, per-stream framing |\n| gRPC | On HTTP/2: service, method, status; body decode with `--proto` |\n| GraphQL | On HTTP: operation name + root field (query / mutation / subscription) |\n| LLM APIs | OpenAI / Anthropic (incl. `/v1/messages/count_tokens` Claude Code uses) / Gemini / Azure / Bedrock / Ollama / OpenAI-compat — provider, endpoint, model, streaming-mode, prompt-bytes |\n| NTLM | Type 1 / 2 / 3 (Negotiate / Challenge / Authenticate); user + domain + workstation in cleartext; works inside HTTP `Authorization: NTLM/Negotiate`, SMB SessionSetup, LDAP SASL GSS-SPNEGO |\n| WebSocket | RFC 6455 frames; follows HTTP/1.1 `101 Upgrade` handshake |\n| Socket.IO / Engine.IO | Event name, namespace, JSON args, ack IDs |\n| TLS 1.0-1.3 | ClientHello / ServerHello SNI + ALPN + cipher-suite inspection; ServerHello warnings for legacy versions, NULL / EXPORT / RC4 / 3DES / CBC-SHA1 / anon-DH ciphers |\n\n### Databases\n\n| Protocol | Notes |\n|---|---|\n| Postgres | Startup, Simple / Extended query, bind parameters |\n| MySQL | COM_QUERY, COM_STMT_PREPARE/EXECUTE |\n| MongoDB wire | `OP_MSG`, `OP_QUERY`, BSON decode |\n| Redis | RESP2 + RESP3 |\n| Cassandra CQL | Opcodes, frame headers, query strings |\n| Memcached | ASCII + binary protocols |\n| Oracle TNS | CONNECT descriptor, SERVICE_NAME / SID / PROGRAM / USER |\n| MS SQL Server TDS | PreLogin / Login7 (user / server / app / db), batch / RPC |\n\n### Messaging + streaming + IoT\n\n| Protocol | Notes |\n|---|---|\n| Kafka wire | Produce / Fetch / Metadata / OffsetCommit, API 0-12 |\n| AMQP 0.9.1 (RabbitMQ) | Full class/method table, basic.publish routing-key + exchange |\n| MQTT 3.1.1 / 5 | CONNECT / PUBLISH / SUBSCRIBE with topic + QoS |\n| CoAP | RFC 7252 on udp/5683; type, code, message-id, assembled Uri-Path / Uri-Query, Content-Format |\n| NATS | Text protocol: PUB / SUB / MSG / HPUB / HMSG / INFO |\n| STUN / TURN | WebRTC signalling, XOR-MAPPED-ADDRESS decode, SOFTWARE |\n\n### Mail + directory\n\n| Protocol | Notes |\n|---|---|\n| IMAP | Tagged command framing, LOGIN redacted |\n| POP3 | USER + PASS (password redacted) |\n| SMTP | HELO / EHLO / AUTH (credentials redacted) + MAIL FROM / RCPT TO |\n| LDAP | BER: BindRequest (password redacted), SearchRequest with scopes |\n| Kerberos v5 | AS-REQ / AS-REP / TGS-REQ / ... with realm + cname + sname |\n\n### Remote access + proxy + telephony\n\n| Protocol | Notes |\n|---|---|\n| SSH | Banner + software identification |\n| RDP | X.224 ConnectionRequest, mstshash= username leak, TLS/CredSSP negotiation |\n| Telnet | IAC option negotiation + cleartext text extraction |\n| FTP | USER / PASS (redacted) / RETR / STOR / MLSD + reply codes |\n| SOCKS4 / SOCKS5 | CONNECT / BIND / UDP-ASSOCIATE with DOMAIN / IPv4 / IPv6 |\n| SIP | INVITE / REGISTER / ... with Call-ID + Via + From/To + User-Agent |\n| RTSP | DESCRIBE / SETUP / PLAY / TEARDOWN on IP cameras + media servers |\n| SMPP | SMS peer-to-peer: bind_* (system_id + redacted password), submit_sm |\n| IRC | PASS (redacted) / NICK / USER / JOIN / PRIVMSG / numeric replies |\n\n### Operational-technology (ICS / SCADA / building automation)\n\n| Protocol | Notes |\n|---|---|\n| Modbus/TCP | Function codes on tcp/502 |\n| Siemens S7comm | TPKT + COTP + S7; ROSCTR + function-code decode |\n| EtherNet/IP + CIP | ODVA encapsulation (tcp/44818 + 2222); session + command decode |\n| DNP3 | IEEE 1815 link-layer framing (tcp/20000) |\n| IEC-104 | Telecontrol APDU: I / S / U frames + ASDU TypeID catalogue |\n| OPC-UA | IEC 62541-6 §7.1.2 binary framing (tcp/4840) |\n| BACnet/IP | BVLC + NPDU + APDU; readProperty / writeProperty / Who-Is / I-Am |\n\n### Infrastructure + auth + management\n\n| Protocol | Notes |\n|---|---|\n| DNS | Questions + answers over tcp/udp 53 |\n| DHCP | Op / transaction / chaddr MAC / options (Host Name, Vendor Class) |\n| TFTP | RRQ / WRQ / DATA / ACK / ERROR / OACK with options |\n| NTP | Full 48-byte header; stratum / mode / ref_id (GPS, PPS, LOCL) |\n| RADIUS | Access-Request / Accept / Reject; User-Name + Called/Calling-Station |\n| TACACS+ | AUTHEN / AUTHOR / ACCT; flags incl. UNENCRYPTED warning |\n| SNMP v1/v2c | Version + community string + PDU type + first OID |\n| Syslog | RFC 3164 + RFC 5424 + RFC 6587 octet-counted framing |\n| SSDP | UDP discovery (mDNS-adjacent) |\n\n### File sharing + VPN\n\n| Protocol | Notes |\n|---|---|\n| SMB2 / SMB3 | TreeConnect share path, Create filename (UCS-2LE), NT_STATUS names |\n| NFS / ONC-RPC | Record-marker framing; NFSv3 / MOUNT / PORTMAP / NLM program + procedure decode |\n| WireGuard | HandshakeInit / Response / CookieReply / TransportData; ephemeral-key preview |\n\n### Library uprobes (local, not on the wire)\n\n| Library | Hook |\n|---|---|\n| libsqlite3 | `sqlite3_prepare_v2` + `sqlite3_exec` — full SQL text + db handle |\n\n### TLS runtimes lifted for plaintext\n\n| Runtime | Hook | Status |\n|---|---|---|\n| OpenSSL / libssl | `SSL_{read,write,read_ex,write_ex}` uprobes | shipped |\n| BoringSSL | same symbols as libssl | shipped |\n| GnuTLS | `gnutls_record_{send,recv}` uprobes | shipped |\n| QUIC v1 | client Initial decrypted with DCID-derived keys (RFC 9001); SNI + ALPN recovered | shipped (1-RTT deferred) |\n| NSS | `PR_Read` / `PR_Write` + `ssl3_SendPlainText` uprobes | planned |\n| Go `crypto/tls` | per-binary symbol scan → `crypto/tls.(*Conn).{R,W}` | planned |\n| Rust `rustls` | per-binary symbol scan → `rustls::Connection` plaintext path | planned |\n\nEach connection carries a protocol state machine that can *upgrade itself*:\nHTTP/1.1 → WebSocket → Socket.IO (on event frames), HTTP/2 → gRPC\n(on `application/grpc`), and any TCP → TLS (on a ClientHello record).\nNothing you configure.\n\nBoth transports captured in both directions — **TCP** via\n`tcp_sendmsg` / `tcp_recvmsg` kprobes and **UDP** via `udp_sendmsg`\n+ `udp_recvmsg` / `udp_recvmsg_ret` (IPv4 + IPv6 — dst / src address\n+ port read off `struct sock` with a `msg-\u003emsg_name` fallback for\nunconnected sockets).\n\nDeferred to v0.2: NSS, Go `crypto/tls`, Rust `rustls`, Java JSSE,\nQUIC 1-RTT payload (needs the TLS master secret shannon never sees).\n\n### Process identity\n\nEvery event also carries the originating process — pid, comm, uid/gid,\nand the resolved container or pod. Cgroup paths are walked at startup\nand refreshed periodically; Docker, containerd, CRI-O, Podman and\nplain systemd units all decode to a friendly name. On Kubernetes\nnodes, `/var/log/pods/` is parsed so you see `payments/billing-7d8c`\ninstead of a UID hex string, and container entries inherit the pod\ncontext as `payments/billing-7d8c/\u003cshort-id\u003e`.\n\n## Privacy\n\n`shannon` sees plaintext payloads. Redaction is **on by default** (`--redact auto`)\nand strips `Authorization`, `Cookie`, `Set-Cookie`, and query-string params matching\n`*token*|*password*|*secret*|api_key`. Run `shannon trace --redact strict` to strip\nall headers and bodies. You opt into visibility with `--redact off`; never the\nother way around. See [SECURITY.md](SECURITY.md).\n\n## Use cases\n\n- **Incident response**. \"Why is the billing service slow?\" → `sudo shannon\n  top --sort p99 --group-by endpoint` on the affected node. No redeploy,\n  no sidecar, no code change.\n- **Debugging microservice calls**. See exact HTTP requests one service is\n  making to another, including headers and body, even over TLS, without\n  touching either service's code.\n- **Tracing production bugs that don't reproduce locally**. Capture real\n  traffic with `shannon record -o live.jsonl.zst`, ship the file to a\n  dev box, replay with `shannon trace --replay`.\n- **Security posture audits**. Which processes talk to which external\n  IPs, and what credentials flow? `shannon trace --peer 0.0.0.0/0\n  --redact off --protocol http` (use responsibly — see disclaimer).\n- **Understanding a black-box binary**. Point shannon at a pid with\n  `shannon trace -p $(pidof mysterybin)` and watch it talk.\n- **CI / integration tests**. Record a baseline of service calls, then\n  assert behavioural invariants in CI.\n- **Capacity planning**. `shannon analyze capture.jsonl.zst` gives\n  per-endpoint RPS and latency distributions from a representative\n  recording.\n\n## Disclaimer\n\nThis is **research-grade software**. It installs eBPF programs into the\nLinux kernel and reads plaintext bytes from every TCP socket on the host\nit runs on.\n\n- **No warranty.** shannon is provided \"as is\". The authors and\n  contributors take no liability for any malfunction, system instability,\n  data loss, damage, or loss of business arising from its use.\n- **No fitness for any particular purpose** — including production,\n  compliance, or legal use — is implied or guaranteed.\n- **You are responsible for how you use it.** shannon is a tool; the\n  operator decides what to point it at. Using it to observe traffic\n  belonging to parties who have not authorised such observation may be\n  illegal in your jurisdiction (wiretap, privacy, data-protection laws).\n  The authors take no responsibility for misuse, abuse, or use in\n  violation of any law, contract, or policy.\n- **Privilege boundary.** Running shannon is equivalent to running as\n  root — treat it accordingly. Do not deploy without understanding\n  [SECURITY.md](SECURITY.md).\n\nUse only on systems you own or have explicit written permission to\nobserve. Respect the privacy of users whose traffic crosses those\nsystems.\n\n## License\n\nLicensed under either of [Apache 2.0](LICENSE-APACHE) or [MIT](LICENSE-MIT) at\nyour option. Unless you explicitly state otherwise, any contribution intentionally\nsubmitted for inclusion in the work by you, as defined in the Apache-2.0 license,\nshall be dual licensed as above, without any additional terms or conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatanoisetv%2Fshannon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatanoisetv%2Fshannon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatanoisetv%2Fshannon/lists"}