{"id":50135056,"url":"https://github.com/evilsocket/audit","last_synced_at":"2026-05-24T22:00:47.384Z","repository":{"id":358877367,"uuid":"1242867671","full_name":"evilsocket/audit","owner":"evilsocket","description":"An 8-stage vulnerability-discovery agent.","archived":false,"fork":false,"pushed_at":"2026-05-23T11:29:28.000Z","size":204,"stargazers_count":443,"open_issues_count":0,"forks_count":62,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-23T21:25:36.745Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/evilsocket.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":"audit/__init__.py","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-18T20:42:44.000Z","updated_at":"2026-05-23T19:32:48.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/evilsocket/audit","commit_stats":null,"previous_names":["evilsocket/audit"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/evilsocket/audit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evilsocket%2Faudit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evilsocket%2Faudit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evilsocket%2Faudit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evilsocket%2Faudit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/evilsocket","download_url":"https://codeload.github.com/evilsocket/audit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evilsocket%2Faudit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33452033,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-24T19:21:36.376Z","status":"ssl_error","status_checked_at":"2026-05-24T19:21:10.562Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-23T21:04:48.041Z","updated_at":"2026-05-24T22:00:47.335Z","avatar_url":"https://github.com/evilsocket.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# audit\n\nAn 8-stage vulnerability-discovery agent, driven by your **Claude Pro / Max\nsubscription** through the official Claude Code Agent SDK. Many narrow agents,\ndeliberate disagreement, and an explicit reachability gate.\n\nMIT-licensed. No API key needed if you already use `claude login`.\n\n## Origin\n\nThis project is a from-scratch reimplementation of the pipeline described in\nCloudflare's [Project Glasswing](https://blog.cloudflare.com/cyber-frontier-models/)\npost, which tested Anthropic's Mythos preview LLM against Cloudflare's own\ncodebase. The blog argues that real-world vulnerability discovery does **not**\ncome from asking one big model \"find bugs here\" — it comes from:\n\n1. **Many narrow agents** working in parallel on tightly-scoped questions\n   (\"Look for command injection in this specific function, with this trust\n   boundary above it\") rather than one exhaustive agent.\n2. **Deliberate disagreement** — a second agent, on a different model, that\n   tries to *disprove* the first agent's findings.\n3. **A reachability trace** as the gating step — most \"is this code buggy?\"\n   findings are noise unless an attacker-controlled input can actually reach\n   the sink from outside the system.\n4. **A feedback loop** so reachable bugs in one place automatically seed\n   hunts for the same pattern elsewhere.\n\nThis repo packages that pipeline into a runnable agent. The Cloudflare post\nshowed the architecture; this codebase ships the prompts, schemas, state\nstore, and orchestrator.\n\n## The 8 stages\n\n![Vulnerability discovery harness — 8 stages](https://raw.githubusercontent.com/evilsocket/audit/main/docs/pipeline.png)\n\n\u003csub\u003eDiagram from Cloudflare's [Project Glasswing](https://blog.cloudflare.com/cyber-frontier-models/) post, reproduced here for reference.\u003c/sub\u003e\n\n| # | Stage    | Default model | Purpose |\n|---|----------|---------------|---------|\n| 1 | Recon    | Opus 4.7  | Map the repo, emit narrowly-scoped Hunt tasks |\n| 2 | Hunt     | Sonnet 4.6 | One attack class per agent; compile/run PoCs |\n| 3 | Validate | Opus 4.7  | Adversarial re-read; tries to **disprove** (different model from Hunt) |\n| 4 | Gapfill  | Sonnet 4.6 | Re-queue under-covered areas |\n| 5 | Dedupe   | Sonnet 4.6 | Cluster findings by root cause |\n| 6 | Trace    | Opus 4.7  | Prove attacker-controlled input reaches the sink |\n| 7 | Feedback | Sonnet 4.6 | Turn reachable traces into new Hunt tasks |\n| 8 | Report   | Sonnet 4.6 | Schema-validated structured report |\n\nEach stage is one markdown prompt in `prompts/` + one JSON Schema in\n`schemas/`. The orchestrator passes the schema into the system prompt so\nevery output is shape-stable on the first try.\n\n## Quickstart\n\n```bash\n# 1. Install\npython -m venv .venv \u0026\u0026 source .venv/bin/activate\npip install -e .\n\n# 2. Auth (pick one)\n#    (a) Already logged in via claude login? You're done.\n#    (b) Or generate a 1-year OAuth token for CI / non-interactive use:\nclaude setup-token\necho \"CLAUDE_CODE_OAUTH_TOKEN=\u003cpaste\u003e\" \u003e .env\n\n# 3. Verify\naudit auth-check\n\n# 4. Run\naudit run --repo /path/to/target --run-id my-run\naudit status --run-id my-run\naudit report --run-id my-run --format md \u003e report.md\n```\n\nBy default the agent uses **subscription billing** via your Claude.ai\nlogin — it does **not** call the metered Anthropic API. The on-disk auth\nmodule scrubs `ANTHROPIC_API_KEY` from the environment so it can't\nsilently route around the OAuth flow.\n\n## Using a different model / provider\n\nThe auth module picks one of three modes, in this order:\n\n1. **LLM gateway** (OpenRouter, custom proxy, etc.) — when\n   `ANTHROPIC_BASE_URL` points away from `anthropic.com` AND\n   `ANTHROPIC_AUTH_TOKEN` is set. The gateway env is left intact;\n   only `ANTHROPIC_API_KEY` is scrubbed (it would otherwise outrank the\n   gateway token).\n2. **Subscription OAuth (headless)** — `CLAUDE_CODE_OAUTH_TOKEN` from\n   `claude setup-token`. Best for CI.\n3. **Subscription OAuth (interactive)** — `~/.claude/.credentials.json`\n   from `claude login`. Best for local dev.\n\n### OpenRouter\n\nOpenRouter exposes Claude-compatible Anthropic-API endpoints behind its\nown credit system; that lets you spend OpenRouter credits instead of an\nAnthropic subscription, and gives you access to Sonnet/Opus *and* other\nmodels through the same SDK path. See [OpenRouter's Agent SDK guide](https://openrouter.ai/docs/guides/community/anthropic-agent-sdk).\n\n```bash\nexport ANTHROPIC_BASE_URL=\"https://openrouter.ai/api\"\nexport ANTHROPIC_AUTH_TOKEN=\"$OPENROUTER_API_KEY\"\nexport ANTHROPIC_API_KEY=\"\"           # must be explicitly empty / unset\n# optional: pick a non-Anthropic model\nexport ANTHROPIC_MODEL=\"anthropic/claude-sonnet-4-6\"\n# or e.g.: ANTHROPIC_MODEL=\"openai/gpt-5\"\n#         ANTHROPIC_MODEL=\"google/gemini-2.5-pro\"\n#         ANTHROPIC_MODEL=\"qwen/qwen3-coder-480b\"\n\naudit auth-check                       # confirms \"using LLM gateway at https://openrouter.ai/api\"\naudit run --repo /path/to/target --run-id orun --max-cost-usd 30\n```\n\nCaveats:\n- Per-stage model overrides in `config/stages.yaml` are model **names**\n  (e.g. `claude-opus-4-7`); OpenRouter accepts slash-prefixed forms like\n  `anthropic/claude-opus-4-7`. Edit the YAML if you want different\n  providers per stage. Otherwise `ANTHROPIC_MODEL` forces every stage\n  onto one model.\n- Non-Claude models may not produce schema-compliant JSON as reliably.\n  The runner's schema-validation + repair turn still applies; quality\n  varies by model.\n- Tool-use semantics (Read/Grep/Glob/Bash) are part of the Claude Code\n  CLI, not the model — they work as long as the gateway speaks the\n  Anthropic Messages API.\n\n### Other gateways / cloud providers\n\nSame recipe — anything that exposes the Anthropic Messages API at a URL\n+ a bearer token works:\n\n```bash\nexport ANTHROPIC_BASE_URL=\"https://your-proxy.example.com\"\nexport ANTHROPIC_AUTH_TOKEN=\"$YOUR_TOKEN\"\nunset ANTHROPIC_API_KEY\n```\n\nFor Amazon Bedrock / Google Vertex / Microsoft Foundry, Claude Code has\nfirst-class env-var flags (`CLAUDE_CODE_USE_BEDROCK=1` etc.) that\noutrank everything else. See the [Claude Code auth docs](https://code.claude.com/docs/en/authentication).\n\n## Cost containment\n\nA real production codebase can produce 15-50 Hunt tasks and 25+ findings to\nvalidate. At default concurrency this gets expensive. Flags to keep it sane:\n\n```bash\naudit run --repo /path/to/target \\\n  --max-concurrency 1 \\           # one claude subprocess at a time\n  --max-recon-tasks 15 \\          # cap initial Hunt fanout\n  --max-cost-usd 30               # abort cleanly if exceeded\n```\n\nThe budget guard fires between *and* within stages — a per-task check in\nHunt cooperatively aborts rather than running 30 more tasks past the cap.\n\n## Live-target reproduction (optional)\n\nIf the target has a running deployment, point the agents at it. Hunt now\n**reproduces** each finding against the live service instead of compiling\na local PoC, Validate **rejects** findings that don't reproduce, and Trace\n**confirms** reachability with real HTTP round-trips. The static path\nremains available — these flags are opt-in.\n\n```bash\naudit run --repo /path/to/target --run-id live \\\n  --max-concurrency 1 --max-cost-usd 30 \\\n  --target-url http://server.local:8888 \\\n  --target-creds email=admin@system.com \\\n  --target-creds password=changechangeme\n```\n\nRules the agents follow when `--target-url` is set:\n- Network egress is restricted to that host + `127.0.0.1`. No other external\n  hosts.\n- A finding that doesn't reproduce against the live target is dropped or\n  rejected (depending on stage) — \"no fabrication\".\n- Credentials flow into every relevant stage's user_input as a dict.\n\n## Scope notes (optional)\n\nTargets often have intentionally-loose-by-design surfaces that aren't bugs\n(e.g. plaintext API keys when that's a feature, test-only Mailpit endpoints,\nanonymous-analytics ingest). Drop them in a text file and pass it in — the\nnotes are appended verbatim to every stage's user_input, and Recon / Hunt /\nValidate honor exclusions you list.\n\n```bash\naudit run --repo /path/to/target --scope-notes target_scope.md\n```\n\nExample `target_scope.md`:\n\n```markdown\n- Mailpit (port 1025) is test-only; ignore.\n- Plaintext API keys in the database are a required feature.\n- Don't flag rate-limit absence on anonymous /ping endpoints.\n- Only consider critical/high severity.\n```\n\n## Recon mines git history\n\nRecon greps the git history for past security patches\n(`CVE`, `sec:`, `fix.*auth`, `sanitize`, …) — patched files are hardened,\nbut **sibling files with the same idiom often aren't**. Findings get seeded\nagainst the unpatched copies. Adds zero cost on repos without that pattern;\ncatches real cross-component bugs on repos that have it.\n\n## Logic chains\n\nThe pipeline's default is one-attack-class-per-task (the Cloudflare paper's\nnarrow-scope rule). Recon can also emit `logic_chain` tasks for high-impact\nmulti-component paths (auth-bypass + IDOR + path-traversal that compose into\nRCE, etc.) — one chain per task, with the `scope_hint` naming the specific\nchain. This is the one allowed exception to single-attack-class scoping.\n\n## Layout\n\n```\nprompts/        8 stage prompts (markdown, loaded as system prompts)\nschemas/        9 JSON schemas — every agent output is validated\nconfig/         stages.yaml — model + concurrency + tool allowlist per stage\naudit/          Python package\n  auth.py       OAuth check + ANTHROPIC_API_KEY scrubbing\n  state.py      SQLite DAO (runs, tasks, findings, traces, dedupe, costs)\n  runner.py     claude-agent-sdk wrapper with schema validation + repair turn\n  orchestrator.py pipeline driver\n  stages/       one module per stage\nwork/           per-Hunt-task scratch dirs (sandbox for PoC compile/run)\nresults/        JSONL artifacts per stage + final report.json\nstate.db        SQLite (gitignored)\n```\n\n## Safety\n\nHunt agents have Bash and run inside per-task scratch dirs. They are **not**\nsandboxed at the OS level. Run the audit inside a disposable VM or container\nwhen you don't trust the target source — a target with malicious build\nscripts could otherwise execute on your host during PoC compilation.\n\nThe agent reads everything you `--add-dir`, including any `.env` or\n`secrets/` directories in the target. Outputs land in `results/\u003crun-id\u003e/`\nwhich is `.gitignore`d but **not** scrubbed of those reads.\n\n## License\n\n[MIT](LICENSE). Reuse freely. No warranty.\n\n## Acknowledgements\n\n- The pipeline design is from Cloudflare's [Project Glasswing](https://blog.cloudflare.com/cyber-frontier-models/)\n  blog post. The credit for the architecture goes there.\n- Built on the official [Claude Code Agent SDK](https://code.claude.com/docs/en/agent-sdk/overview).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevilsocket%2Faudit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fevilsocket%2Faudit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevilsocket%2Faudit/lists"}