{"id":50496627,"url":"https://github.com/anthropics/defending-code-reference-harness","last_synced_at":"2026-06-02T08:00:47.385Z","repository":{"id":361915530,"uuid":"1246839757","full_name":"anthropics/defending-code-reference-harness","owner":"anthropics","description":"Skills for threat modeling, scanning, triage, patching, plus an autonomous scanning harness you can /customize","archived":false,"fork":false,"pushed_at":"2026-06-01T20:40:25.000Z","size":815,"stargazers_count":133,"open_issues_count":0,"forks_count":14,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-01T21:10:30.975Z","etag":null,"topics":["security"],"latest_commit_sha":null,"homepage":"https://claude.com/blog/using-llms-to-secure-source-code","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anthropics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"docs/security.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-22T16:00:56.000Z","updated_at":"2026-06-01T20:32:50.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/anthropics/defending-code-reference-harness","commit_stats":null,"previous_names":["anthropics/defending-code-reference-harness"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/anthropics/defending-code-reference-harness","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anthropics%2Fdefending-code-reference-harness","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anthropics%2Fdefending-code-reference-harness/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anthropics%2Fdefending-code-reference-harness/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anthropics%2Fdefending-code-reference-harness/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anthropics","download_url":"https://codeload.github.com/anthropics/defending-code-reference-harness/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anthropics%2Fdefending-code-reference-harness/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33812204,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-02T02:00:07.132Z","response_time":109,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["security"],"created_at":"2026-06-02T08:00:46.626Z","updated_at":"2026-06-02T08:00:47.373Z","avatar_url":"https://github.com/anthropics.png","language":"Python","funding_links":[],"categories":["Python","Official Resources"],"sub_categories":[],"readme":"# Defending Code Reference Harness\n\nA reference implementation for autonomous vulnerability discovery and\nremediation with Claude, based on our learnings from [partnering with security\nteams at several organizations](https://www.anthropic.com/glasswing)\nsince launching Claude Mythos Preview. For a write up of these learnings along with\nbest practices, see the [accompanying blog post](https://claude.com/blog/using-llms-to-secure-source-code)\n(also available in [`blog-post.md`](docs/blog-post.md)). For a lightweight SDK-only \nwalkthrough of the same recon → find → triage → report → patch loop, see the \n[companion cookbook](https://platform.claude.com/cookbook/claude-agent-sdk-06-the-vulnerability-detection-agent).\n\nThis repo is not maintained and is not accepting contributions.\n\n\u003e 🔒 **Want a managed option?** Anthropic offers\n\u003e [Claude Security](https://claude.com/product/claude-security), a hosted product\n\u003e that finds and fixes vulnerabilities in your source code across multiple\n\u003e projects. Claude Security scans your repository for vulnerabilities,\n\u003e applies a multi-stage verification pipeline to reduce false positives, and\n\u003e lets you manage findings through their lifecycle: triage, fix validation,\n\u003e and rapid fix generation.\n\u003e\n\u003e This repository is an open-source reference implementation based on general\n\u003e best practices for finding vulnerabilities using Claude. You can use it to\n\u003e build your own vulnerability finding pipeline, customize the logic, and it\n\u003e can be used with whatever access you have to Claude APIs (including\n\u003e Bedrock, Vertex, or Azure).\n\n## Contents\n\n- **Claude Code skills**: `/quickstart`, `/threat-model`, `/vuln-scan`,\n  `/triage`, `/patch`, `/customize`: interactive scoping, scanning, triage,\n  and patching. Open this repo in Claude Code and run `/quickstart` to get\n  oriented.\n- **`harness/`**: the autonomous reference pipeline (recon → find → verify\n  → report → patch), configured for finding C/C++ memory vulnerabilities\n  using Docker and ASAN. This harness is a **reference, not a product**. \n  The general shape, prompts, and sandboxing are reusable, but the harness\n  will not work on every codebase out of the box. Run `/customize` to port it \n  to your language, detector, or vuln class.\n\n\u003e ⚠️ **Security:** `/quickstart`, `/threat-model`, `/vuln-scan`, and `/triage`\n\u003e only read and write files. Running `/patch` on static findings (`TRIAGE.json`\n\u003e or `VULN-FINDINGS.json`) is likewise read- and write-only. `/customize` edits\n\u003e the harness code and runs validation commands. Any of these skills are safe to\n\u003e run unsandboxed, as long as you review and approve each tool use in Claude Code.\n\u003e The autonomous reference pipeline (including `/patch` on pipeline results)\n\u003e **executes target code**, so it refuses to run outside of a gVisor sandbox\n\u003e unless explicitly overridden. To get set up, run `scripts/setup_sandbox.sh` once,\n\u003e then invoke the pipeline via `bin/vp-sandboxed`. See [docs/security.md](docs/security.md)\n\u003e and [docs/agent-sandbox.md](docs/agent-sandbox.md) for more details.\n\n## Getting Started\n\n```bash\ngit clone https://github.com/anthropics/defending-code-reference-harness\ncd defending-code-reference-harness\nclaude\n\n# 30-sec intro + guided first run on the canary target\n\u003e /quickstart\n\n\u003e /quickstart how do I port the pipeline to Java?\n\u003e /quickstart how do I triage all these bugs?\n```\n\n## Further Reading\n\n- [**Blog Post**](docs/blog-post.md) · The accompanying blog post with learnings + best practices\n- [**Pipeline**](docs/pipeline.md) · How it works: diagram, stages, CLI flags\n- [**Security**](docs/security.md) · Sandboxing, what not to mount\n- [**Agent sandbox**](docs/agent-sandbox.md) · gVisor isolation + egress allowlist for every agent\n- [**Customize**](docs/customizing.md) · Port to my stack; which files change and why\n- [**Patching**](docs/patching.md) · Generate and verify fixes for verified crashes\n- [**Troubleshooting**](docs/troubleshooting.md) · Duplicates, rate limits, subagent model pinning\n- [**Safeguards**](https://support.claude.com/en/articles/14604842-real-time-cyber-safeguards-on-claude) · Block for dangerous cyber work\n\n---\n\n## Ramp Up\n\nThe most successful security teams we've partnered with are those \nthat have gotten hands-on the fastest. Though it's tempting to \nspend months designing the perfect pipeline, we recommend starting\nsmall on Day 1 and building from there as learnings come. The\nsteps below follow that pattern and set an ambitious (but reasonable)\npace based on what we've seen.\n\n|                                                                                     |              |                                                              |\n|-------------------------------------------------------------------------------------|--------------|--------------------------------------------------------------|\n| [Step 1](#step-1-day-1-build-a-threat-model-and-run-your-first-static-scan--triage) | **Day 1**    | Build a threat model and run your first static scan + triage |\n| [Step 2](#step-2-day-2-run-the-reference-pipeline-on-a-cc-library)                  | **Day 2**    | Run the reference pipeline on a C/C++ library                | \n| [Step 3](#step-3-days-3-5-customize-the-pipeline-for-your-target)                   | **Days 3-5** | Customize the pipeline for your target                       |\n| [Step 4](#step-4-week-2-start-autonomous-scanning-triage-and-patching)              | **Week 2**   | Start autonomous scanning, triage, and patching              | \n\n### Step 1 (Day 1): Build a threat model and run your first static scan + triage\n\nDay 1 is focused on seeing the whole loop end-to-end. Using only the \ninteractive skills, you'll build a threat model, run a static scan scoped \nby it, triage what comes back, and draft candidate fixes. You'll finish \nthe day with a threat model, a ranked list of static findings, and candidate \npatches.\n\nThe relevant skills **only read and write files** in your repo. As long as you \nrun Claude Code interactively and approve each tool use, no sandbox is needed.\n\n```bash\n# Pin every subagent to the model you want\nexport CLAUDE_CODE_SUBAGENT_MODEL=\u003cmodel-id\u003e\nclaude\n\n# 0. intro + guided first run\n\u003e /quickstart\n\n# 1. Build a threat model (aim before you shoot)\n\u003e /threat-model bootstrap targets/canary\n\n# 2. Run a static scan, scoped by that threat model\n\u003e /vuln-scan targets/canary\n\n# 3. Verify, dedupe, and rank what came back\n\u003e /triage targets/canary/VULN-FINDINGS.json\n\n# 4. Generate candidate fixes for the verified findings\n\u003e /patch ./TRIAGE.json --repo targets/canary\n```\n\nThis flow produces `THREAT_MODEL.md`, `VULN-FINDINGS.{json,md}`, \n`TRIAGE.{json,md}`, and `PATCHES/`.\n\nThe vulnerability candidates produced in Step 1 come from Claude's static \nreview of the source (nothing is built or run), so expect more false positives on \nany non-canary targets. In Step 2, you'll produce *execution-verified* findings.\n\n\u003e **Note:** on the canary target, `/triage` may dismiss the scan's findings\n\u003e as false positives. `entry.c` announces itself as deliberately vulnerable\n\u003e demo code, and `/triage` correctly excludes bugs in test / fixture code.\n\u003e To see the full confirm / dedupe / false positive flow, run it on the\n\u003e curated fixture instead (`/triage .claude/skills/triage/fixtures/canary-findings.json\n\u003e --repo targets/canary`) or point the Step 1 skills at your own code.\n\n### Step 2 (Day 2): Run the reference pipeline on a C/C++ library\n\nOn Day 2, you'll move from interactive skills to your first autonomous\nrun using the reference pipeline. You'll run the full recon → find → \nverify → report loop in your environment on a known-vulnerable open-source\nlibrary, then generate a candidate patch for what it finds. You'll finish\nwith a set of reproducible crashes, exploitability reports, and candidate patches,\nalong with a feel for how the pipeline works.\n\nRunning the pipeline is simple:\n\n```bash\n# One-time setup\npython3 -m venv .venv \u0026\u0026 .venv/bin/pip install -e .\n./scripts/setup_sandbox.sh   # installs gVisor, builds the agent images, and verifies isolation; note: requires Docker\nexport ANTHROPIC_API_KEY=sk-ant-...   # or CLAUDE_CODE_OAUTH_TOKEN; the pipeline requires one in env\n\n# Run the recon → find → verify → report loop\nbin/vp-sandboxed run drlibs --model \u003cmodel-id\u003e --runs 3 --parallel --stream --auto-focus\n# Generate a candidate patch for each finding\nbin/vp-sandboxed patch results/drlibs/\u003ctimestamp\u003e/ --model \u003cmodel-id\u003e\n\n# Or, ask Claude Code to launch the pipeline and watch the run for you\nclaude\n\u003e run the pipeline on drlibs and explain findings as they come\n```\n\nResults from the loop land in a `results/drlibs/\u003ctimestamp\u003e/` directory. With \nthe `--stream` flag, the first report will appear in minutes under `reports/bug_NN/`.\n\n\u003e ⚠️ **`run` spawns autonomous agents.** The pipeline runs each agent\n\u003e inside a gVisor container with egress restricted to the Claude API.\n\u003e Agent-spawning subcommands refuse to start outside it unless explicitly \n\u003e overridden. For more information, see [docs/security.md](docs/security.md)\n\u003e and [docs/agent-sandbox.md](docs/agent-sandbox.md).\n\nUnder the hood, the pipeline walks through seven stages:\n\n1. **Build**: Compiles the target into a Docker image with ASAN (the memory\nerror detector for C and C++). The pipeline builds this image automatically\non first run using the target's `Dockerfile`.\n2. **Recon**: A lightweight agent reads the source inside a network-isolated\ncontainer and proposes a partition, i.e., *\"here are N distinct input-parsing \nsubsystems worth attacking separately\"*, so that parallel find agents explore\ndifferent areas instead of converging on the same bug. Without the `--auto-focus`\nflag, the pipeline uses the `focus_areas` list from the target's `config.yaml`.\n3. **Find**: N agents run in parallel, each in its own isolated container.\nEach agent reads the source, crafts malformed inputs, and runs the ASAN\nbinary until a given input produces a crash 3 out of 3 times.\n4. **Verify**: A separate grader agent reproduces each crash in a fresh\ncontainer that the find agent hasn't touched. The only thing that crosses over\nfrom the find agent to the grader is the proof of concept it produced.\n5. **Dedupe**: A judge agent compares verified crashes against bugs already\nreported and decides whether each is a new bug, a better example of a known\nbug, or a duplicate to skip.\n6. **Report**: A report agent writes a structured exploitability analysis per\nunique bug, including details on primitive class, reachability, escalation\npath, and severity.\n7. **Patch** (the separate patch command above): A patch agent writes a proposed\nfix, and a grader agent confirms that the new code builds, that the original \nproof of concept input no longer crashes, that the target's test suite still \npasses, and that a fresh find agent can't find a way around the fix.\n\nFor more details, see [docs/pipeline.md](docs/pipeline.md).\n\n### Step 3 (Days 3-5): Customize the pipeline for your target\n\nOn Days 3-5, you'll customize the harness for your own target. First, you'll\npoint the Step 1 skills at your code, then you'll use `/customize` to port the\npipeline to your stack. By the end of the week, you'll have a `targets/\u003cyour-service\u003e/`\ndirectory that the pipeline can run against, validated with a single smoke run\nof the pipeline, and ready to scale up in Step 4.\n\nWhile the reference pipeline is designed for finding memory vulnerabilities in C and C++\ncode, its shape is generic. Porting it to a new vuln class or language just means\nanswering the following questions for your target stack:\n\n| Question                                | C/C++ Reference                   | Your target (examples)                         |\n|-----------------------------------------|-----------------------------------|------------------------------------------------|\n| What signals a finding?                 | ASAN crash signature              | exception / canary file / DNS callback         |\n| What does a proof of concept look like? | crashing input file               | HTTP request sequence / tx list / test harness |\n| How is the target built and run?        | `Dockerfile` (using clang + ASAN) | your language's build in a container           |\n\nBefore customizing, point the Step 1 skills at your own code. As a reminder,\nthey're read- and write-only, so they can run unsandboxed.\n\n```bash\nclaude\n\n\u003e /quickstart how do I customize this for ~/code/my-service?\n\n\u003e /threat-model bootstrap-then-interview ~/code/my-service\n\u003e /vuln-scan ~/code/my-service\n\u003e /triage ~/code/my-service/VULN-FINDINGS.json --repo ~/code/my-service\n```\n\nThen, use the artifacts produced by those skills in the `/customize` skill, \nwhich modifies the harness for your codebase.\n\n```bash\n\u003e /customize use ~/code/my-service/{THREAT_MODEL.md,VULN-FINDINGS.json} and ./TRIAGE.md\n```\n\nWhen `/customize` is done, you'll have a `targets/my-service/` directory \nset up. Validate it with a smoke run of the pipeline before scaling up.\n\n```bash\nbin/vp-sandboxed run my-service --model \u003cmodel-id\u003e --runs 1\n```\n\nFor more details, see [docs/customizing.md](docs/customizing.md).\n\n### Step 4 (Week 2): Start autonomous scanning, triage, and patching\n\nIn Week 2, you'll use the pipeline you customized in Step 3 on your own\ntargets, adding an *outer* loop to the inner pipeline loop - run multiple\npipeline scans, triage the findings from across those runs, patch based\non prioritization, and repeat.\n\n```bash\n# Scan - run a wave of parallel runs against your target\nbin/vp-sandboxed run my-service --model \u003cmodel-id\u003e --runs 5 --parallel --stream --auto-focus\n\n# Triage - dedupe and rank every finding across all waves using your threat model\n\u003e /triage results/my-service/ --repo ~/code/my-service --auto --votes 5\n\n# Patch - generate and validate fixes, starting with what triage ranked the highest\n\u003e /patch results/my-service/\u003ctimestamp\u003e/ --model \u003cmodel-id\u003e\n```\n\n\u003e ⚠️ Follow the same sandboxing guidelines as in \n\u003e [Step 2](#step-2-day-2-run-the-reference-pipeline-on-a-cc-library)\n\nA given pipeline run already verifies and deduplicates its own findings.\n`/triage` works across many pipeline runs. When pointed at the `results/`\ndirectory, it collapses duplicates across all runs (and any static findings\nfrom `/vuln-scan` if present), recalibrates severity ratings against your\nthreat model, and attempts to route every finding to the component owner.\n\nWhen possible, patching findings quickly helps keep the outer loop as \nproductive as possible. When findings are fixed, the model can't re-find\nthem, and instead will surface net new, typically deeper issues. As you run\nmore pipeline waves, the number of findings will likely go down, but the\ncomplexity will likely also go up. If quick patching isn't possible, even\njust recording prior findings in the target's `known_bugs` can help steer\nfuture runs toward newer bugs.\n\nAutonomous triage and patching are still open issues, and this reference\nharness doesn't fully solve them. The verification strategies in `/patch`\nhelp raise the bar, but severity and prioritization are ultimately\njudgments about your environment, and verified patches are not always\nupstreamable. Many partners have reported these steps as their current\nbottlenecks, and you should budget real engineering time for them.\n\nFor more details, see [docs/triage.md](docs/triage.md) and \n[docs/patching.md](docs/patching.md).\n\n## Looking Forward\n\nAfter the initial ramp up, the teams we've worked with have tended to invest in a\nfew directions:\n\n1. Reviewing all their internal repos and key open-source dependencies,\nranking which are the most important to scan (e.g., based on their exposure, \nhistory of CVEs, business-criticality), then working through scanning the\nlist in priority order.\n2. Setting up bespoke infrastructure for scanning to move scans off of laptops\nor one-off VMs. The most successful teams resist the urge to build the perfect\nscanning platform before scaling up.\n3. Incorporating scans into their SDLC. Some teams have set up recurring scans \n(e.g., daily, weekly) or have added scanning into their CI pipelines.\n4. Testing and experimenting with the models to find what works best for them.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanthropics%2Fdefending-code-reference-harness","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanthropics%2Fdefending-code-reference-harness","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanthropics%2Fdefending-code-reference-harness/lists"}