{"id":51011345,"url":"https://github.com/basilisk-labs/agentplane-bitgn-adapter","last_synced_at":"2026-06-21T03:02:51.371Z","repository":{"id":357860866,"uuid":"1238864793","full_name":"basilisk-labs/agentplane-bitgn-adapter","owner":"basilisk-labs","description":"BitGN adapter for benchmarking AgentPlane-backed Codex execution","archived":false,"fork":false,"pushed_at":"2026-05-14T15:06:40.000Z","size":50,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-14T16:41:58.202Z","etag":null,"topics":["agentplane","agents","benchmark","bitgn","codex"],"latest_commit_sha":null,"homepage":"https://bitgn.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/basilisk-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-14T14:25:45.000Z","updated_at":"2026-05-14T15:06:45.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/basilisk-labs/agentplane-bitgn-adapter","commit_stats":null,"previous_names":["basilisk-labs/agentplane-bitgn-adapter"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/basilisk-labs/agentplane-bitgn-adapter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basilisk-labs%2Fagentplane-bitgn-adapter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basilisk-labs%2Fagentplane-bitgn-adapter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basilisk-labs%2Fagentplane-bitgn-adapter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basilisk-labs%2Fagentplane-bitgn-adapter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/basilisk-labs","download_url":"https://codeload.github.com/basilisk-labs/agentplane-bitgn-adapter/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/basilisk-labs%2Fagentplane-bitgn-adapter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34592057,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-21T02:00:05.568Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentplane","agents","benchmark","bitgn","codex"],"created_at":"2026-06-21T03:02:48.232Z","updated_at":"2026-06-21T03:02:51.357Z","avatar_url":"https://github.com/basilisk-labs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AgentPlane BitGN Adapter\n\n[![CI](https://github.com/basilisk-labs/agentplane-bitgn-adapter/actions/workflows/ci.yml/badge.svg)](https://github.com/basilisk-labs/agentplane-bitgn-adapter/actions/workflows/ci.yml)\n\nAdapter scaffold for running AgentPlane-backed Codex execution against BitGN\nbenchmarks.\n\nAgentPlane is not submitted as a model. It is used as a control-plane profile\naround an executor:\n\n- benchmark runtime: BitGN PCM or ECOM\n- executor: Codex CLI\n- control layer: policy, step loop, proof bundle, score-detail capture\n\n## Status\n\nExperimental. Current proven coverage is deliberately narrow:\n\n- `bitgn/sandbox t01`: pass, score `1.00`.\n- `bitgn/pac1-dev t01`: pass, score `1.00`.\n- `bitgn/ecom1-dev t01`: pass, score `1.00`.\n\nAll non-`t01` PAC1 and ECOM1 tasks are not passing in current evidence and must\nbe treated as failing/unsupported until a live run proves otherwise. This repo\nis not leaderboard-ready.\n\n## Why this exists\n\nBitGN evaluates observable agent behavior: runtime tool calls, files, task\nstate, side effects, outcome codes, compliance, and security posture. That is\nthe same surface where AgentPlane can add value: bounded policy, traceability,\nexplicit outcomes, and failure evidence.\n\nThe near-term goal is not \"AgentPlane beats everyone\". The useful public claim\nis narrower:\n\n\u003e AgentPlane can wrap a strong executor, preserve BitGN benchmark validity, and\n\u003e produce auditable evidence for why trials passed or failed.\n\n## Install\n\n```bash\nmake sync\n```\n\nInstall BitGN SDK dependencies from the same Buf registry used by the upstream\nsamples:\n\n```bash\nmake sync-bitgn\n```\n\nThe SDK currently tracks Python 3.14 in the sample agents, so the Make targets\ncreate a Python 3.14 uv environment.\n\n## Authentication\n\nCodex can use ChatGPT subscription auth:\n\n```bash\ncodex login\ncodex login status\n```\n\nThat path is useful for local smoke runs because the adapter invokes `codex\nexec`. For reproducible public runs, API-key auth is still cleaner because it is\neasier to document and recreate in CI or another machine.\n\nBitGN official runs still need:\n\n```bash\nexport BITGN_API_KEY=\"...\"\n```\n\n## PAC1 smoke\n\n```bash\ncp .env.example .env.local\n$EDITOR .env.local\nmake oauth\nmake sandbox\n```\n\n`scripts/bitgn_smoke.sh` loads `.env` and then `.env.local`; keep secrets in\none of those ignored files, not in committed config.\n\nSandbox is the first end-to-end check because it does not require a BitGN\nPlatform key. PAC1 is the next check:\n\n```bash\nmake pac1\n```\n\n## ECOM smoke\n\nSet:\n\n```bash\nBENCHMARK_ID=bitgn/ecom1-dev\nBITGN_RUNTIME=ecom\n```\n\nThen run a single task:\n\n```bash\nmake ecom\n```\n\n## Proof bundle\n\nEach trial writes:\n\n```text\n.agentplane-bitgn/\u003cbenchmark-id\u003e/\u003cruntime\u003e/\u003ctask-id\u003e/\u003ctrial-id\u003e/\n  AGENTS.md\n  proof.json\n```\n\nThe proof bundle captures:\n\n- benchmark id and runtime\n- model id\n- task id and trial id\n- each JSON tool command requested by Codex\n- runtime observations, truncated for readability\n- final status\n\n## Documentation\n\n- [Runbook](docs/runbook.md)\n- [Test strategy](docs/test-strategy.md)\n- [Coverage matrix](docs/coverage.md)\n- [Leaderboard plan](docs/leaderboard.md)\n- [Evidence report template](docs/evidence-template.md)\n- [Cost notes](docs/cost.md)\n- [OAuth notes](docs/oauth.md)\n- [Smoke results](docs/test-results.md)\n\n## Leaderboard realism\n\nPAC1 live already has multiple 104/104 runs. A naive scaffold is unlikely to\nstand out there. The best AgentPlane path is:\n\n1. Use PAC1 DEV to harden outcome selection, grounding refs, structured writes,\n   and injection refusal.\n2. Mine `score_detail` into regression cases.\n3. Move to ECOM1, where policy books, payment state, SQL, fraud controls, and\n   audit trails are closer to AgentPlane's control-plane strengths.\n4. Publish a proof-backed run rather than only a score screenshot.\n\n## Integrity rules\n\nDo not:\n\n- fetch benchmark solutions from the internet;\n- inspect hidden graders or oracle solutions;\n- alter BitGN scoring, task sets, or runtime contracts;\n- inject task-specific hints into the adapter policy;\n- claim leaderboard readiness without a reproducible run id and proof bundle.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbasilisk-labs%2Fagentplane-bitgn-adapter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbasilisk-labs%2Fagentplane-bitgn-adapter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbasilisk-labs%2Fagentplane-bitgn-adapter/lists"}