{"id":51951705,"url":"https://github.com/dubsopenhub/dark-factory","last_synced_at":"2026-07-29T06:01:28.230Z","repository":{"id":340315186,"uuid":"1165154401","full_name":"DUBSOpenHub/dark-factory","owner":"DUBSOpenHub","description":"🏭 Agentic dark factory for AI building with sealed-envelope testing","archived":false,"fork":false,"pushed_at":"2026-07-28T21:19:17.000Z","size":300,"stargazers_count":18,"open_issues_count":0,"forks_count":4,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-28T22:15:03.600Z","etag":null,"topics":["agentic-ai","ai-agents","copilot-cli","copilot-extensions","multi-agent","orchestration","prompt-engineering","sealed-testing"],"latest_commit_sha":null,"homepage":"https://dubsopenhub.github.io/dark-factory/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DUBSOpenHub.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-02-23T22:03:33.000Z","updated_at":"2026-07-28T21:19:22.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/DUBSOpenHub/dark-factory","commit_stats":null,"previous_names":["dubsopenhub/dark-factory"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/DUBSOpenHub/dark-factory","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Fdark-factory","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Fdark-factory/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Fdark-factory/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Fdark-factory/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DUBSOpenHub","download_url":"https://codeload.github.com/DUBSOpenHub/dark-factory/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DUBSOpenHub%2Fdark-factory/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":36020486,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-29T02:00:04.910Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentic-ai","ai-agents","copilot-cli","copilot-extensions","multi-agent","orchestration","prompt-engineering","sealed-testing"],"created_at":"2026-07-29T06:01:27.126Z","updated_at":"2026-07-29T06:01:28.217Z","avatar_url":"https://github.com/DUBSOpenHub.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🏭 Dark Factory — Lights Out Builds\n\n![Validate](https://github.com/DUBSOpenHub/dark-factory/actions/workflows/validate.yml/badge.svg)\n![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)\n[![Security Policy](https://img.shields.io/badge/Security-Policy-brightgreen?logo=github)](SECURITY.md)\n![Version: v1.0.0](https://img.shields.io/badge/version-v1.0.0-5E5E5E.svg)\n![Platform: Copilot CLI](https://img.shields.io/badge/platform-Copilot%20CLI-232F3E.svg)\n![Language: Markdown](https://img.shields.io/badge/written%20in-Markdown-000000.svg)\n[![Shadow Score Spec](https://img.shields.io/badge/Shadow%20Score-Spec%20v2.0%20%7C%20Level%204-brightgreen.svg)](https://github.com/DUBSOpenHub/shadow-score-spec)\n[![Website](https://img.shields.io/badge/website-dark--factory-1F6FEB.svg)](https://dubsopenhub.github.io/dark-factory/)\n\n**🌐 [dubsopenhub.github.io/dark-factory](https://dubsopenhub.github.io/dark-factory/)**\n\nDark Factory is a GitHub Copilot CLI skill that turns a short free-text goal into a production-grade pull request. It isolates the work in a disposable git worktree, orchestrates eight specialist agents drawn from **different model families**, and measures quality with [sealed-envelope testing](https://github.com/DUBSOpenHub/shadow-score-spec) — builders never see the hidden acceptance suite that judges them, and the suite is never written by a mind that thinks like theirs.\n\n\u003e ⚡ **Get started fast!** Copy this right into the [Copilot CLI](https://github.com/github/copilot-cli):\n\u003e ```\n\u003e /skills add DUBSOpenHub/dark-factory\n\u003e ```\n\n## Contents\n1. [Why Sealed-Envelope Testing?](#why-sealed-envelope-testing)\n2. [Why Model Independence?](#why-model-independence)\n3. [Pipeline Overview](#pipeline-overview)\n4. [Command Reference](#command-reference)\n5. [Installation \u0026 Setup](#installation--setup)\n6. [Configuration Reference](#configuration-reference)\n7. [Usage Examples](#usage-examples)\n8. [Architecture Overview](#architecture-overview)\n9. [Factory Operating Manual](#factory-operating-manual)\n10. [FAQ](#faq)\n11. [Troubleshooting](#troubleshooting)\n12. [Contributing](#contributing)\n13. [License](#license)\n\n## Why Sealed-Envelope Testing?\nSealed testing creates a blindfolded QA loop: the QA Sealed agents write acceptance tests before any code exists, hide them in a vault **outside the repository**, and the builder never sees them. The gap between what the sealed suite demands and what the code delivers reveals whether the build is trustworthy.\n\n**Benefits**\n- **Prevents overfitting.** Builders can’t “teach to the test” because they never see the sealed suite.\n- **Quantifies quality.** [Shadow scores](https://github.com/DUBSOpenHub/shadow-score-spec) (sealed failures ÷ sealed total) expose blind spots numerically.\n- **Automates escalation.** Hardening cycles fire automatically, but the engineer still sees only failure messages.\n- **Retains speed.** Express mode still produces sealed tests immediately after setup, so even fast fixes retain coverage.\n\n**Comparison**\n| Approach | What builders see | When tests are written | Blind-spot risk |\n|----------|------------------|------------------------|-----------------|\n| Classic TDD | Entire suite | During implementation | Spec drift if requirements change mid-build |\n| Manual QA | Human docs | Post-build | Slow feedback, inconsistent coverage |\n| **Dark Factory** | Only failure messages | Before Phase 3, hidden until validation | Shadow score proves whether the spec was truly covered |\n\n## Why Model Independence?\n\n\u003e A Shadow Score is only meaningful if the tests and the code come from **different minds**.\n\nHiding the tests answers *“can the builder **see** the tests?”* It does not answer *“does the builder **think like** the test author?”*\n\nv0.1.0 assigned the same model to both `qa_sealed` and `lead_eng`. Two instances of one model family share training data, reasoning priors, and — critically — **failure modes**. If the family doesn’t think to test Unicode homographs, it also doesn’t think to normalise them. The sealed test is never written, the code is never hardened, the score reads **0%**, and the bug ships with a green light.\n\n**The bias is not random. It always points toward 0%** — the system overclaims quality exactly when it is least entitled to.\n\nDark Factory v1.0 makes cross-family separation an **invariant**, checked at Phase 0 and in CI:\n\n| Pair | If same family | Verdict |\n|------|----------------|---------|\n| Seal author == Implementer | Score too low → overclaims quality | ❌ **forbidden** |\n| PM == Implementer | Builder infers unstated assumptions | ❌ **forbidden** |\n| Arch Critic == Architect | Critic inherits the designer’s blind spots | ❌ **forbidden** |\n| Red Team == Implementer | Attacker inherits the builder’s assumptions | ❌ **forbidden** |\n| PM == Seal author | Score too high → conservative | ⚠️ allowed, must be disclosed |\n\nNote the asymmetry: **not every overlap is equally harmful.** A pairing that biases the score *upward* is conservative and therefore tolerable; a pairing that biases it *downward* is not, because it hides defects. Blanket “everything must differ” rules miss this distinction.\n\nEvery report carries its own provenance, so a score can be trusted or discarded on evidence:\n\n```json\n\"report\": {\n  \"independence\": \"strong\",\n  \"seal_author_families\": [\"openai\", \"google\"],\n  \"implementer_family\": \"anthropic\"\n}\n```\n\nIf the invariant is violated and `on_violation: warn` is set, the run continues but every report is stamped ⚠️ **ADVISORY** and is explicitly non-authoritative. See [`protocols/model-independence.md`](protocols/model-independence.md).\n\n### Seal plurality\n\nOne sealed suite tells you whether the **code** is right. Two independent sealed suites tell you whether the **specification** is right.\n\nWhen two model families read the same PRD and write *contradictory* assertions, at least one of them inferred something the spec never stated. Dark Factory measures this as `spec_ambiguity` and refuses to build against a spec that is too ambiguous to build against — because the code would be correct against one reading and wrong against the other, and nobody would know which.\n\n## Pipeline Overview\n```\n                    🏭 DARK FACTORY — LIGHTS OUT LINE\n\n  Phase 0        Phase 1        Phase 2a/2b      Phase 2.5\n  Setup +    →   Product    →   QA 🔒 ×N     →   Ambiguity +\n  Invariants     Spec           + Architecture    Arch Critique\n                                                       │\n        ┌──────────────────────────────────────────────┘\n        ▼\n  Phase 3        Phase 4          Phase 4.5      Phase 5\n  Build +    →   Verify        →  Red Team   →   Hardening\n  Open Tests     (disposable       (attacks       (multi-turn,\n                  worktree)         the spec)      escalating)\n                       │\n                       ▼\n        Phase 6 Delivery — always human-approved\n                       │\n                       ▼\n        Phase 7 Outcome Eval + Calibration (optional)\n```\n\n**What each new phase buys you**\n\n| Phase | Adds | Catches |\n|-------|------|---------|\n| **0** | Invariant enforcement | A biased run, *before* spending a cent on it |\n| **2.5a** | Spec ambiguity gate | Requirements that support two different systems |\n| **2.5b** | Architecture critique | Design flaws while they cost one agent turn, not a rewrite |\n| **4.5** | Red team | Vulnerabilities the spec never thought to forbid |\n\n*Express mode* condenses to: Phase 0 → (express QA Sealed from raw goal) → Phase 3 → Phase 4 → Phase 5 (hardening, runs only when shadow score \u003e 0%) → Phase 6. Express skips **specification** phases, never quality gates.\n\n*Tournament mode* runs N cross-family competitors through Phase 3 in parallel, grades them all against the **same** envelope, and delivers only the winner.\n\n## Command Reference\n| Command | Purpose |\n|---------|---------|\n| `dark factory — \u003cgoal\u003e` | Full pipeline with checkpoints at every phase. |\n| `dark factory dark — \u003cgoal\u003e` | Full pipeline, no phase prompts. Gates still fire; delivery still needs a human. |\n| `dark factory express — \u003cgoal\u003e` | Skips PRD/ARCH, still seals tests from the raw goal, single checkpoint at delivery. |\n| `dark factory tournament — \u003cgoal\u003e` | N cross-family competitors, one shared envelope, leaderboard + failure-overlap report. |\n| `dark factory pareto — \u003cgoal\u003e` | Tournament that reports the non-dominated (quality, cost) set instead of one winner. |\n| `dark factory premium — \u003cgoal\u003e` | Routes all agents through `config.roles.*.premium` for one run. |\n| `dark factory conformance` | Prints the resolved families, invariant status, and Shadow Score Spec level. |\n| `dark factory resume` | Reloads the most recent `state.json` and continues from the saved phase. |\n| `dark factory status` | Prints `state.json` plus any pending outcome evaluations without mutating state. |\n| `dark factory evaluate \u003crun-id\u003e` | Launches Phase 7 Outcome Evaluator for an archived run. |\n\n## Installation \u0026 Setup\n### Prerequisites\n- Git 2.35+ with worktree support.\n- GitHub Copilot CLI `\u003e= 0.13.0` (skills `/skills add` available).\n- Access to **at least two model families** (e.g. Anthropic + OpenAI, or OpenAI + Google). Without this, Level 4 conformance is unreachable and scores are advisory only.\n- Python 3.10+ with `pyyaml` to run the conformance validator; Node 18+ for markdown lint.\n\n### Steps\n1. **Clone**: `git clone https://github.com/DUBSOpenHub/dark-factory.git \u0026\u0026 cd dark-factory`\n2. **Register the skill**: `/skills add .`\n3. **Tune `config.yml`** — start with `roles` and confirm your families are available.\n4. **Verify conformance**: `python3 .github/scripts/validate_conformance.py`\n5. **Smoke test**: `dark factory express — say hello world` to confirm isolation + sealed QA work end to end.\n6. **Optional**: install markdownlint + yamllint to reproduce CI locally.\n\n\u003e Run step 4 after every `config.yml` edit. A configuration that violates an invariant will abort at Phase 0 anyway — catching it in one second beats catching it after the run has already paid for a PRD.\n\n## Configuration Reference\nDark Factory reads `config.yml` on every run and never hardcodes tunables. Run `python3 .github/scripts/validate_conformance.py` after editing it — the independence invariants are machine-checked.\n\n### `invariants` — hard preconditions (Phase 0 aborts on violation)\n| Key | Default | Purpose |\n|-----|---------|---------|\n| `cross_family_required` | `true` | Seal author and implementer must be different model families. |\n| `seal_plurality_min` | `2` | Minimum independent families authoring sealed suites. `\u003e= 2` enables the ambiguity gate. |\n| `workspace_isolation` | `strict` | `strict` = disposable verify worktree. `legacy` = Spec v1.0.0 §4.3 copy-in behaviour. |\n| `on_violation` | `abort` | `abort` or `advisory` (run continues, report stamped ⚠️ ADVISORY). |\n\n### `families` / `model_capabilities`\n`families` maps every model to a family so invariants can be evaluated. `model_capabilities` records which knobs each model actually accepts — several models take no `reasoning_effort` at all, and passing one anyway is a silent failure. The orchestrator **omits** unsupported parameters rather than downgrading them, and CI validates every configured dispatch against this map.\n\n### `roles` — model + effort + context tier, per role\n| Role | Default | Effort | Why |\n|------|---------|--------|-----|\n| `product_mgr` | `gpt-5.5` | `medium` | PRD author. |\n| `architect` | `claude-opus-5` | `xhigh` | Highest-leverage one-shot decision in the run. |\n| `arch_critic` | `gpt-5.6-terra` | `xhigh` | Must differ in family from `architect`. |\n| `qa_sealed` | `gpt-5.6-terra` | `max` | Must differ in family from `lead_eng`. Best dollar in the pipeline. |\n| `lead_eng` | `claude-opus-4.8` | `medium` | Builder + hardening. |\n| `qa_validator` | `gemini-3.6-flash` | `low` | Mechanical — runs a test runner and counts. Don't pay to think. |\n| `red_team` | `gpt-5.6-sol` | `xhigh` | Attack is a reasoning task. Must differ from `lead_eng`. |\n| `outcome_evaluator` | `claude-sonnet-5` | `high` | Phase 7 analyst. |\n| `classifier` | `gemini-3.6-flash` | `minimal` | Sub-second express/full triage. |\n\n\u003e `reasoning_effort` is the cost lever that matters. Buying quality by swapping the model SKU is the 2025 move; effort is cheaper and far more targeted. Spend it on sealed test authoring, architecture, and red teaming — not on running a test runner.\n\n### `seal_plurality`\n| Key | Default | Purpose |\n|-----|---------|---------|\n| `enabled` | `true` | Author N sealed suites from N families. |\n| `authors` | `gpt-5.6-terra`, `gemini-3.1-pro-preview` | One entry per independent suite. |\n| `express_authors` | `1` | Express uses a single author to stay fast. |\n\n### `tournament`\n| Key | Default | Purpose |\n|-----|---------|---------|\n| `enabled` | `false` | Opt in per run with `dark factory tournament`. |\n| `competitors` | anthropic / moonshot / microsoft | No competitor may share a family with a seal author. |\n| `enforce_judge_independence` | `true` | CI-checked. |\n| `all_fail_spec_review_threshold` | `40` | If every competitor scores above this, the **spec** is the defect — escalate instead of hardening. |\n\n### `hardening`\n| Key | Default | Purpose |\n|-----|---------|---------|\n| `max_cycles` | `4` | Cycles before escalation. |\n| `ladder` | 4 rungs | Each rung escalates. Rung 3 switches **family** — identical retries are rerolling, not hardening. |\n| `max_reveal` | `assertions` | Progressive disclosure ceiling. **Never** includes test source. |\n\n### `autonomy`\n| Key | Default | Purpose |\n|-----|---------|---------|\n| `mode` | `supervised` | `dark` / `supervised` / `manual`. |\n| `gates.spec_ambiguity_max` | `0.25` | Sealed-suite disagreement ratio. |\n| `gates.arch_critic_severity` | `high` | Escalate on findings at or above this severity. |\n| `gates.shadow_score_max` | `10` | Sealed failure percentage. |\n| `gates.red_team_high_findings_max` | `0` | Any high-severity finding stops the line. |\n| `gates.cost_ceiling_usd` | `5.00` | Hard stop. |\n| `on_gate_breach` | `park_and_notify` | `park_and_notify` or `abort`. |\n\n\u003e **Delivery always requires human approval, in every mode.** Not configurable.\n\n### `safety`\n| Key | Default | Purpose |\n|-----|---------|---------|\n| `summarize_artifacts` | `false` | v0.1.0 summarised by default. With long-context tiers that is now **harmful** — it lossy-compresses the contract both the tests and the code derive from, so the Shadow Score ends up measuring the summariser. |\n| `max_prd_lines` | `2000` | Emergency backstop only. |\n| `max_artifact_lines` | `5000` | Emergency backstop only. |\n\nAcceptance criteria always pass through verbatim. If a backstop ever fires, the run is flagged `spec_truncated` and its score is reported as ADVISORY.\n\n### `isolation`\n| Key | Default | Purpose |\n|-----|---------|---------|\n| `workspace_root` | `.factory` | Root for worktrees. |\n| `branch_prefix` | `factory/` | Branch prefix per run. |\n| `vault_dir` | `~/.factory-vault` | Sealed tests live **outside the repo** — `.factory/sealed/` sat inside the builder's world, reachable by its unscoped `bash`/`glob`/`grep`. |\n| `verify_dir` | `.factory/verify` | Disposable worktree built from the engineer's commit, destroyed after validation. |\n| `canary_enabled` | `true` | A canary in the vault; access forces the score to INVALID. |\n| `state_file` | `.factory/state.json` | Crash-recovery ledger. |\n\n### `learning`\n| Key | Default | Purpose |\n|-----|---------|---------|\n| `blind_spot_memory` | `true` | **Loop A** — seed QA Sealed with defect classes that historically escaped in this stack. |\n| `empirical_routing` | `true` | **Loop B** — route roles by measured Shadow Score instead of by guesswork in this file. |\n| `calibration` | `true` | **Loop C** — compare Phase 4 score against Phase 7 outcome; if 0% runs still fail post-ship, the suites are too shallow. |\n\n### `premium`\nPer-role overrides for `dark factory premium`. v0.1.0 applied a single `premium_model` to every role — that is now a conformance bug, because routing all agents through one flagship model collapses the pipeline into one family and silently voids the Shadow Score. Premium raises capability **within** each role's family.\n\n### `reporting`\n| Key | Default | Purpose |\n|-----|---------|---------|\n| `spec_version` | `2.0.0` | Stamped into every report. |\n| `conformance_level` | `4` | Adversarial Independence. |\n| `emit_json` | `true` | Emit `SHADOW-REPORT.json` with the Spec §5.2 required fields, so scores are comparable across tools instead of regex-scraped out of prose. |\n\n## Usage Examples\n### 1. Full build (“Lights Out”)\n```\ndark factory — build a CLI that audits dependencies for GPL licenses\n```\nExpected output snippet:\n```\n🏭 Run: run-20260401-1215 | Mode: FULL | Autonomy: SUPERVISED\n🏭 Phase 0 ✅ Invariants OK — impl=anthropic  seals=openai,google  independence=strong\n🏭 Phase 2 🔒 SHA-256 (sealed envelope, 2 suites): sha256:d3b1...\n🏭 Phase 2.5 ✅ spec_ambiguity 0.06 ≤ 0.25 | arch_critic: medium ≤ high\n🏭 SHADOW_SCORE: 11.1% (2 sealed failures — 1 security, 1 edge_case)\n🏭 Phase 4.5 ✅ Red team: 0 high / 1 medium\n🏭 Delivery report ready — approve / reject\n```\nTips: provide scope (“CLI”, “API”) so the PRD agent knows what to model; use checkpoints to add clarifications without restarting.\n\n### 2. Express fix\n```\ndark factory express — add rate limiting middleware to REST API\n```\nExpected output snippet:\n```\n🏭 Mode: EXPRESS (classifier: single file, no new deps, no auth surface)\n🏭 QA Sealed (express) derived acceptance criteria from raw goal\n🏭 Phase 6 — delivery checkpoint (only gate)\n```\nUse express for quick hardening tasks or doc-only fixes. Sealed QA still runs from the goal text, and gates still fire.\n\n### 3. Tournament\n```\ndark factory tournament — build a CSV import CLI with validation\n```\nExpected output snippet:\n```\n🏭 Mode: TOURNAMENT | 3 competitors: anthropic, moonshot, microsoft\n🏭 Sealed envelope written BEFORE any competitor started: sha256:9f3c1a\n🥇 claude-opus-4.8   5.6%  $1.90\n🥈 kimi-k2.7-code   11.1%  $0.60\n🥉 mai-code-1-flash 22.2%  $0.20\n⚠️  test_rejects_path_traversal failed for ALL competitors → spec defect, not a model defect\n```\nModel selection stops being folklore and becomes an **output** of the pipeline. The failure-overlap table is the real product: when every competitor fails the same test, no amount of model swapping will fix it.\n\n### 4. Outcome evaluation\n```\ndark factory evaluate run-20260401-1215\n```\nExpected output snippet:\n```\n🏭 OUTCOME EVALUATION — Run run-20260401-1215\n🏭 PRD Criteria Met: 6/7 | KPIs On Track: 2/3\n🏭 Outcome Score: 75/100\n🏭 Calibration: ⚠️ optimistic — Shadow 0% but 2 escaped defects (1 spec gap, 1 suite gap)\n```\nIf `auto_evaluate_after_days \u003e 0`, the factory will prompt you when a run is due even without this command. Calibration feeds Learning Loop C: an escaped defect is the highest-value signal the system produces, and v0.1.0 threw it away.\n\n## Architecture Overview\n- **Factory Manager (SKILL.md):** Loads config, enforces the Phase 0 invariants, determines mode, and routes each `task()` call to the right model with the right `reasoning_effort` and `context_tier` — omitting parameters the model doesn't support.\n- **Agent Team (8):** Product Manager → QA Sealed ×N \u0026 Architect (parallel) → Arch Critic → Lead Engineer → QA Validator → Red Team → Outcome Evaluator. Agents are stateless and receive only what the handoff manifest passes. The exception is hardening, which continues the **same** engineer via `write_agent` so it never re-tries an approach it already ruled out.\n- **Agent types:** the pipeline uses purpose-built CLI agent types rather than making everything `general-purpose` — `rubber-duck` for the Arch Critic, `security-review` for the Red Team, `task` for the QA Validator, `explore` for complexity triage.\n- **Isolation:** Every run lives in `.factory/runs/\u003crun-id\u003e`. Sealed tests live in `~/.factory-vault/\u003crun-id\u003e`, **outside the repository**, and are executed in a throwaway worktree. The builder's tree never contains sealed files at any point.\n- **Independence:** Every role resolves to a model family. Seal authors and the implementer may never share one. Enforced at Phase 0 and in CI.\n- **State \u0026 Reporting:** `state.json` persists mode, phase, artifacts, sealed hash, gate results, and checkpoint decisions. SQLite tables (`factory_runs`, `phase_results`, `sealed_suites`, `blind_spots`) carry provenance, cost, hardening velocity, and cross-run memory.\n- **Templates \u0026 Protocols:** Agents follow the markdown templates in `templates/` and the invariants in `protocols/` (sealed envelope, model independence, gates).\n\n## Factory Operating Manual\n### Sealed Test Lifecycle\n1. **Authoring (Phase 2a or Express QA):** N QA Sealed agents from N different families write runnable tests using only PRD text or raw goals — independently, never seeing each other's work.\n2. **Ambiguity check (Phase 2.5a):** Suites are compared for contradictions. Contradictions mean the *spec* is ambiguous and gate the run; divergences are the healthy coverage dividend of plurality.\n3. **Hashing:** The Factory Manager records a SHA-256 hash over the merged envelope and stores it in `state.json`.\n4. **Vault storage:** Files live in `~/.factory-vault/\u003crun-id\u003e` (mode `700`) with a canary file — **outside the repository**, because the builder's `bash`/`glob`/`grep` are unscoped and anything inside its worktree is reachable regardless of what the prompt says.\n5. **Execution (Phase 4):** A disposable worktree is created from the engineer's commit, both suites run there, and the worktree is destroyed. Sealed tests are never written into the builder's workspace — not even temporarily.\n6. **Integrity:** A canary atime change sets `seal_broken: true` and invalidates the score.\n7. **Archive:** On delivery approval, sealed files plus PRD/ARCH/SHADOW/RED-TEAM reports are archived under `.factory/archive/\u003crun-id\u003e` for outcome evaluation.\n\n### State File Anatomy\n| Key | Meaning | Example |\n|-----|---------|---------|\n| `run_id` | Unique identifier for the current build | `run-20260401-1215` |\n| `mode` | `full`, `express`, or `tournament` | `full` |\n| `autonomy` | `dark`, `supervised`, or `manual` | `supervised` |\n| `current_phase` | Phase identifier (`0`–`7`, including `2.5` and `4.5`) | `3` |\n| `independence` | `strong` or `weak` | `strong` |\n| `seal_author_families` | Families that authored the envelope | `[\"openai\",\"google\"]` |\n| `sealed_hash` | Integrity check for hidden tests | `sha256:d3b1...` |\n| `initial_shadow_score` | First validation, before any hardening | `22.2` |\n| `gates` | Measured value, threshold, and status per gate | `{ \"shadow_score\": { \"value\": 4.0, ... } }` |\n| `checkpoints` | Map of checkpoint decisions keyed by checkpoint number | `{ \"1\": { \"status\": \"approved\", ... } }` |\n\n\u003e **Checkpoint entry format** (see [`protocols/checkpoint-gate.md`](protocols/checkpoint-gate.md)): each entry is `{ \"status\": \"approved\", \"feedback\": null, \"decided_at\": \"2026-02-23T21:35:00Z\" }`. Status may be `approved`, `modified`, or `aborted`.\n\n| `evaluation_due_at` | Timestamp when Phase 7 should auto-trigger | `2026-04-08T13:00:00Z` |\n| `artifacts` | Map of produced files (prd, arch, shadow_report, red_team) | `{ \"prd\": \"PRD.md\", ... }` |\n\n### Quality Gates\nGates are evaluated automatically in **every** autonomy mode, and a breach must print the measured value, the threshold, and the specific findings. A breach that cannot explain itself is not actionable.\n\n| Gate | Phase | Default | Meaning of a breach |\n|------|-------|---------|---------------------|\n| Spec ambiguity | 2.5a | `0.25` | Two families read the spec differently — the requirements support two different systems. |\n| Architecture critique | 2.5b | `high` | A design flaw that guarantees a defect class, caught before a line of code exists. |\n| Shadow Score | 4 | `10%` | Sealed failures. Above 25% suggests spec/test misalignment, not just a buggy build. |\n| Red team | 4.5 | `0 high` | An exploitable vulnerability the spec never thought to forbid. |\n| Cost | any | `$5.00` | Hard stop. |\n| Seal integrity | 4 | — | Canary read or hash mismatch. **Always aborts** — the score is meaningless. |\n\nBeyond the gates: exceeding `hardening.max_cycles` triggers the continue/deliver/abort decision, and anything under 70/100 in OUTCOME-REPORT.md triggers a follow-up action item.\n\n### Operator Checklist\n- [ ] Confirm `.factory/` is gitignored (see `.gitignore`).\n- [ ] Run `python3 .github/scripts/validate_conformance.py` after any `config.yml` change.\n- [ ] Keep agent prompts under 200 lines (`wc -l agents/*.md`).\n- [ ] Update `catalog.yml` version **and** `links.agents` whenever prompts change or are added.\n- [ ] Run `.github/workflows/validate.yml` locally (or via PR) before merging.\n- [ ] Archive finished runs you care about; delete stale entries with `rm -rf .factory/archive/run-*`.\n- [ ] Review [`docs/ADR.md`](docs/ADR.md) before proposing governance changes.\n\n### Run Artifacts \u0026 Locations\n| Artifact | Produced By | Stored In | Notes |\n|----------|-------------|-----------|-------|\n| `PRD.md` | Product Manager | Worktree root \u0026 archive | Acceptance criteria pass through verbatim. |\n| `ARCH.md` | Architect | Worktree root \u0026 archive | Contains diagrams + contracts. |\n| `ARCH-CRITIQUE.md` | Arch Critic | Worktree root \u0026 archive | Requirement-coverage walk + severity-graded findings. |\n| `AMBIGUITY.md` | Factory Manager | Worktree root \u0026 archive | Names disputed **behaviours**, never test source. |\n| `SHADOW-REPORT.md` | QA Validator | Worktree root \u0026 archive | Score, coverage comparison, failure table, trend. |\n| `SHADOW-REPORT.json` | QA Validator | Worktree root \u0026 archive | Spec §5.2 required fields + provenance. |\n| `RED-TEAM.md` | Red Team | Worktree root \u0026 archive | Breach Score; findings classed spec-gap vs implementation-bug. |\n| `TOURNAMENT.md` | Factory Manager | Worktree root | Leaderboard + failure overlap (tournament mode only). |\n| `FACTORY-REPORT.md` | Factory Manager | Worktree root | Delivery summary. |\n| Sealed tests | QA Sealed ×N | `~/.factory-vault/\u003crun-id\u003e` | Outside the repo; hashed; canary-protected. |\n| `OUTCOME-REPORT.md` | Outcome Evaluator | Worktree \u0026 archive | KPIs, escaped defects, calibration. |\n\n### Data Retention \u0026 Cleanup\n- `.factory/runs/` should contain only active runs. Delivery/abort must delete the worktree and branch immediately.\n- `.factory/archive/` feeds Phase 7; keep what you need, prune the rest.\n- `state.json` is recreated for every run and may be deleted once the run is merged and archived.\n\n## FAQ\n1. **Does express mode skip QA?** No. QA Sealed still writes tests — it just uses the raw goal as its spec. Express skips *specification* phases, never quality gates.\n2. **Why can't I use the same model for everything?** Because then the tests and the code come from the same mind. Correlated blind spots make the sealed test never fire, and the score reads 0% on a build that has real defects. The bias always points toward overclaiming quality. See [Why Model Independence?](#why-model-independence).\n3. **What if I only have access to one model family?** Set `invariants.on_violation: advisory`. The run proceeds, but every report is stamped ⚠️ ADVISORY and is explicitly non-authoritative. An honest \"we can't measure this\" beats a confident wrong number.\n4. **Is seal plurality worth the extra cost?** It is the only way to measure whether your *specification* is ambiguous. One suite grades the code; two independent suites grade the requirements. Set `seal_plurality.enabled: false` and `invariants.seal_plurality_min: 1` to opt out.\n5. **Where do sealed tests live?** `~/.factory-vault/\u003crun-id\u003e` — outside the repository. `.factory/sealed/` sat inside the builder's own worktree, and the builder has unscoped `bash`/`glob`/`grep`. Isolation has to be filesystem topology, not prompt wording.\n6. **Are sealed tests copied into the build workspace?** No. A disposable worktree is created from the engineer's commit, both suites run there, and it is destroyed. This is a deliberate divergence from Spec v1.0.0 §4.3 — set `invariants.workspace_isolation: legacy` to restore the old behaviour.\n7. **Can I run different models per agent?** Yes — that's mandatory now. Edit `config.roles.*`, then run the conformance validator.\n8. **What is `reasoning_effort` and where should I spend it?** It's the per-call thinking budget. Spend it on sealed test authoring, architecture, and red teaming; not on the validator, which just runs a test runner and counts. Note that some models accept no effort parameter at all — check `config.model_capabilities`.\n9. **Does hardening ever show the engineer the tests?** Never the source. At the final rung `max_reveal: assertions` exposes the failing assertion text, which turns an unbounded guess into a solvable problem without breaking the seal that produced the score.\n10. **What does a 0% Shadow Score actually prove?** That the implementation satisfies the spec *as tested*, by test authors who don't think like the builder. It says nothing about what the spec forgot — that's the Red Team's job, and the calibration loop's.\n11. **When is Phase 7 triggered automatically?** If `auto_evaluate_after_days \u003e 0`, `dark factory status` or `resume` checks for due runs and launches the Outcome Evaluator.\n12. **What’s the “Lights Out” codename about?** Building in the dark — sealed tests, no human on the floor — until the delivery checkpoint turns the lights back on. Autonomy means autonomous *with evidence*, not autonomous with permission.\n\n## Troubleshooting\n| Symptom | Likely Cause | Fix |\n|---------|--------------|-----|\n| `⛔ INVARIANT VIOLATION: cross_family_required` | Seal author and implementer resolve to the same family | Change `config.roles.qa_sealed.model` or `lead_eng.model`, then run the conformance validator. |\n| `model 'X' does not support reasoning_effort 'Y'` | A role is configured with a knob its model doesn't accept | Check `config.model_capabilities`; omit the parameter for models with an empty `effort` list. |\n| Run stops at `spec_ambiguity 0.33 \u003e 0.25` | Two families read the PRD differently | Read `AMBIGUITY.md`, resolve the contradictions in the PRD, re-run. This is the gate working. |\n| Shadow Score stamped ⚠️ ADVISORY | `on_violation: advisory` with a violated invariant | Fix the family assignment for an authoritative score. |\n| `seal_broken: true` | The vault canary was read during the build | Treat the score as invalid. Check for a tool with vault path access; re-run. |\n| `fatal: worktree add` fails | Dirty git state/untracked files | Commit/stash changes, ensure `.factory/` is gitignored, rerun. |\n| QA Sealed crashes citing “unknown stack” | Goal/PRD lacked tech hints | Add a “Technical Constraints” section or mention the runtime in the goal; rerun Phase 2. |\n| Shadow Score stays \u003e0 after max cycles | Hardening limit reached | Choose **continue-hardening** to reset the counter or **deliver-as-is** to surface outstanding failures. Check `hardening_velocity` — a flat trend means the *spec* is the problem. |\n| Every tournament competitor fails the same test | Specification defect, not a model defect | Fix the PRD. Swapping models will not help. |\n| `state.json` missing for resume | Worktree cleaned manually | Use `dark factory status` to list available runs; if none, restart from Phase 0. |\n| Markdown lint fails in CI | Missing newline at EOF or long lines | Run `markdownlint-cli2` locally. |\n| YAML validation job fails | Syntax error in `config.yml` or `catalog.yml` | Run `python -c \"import yaml; yaml.safe_load(open('config.yml'))\"` to pinpoint the issue. |\n| “agent files exist but are not listed in catalog.yml” | Added a prompt without updating the catalog | Add the path to `catalog.yml` under `links.agents`. |\n\n## Contributing\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for the full workflow (branch naming, testing playbooks, PR template). Before opening a PR, run `.github/workflows/validate.yml` locally or via GitHub Actions.\n\n**Quick ways to help:**\n- 🐛 [Report a bug](https://github.com/DUBSOpenHub/dark-factory/issues/new?template=bug_report.md)\n- 💡 [Suggest a feature](https://github.com/DUBSOpenHub/dark-factory/issues/new?template=feature_request.md)\n- 🏭 [Share a build](https://github.com/DUBSOpenHub/dark-factory/discussions/new?category=show-and-tell)\n\n## License\nReleased under the [MIT License](LICENSE) © 2026 DUBSOpenHub.\n\n## Links\n- 🌐 **Website** — [dubsopenhub.github.io/dark-factory](https://dubsopenhub.github.io/dark-factory/)\n- 📐 **Shadow Score Spec** — [DUBSOpenHub/shadow-score-spec](https://github.com/DUBSOpenHub/shadow-score-spec)\n- 📓 **Changelog** — [CHANGELOG.md](CHANGELOG.md)\n- 🏷️ **Releases** — [github.com/DUBSOpenHub/dark-factory/releases](https://github.com/DUBSOpenHub/dark-factory/releases)\n\n---\n\n🐙 Created with 💜 by [@DUBSOpenHub](https://github.com/DUBSOpenHub) with the [GitHub Copilot CLI](https://docs.github.com/copilot/concepts/agents/about-copilot-cli).\n\nLet's build! 🚀✨\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdubsopenhub%2Fdark-factory","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdubsopenhub%2Fdark-factory","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdubsopenhub%2Fdark-factory/lists"}