{"id":13992902,"url":"https://github.com/google-deepmind/debate","last_synced_at":"2025-06-17T00:40:12.991Z","repository":{"id":209448263,"uuid":"723076331","full_name":"google-deepmind/debate","owner":"google-deepmind","description":"Formalizing stochastic doubly-efficient debate","archived":false,"fork":false,"pushed_at":"2024-10-08T13:17:43.000Z","size":87,"stargazers_count":93,"open_issues_count":2,"forks_count":17,"subscribers_count":10,"default_branch":"main","last_synced_at":"2024-11-29T13:40:19.431Z","etag":null,"topics":["lean4"],"latest_commit_sha":null,"homepage":"","language":"Lean","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-deepmind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-24T16:14:28.000Z","updated_at":"2024-11-22T20:50:05.000Z","dependencies_parsed_at":"2024-08-14T15:33:12.818Z","dependency_job_id":"04189630-af67-4bc1-a43d-e7d27a69338a","html_url":"https://github.com/google-deepmind/debate","commit_stats":null,"previous_names":["google-deepmind/debate"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/google-deepmind/debate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fdebate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fdebate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fdebate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fdebate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-deepmind","download_url":"https://codeload.github.com/google-deepmind/debate/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fdebate/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260268635,"owners_count":22983601,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lean4"],"created_at":"2024-08-09T14:02:10.287Z","updated_at":"2025-06-17T00:40:12.964Z","avatar_url":"https://github.com/google-deepmind.png","language":"Lean","funding_links":[],"categories":["Lean"],"sub_categories":[],"readme":"Stochastic doubly-efficient debate formalization\n================================================\n\n**Summary:** We formalize the correctness of the main stochastic oracle\ndoubly-efficient debate protocol from\n[Brown-Cohen, Irving, Piliouras (2023), Scalable AI safety via doubly-efficient debate](https://arxiv.org/abs/2311.14125)\nin Lean 4.\n\n[Irving, Christiano, Amodei (2018), AI safety via debate](https://arxiv.org/abs/1805.00899)\nis one approach to AI alignment of strong agents, using two agents (\"provers\")\ncompeting in a zero-sum game to convince a human judge (\"verifier\") of the truth\nor falsity of a claim.  Theoretically, if we model the judge as a polynomial\ntime Turing machine, optimal play in the debate game can convince the judge of\nany statement in PSPACE.  However, this theoretical model is limited in several\nways: the agents are assumed to have unbounded computational power, which is not\na realistic assumption for ML agents, and the results consider only\ndeterministic arguments.\n\n[Brown-Cohen, Irving, Piliouras (2023), Scalable AI safety via doubly-efficient debate](https://arxiv.org/abs/2311.14125)\nimproves the complexity theoretic model of debate to be \"doubly-efficient\": both\nthe provers and the verifier have limited computational power.  It also treats\nstochastic arguments: the provers try to convince the judge of the result of a\nrandomized computation involving calls to a random oracle.  Concretely, the main\nformalized result is\n\n**Definition 6.1 (Lipschitz oracle machines).** An oracle Turing machine\n$M^\\mathcal{O}$ is $K$-Lipschitz if, for any other oracle $\\mathcal{O}'$ s.t.\n$|\\Pr(\\mathcal{O}(z)) - \\Pr(\\mathcal{O}'(z))| \\le \\epsilon$, we have\n$|\\Pr(M^\\mathcal{O}(x)) - \\Pr(M^\\mathcal{O'}(x))| \\le K \\epsilon$.\n\n**Theorem 6.2 (doubly-efficient stochastic oracle debate).** Let $L$ be a\nlanguage decidable by a $K$-Lipschitz probabilistic oracle Turing machine\n$M^\\mathcal{O}$ in time $T = T(n)$, measuring oracle queries only as costing\ntime.  Then there is an $O(K^2 T \\log T)$ prover time, $O(K^2)$ verifier time\ndebate protocol with cross-examination deciding $L$ with completeness $3/5$ and\nsoundness $3/5$.\n\nThe formalized result differs from the paper result slightly in that we focus\nonly on correctness: we do not yet formalize space complexity, count only oracle\nqueries for time complexity, and represent those queries only via the code for\nthe protocol.  We also define Lipschitz oracle machines differently: the paper\nfixes $M$ and lets both $\\mathcal{O}$ and $\\mathcal{O}'$ vary, while we fix both\n$M$ and $\\mathcal{O}$ and let only $\\mathcal{O}'$ vary (this slightly\nstrengthens the resulting theorem).\n\n1. `Prob/Defs.lean`: Finitely supported probability distributions, representing stochastic computations.\n2. `Comp/Defs.lean`: Stochastic computations relative to a set of oracles.\n3. `Comp/Oracle.lean`: Our computation model, including the definition of Lipschitz oracles.\n4. `Debate/Protocol.lean`: The debate protocol, honest players, and the definition of correctness.\n5. `Debate/Correct.lean`: The final correctness theorems.\n6. `Debate/Details.lean`: Proof details.\n7. `Debate/Cost.lean`: Query complexity of each player.\n\n## Acknowledgements\n\nWe thank [Eric Wieser](https://github.com/eric-wieser) for his careful review\nand many helpful suggestions.  The code is much better as a result!\n\n## Installation\n\n1. Install Lean 4 and Lake via elan: https://github.com/leanprover/elan\n2. Run `lake build` within the directory.\n\n## License and disclaimer\n\nCopyright 2023 DeepMind Technologies Limited\n\nAll software is licensed under the Apache License, Version 2.0 (Apache 2.0);\nyou may not use this file except in compliance with the Apache 2.0 license.\nYou may obtain a copy of the Apache 2.0 license at:\nhttps://www.apache.org/licenses/LICENSE-2.0\n\nAll other materials are licensed under the Creative Commons Attribution 4.0\nInternational License (CC-BY). You may obtain a copy of the CC-BY license at:\nhttps://creativecommons.org/licenses/by/4.0/legalcode\n\nUnless required by applicable law or agreed to in writing, all software and\nmaterials distributed here under the Apache 2.0 or CC-BY licenses are\ndistributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,\neither express or implied. See the licenses for the specific language governing\npermissions and limitations under those licenses.\n\nThis is not an official Google product.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fdebate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-deepmind%2Fdebate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fdebate/lists"}