{"id":32443365,"url":"https://github.com/leanprover/comparator","last_synced_at":"2025-10-26T03:02:10.834Z","repository":{"id":320162168,"uuid":"997336954","full_name":"leanprover/comparator","owner":"leanprover","description":null,"archived":false,"fork":false,"pushed_at":"2025-10-22T08:18:29.000Z","size":19,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-10-22T10:15:35.542Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Lean","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leanprover.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-06-06T11:02:28.000Z","updated_at":"2025-10-22T08:18:33.000Z","dependencies_parsed_at":"2025-10-22T10:15:49.342Z","dependency_job_id":null,"html_url":"https://github.com/leanprover/comparator","commit_stats":null,"previous_names":["leanprover/comparator"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/leanprover/comparator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leanprover%2Fcomparator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leanprover%2Fcomparator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leanprover%2Fcomparator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leanprover%2Fcomparator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leanprover","download_url":"https://codeload.github.com/leanprover/comparator/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leanprover%2Fcomparator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281051335,"owners_count":26435741,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-26T02:00:06.575Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-26T03:01:08.136Z","updated_at":"2025-10-26T03:02:10.829Z","avatar_url":"https://github.com/leanprover.png","language":"Lean","funding_links":[],"categories":[],"sub_categories":[],"readme":"**IMPORTANT** While much care has been put into making the program robust against malicious inputs, this has not been independently test yet.\n\n# Comparator\nComparator is a trustworthy judge for Lean proofs. It relies having an existing Lean installation as\nwell as two additional binaries in `PATH`:\n1. [`landrun`](https://github.com/Zouuup/landrun)\n2. [`lean4export`](https://github.com/leanprover/lean4export/) at a version that is compatible with\n   whatever Lean version your project is targeting.\n\nThe comparator is configured through a JSON file:\n```\n{\n    \"challenge_module\": \"Challenge\",\n    \"solution_module\": \"Solution\",\n    \"theorem_names\": [\"todo1\"],\n    \"permitted_axioms\": [\"propext\", \"Quot.sound\", \"Classical.choice\"]\n}\n```\nWhere `Challenge.lean` contains at least a theorem named `todo1` that has a `sorry` (or any other proof)\nand `Solution.lean` is provided by a party trying to convince you that they have proven `todo1` by\nwriting out the same theorem but with a proper proof attached.\n\nGiven the following assumptions:\n1. Only the `Solution.lean` file is controlled by a potentially malicious party, all other files\n   including most crucially `lakefile.toml`/`lakefile.lean` and `Challenge.lean` are controlled by\n   you.\n2. You have not previously tried to compile the `Solution` file (as that might compromise your\n   `Challenge` file to make it seem like you are looking for a different proof than you actually are)\n3. You have the `landrun` and `lean4export` binary in `PATH`\n4. `landrun` works correctly on your system\n5. The Lean kernel is correct (in the future we will add support for running different kernels as\n   well to increase trust further)\n6. You are not running this under a privileged user\n\nIf the following command succeeds:\n```\nlake env path/to/comparator/binary path/to/config.json\n```\n\nAll theorems in `Solution` that are listed in `theorem_names` are guaranteed to:\n1. Prove the same statement as provided in `Challenge`\n2. Use no more axioms than listed in `permitted_axioms`\n3. Be accepted by the Lean kernel\n\nNote that running `lake exe cache get` to download a Mathlib cache is acceptable before running the\ncomparator if you trust the cache to not be modified as to, e.g. contain different definitions from\nthe one you would expect to make proofs trivial.\n\n\n## Internals:\nWe generally adopt a policy of not loading olean files as they just get mmaped into our address\nspace and then dereferenced and are as such a potential point of attack for sophisticated adversaries.\n\nThe comparator performs the following steps to ensure these properties:\n1. Build `Challenge` using `lake` in a `landrun` sandbox that has:\n  - read access to the entire file system and write access to `/dev`\n  - write access to the `.lake` directory of the project\n2. Run `lean4export` on the produced `Challenge.olean` in a `landrun` sandbox that has:\n  - read access to the entire file system and write access to `/dev`\n3. Repeat the same build sandboxed and export sandboxed steps with `Solution`\n4. Verify that all declarations used in the statement of all relevant theorems in `Challenge`\n   are the same as in the `Solution` environment.\n5. Verify that the body of all relevant theorems in the `Solution` environment only uses axioms\n   listed in `permitted_axioms`\n6. Replay the `Solution` environment into the Lean kernel. Doing this within the same process as the\n   comparator should be safe as the worst thing that can happen at this point is an exploit that\n   makes the kernel accept when it should reject and that same exploit should also be applicable\n   from within an external process.\n\nNote that as `Challenge` is trusted, both the sandbox and lean4export step for `Challenge` are not\nnecessary to the best of our knowledge. We still adopt these rather free measures as additional\nparanoia in case an adversary comes up with a mean of attack anyway.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleanprover%2Fcomparator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleanprover%2Fcomparator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleanprover%2Fcomparator/lists"}