{"id":29671422,"url":"https://github.com/anoma/zkp-compiler-shootout","last_synced_at":"2025-07-22T20:07:49.364Z","repository":{"id":58662892,"uuid":"520536166","full_name":"anoma/zkp-compiler-shootout","owner":"anoma","description":"Evaluating \u0026 benchmarking ZKP compilation strategies.","archived":false,"fork":false,"pushed_at":"2023-12-19T07:31:05.000Z","size":21344,"stargazers_count":118,"open_issues_count":7,"forks_count":12,"subscribers_count":23,"default_branch":"main","last_synced_at":"2024-03-20T09:51:00.793Z","etag":null,"topics":["benchmarking","zero-knowledge","zkp"],"latest_commit_sha":null,"homepage":"https://anoma.github.io/zkp-compiler-shootout/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/anoma.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.org","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-08-02T14:41:00.000Z","updated_at":"2024-01-30T18:06:50.000Z","dependencies_parsed_at":"2023-09-27T03:25:18.074Z","dependency_job_id":null,"html_url":"https://github.com/anoma/zkp-compiler-shootout","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/anoma/zkp-compiler-shootout","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anoma%2Fzkp-compiler-shootout","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anoma%2Fzkp-compiler-shootout/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anoma%2Fzkp-compiler-shootout/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anoma%2Fzkp-compiler-shootout/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/anoma","download_url":"https://codeload.github.com/anoma/zkp-compiler-shootout/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/anoma%2Fzkp-compiler-shootout/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266563915,"owners_count":23948689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarking","zero-knowledge","zkp"],"created_at":"2025-07-22T20:07:48.686Z","updated_at":"2025-07-22T20:07:49.342Z","avatar_url":"https://github.com/anoma.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ZKP (circuit) compiler shootout\n\nEvaluating \u0026amp; benchmarking ZKP compilation strategies.\n\nCurrently we are testing the following Zero Knowledge machines\n\n1. [RISC0](https://github.com/risc0/risc0)\n2. [Miden](https://github.com/maticnetwork/miden)\n3. [Triton](https://github.com/TritonVM/triton-vm/)\n4. [Plonk](https://github.com/ZK-Garage/plonk)\n5. [Halo2](https://github.com/zcash/halo2)\n\nIf you would like your machine / framework / compilation strategy to\nbe benchmarked against the standard suite, please submit a PR! You can\nfind instructions below.\n\nResults can be seen in the following files:\n\n- [BENCHMARKS-x86.md](./BENCHMARKS-x86.md)\n  + The results rendered in a mark down table\n- [BENCHMARKS-arm64.md](./BENCHMARKS-arm64.md)\n  + The results rendered in a mark down table\n- [anoma.github.io/zkp-compiler-shootout/](https://anoma.github.io/zkp-compiler-shootout/)\n  + The results with graphs and standard deviations, means, etc.\n\nThe results were collected on a `AMD Ryzen 7 5700X 8-Core @ 16x\n3.4GHz` CPU.\n\nTo see very rough notes about the languages in the benchmark and\npotential improvement points please read [my notes\nfile](./shootout/notes.org).\n## Dependencies\n\nFor some benchmark backends some external dependencies are required\n\n### Risc0\n\n1. `risczero`\n2. `cargo-risczero`\n\nwhich can be installed by the following command\n\n```sh\ncargo install cargo-risczero\ncargo risczero install\n```\n\n\n### Make table\n\n1. `cargo-criterion`\n2. `criterion-table`\n\nthese can both be installed with the following command.\n\n```sh\ncargo install cargo-criterion\ncargo install criterion-table\n```\n\n\n## How to get benchmark results\n\nThere are two commands for producing results\n\n1. `make table arg=\"desired_backend\"`\n2. `make bench arg=\"desired_backend`\n\nIt is recommended to run `make table`. However it requires two\nexternal dependencies\n\n\nsee the `dependencies` section for instructions for installation.\n\n\nMake sure cargo packages are on your path.\n\nEither way, the results can be seen\n\n1. run `make table arg=\"desired_flags\"`\n   - or `make bench arg=\"desired_flags`, if one does not wish to install the cargo packages\n2. sit back and watch your CPU spin\n3. The HTML results should be in `./shootout/target/criterion/reports/index.html`\n4. If `make table` was run, the updated benchmark results should be\n   seen in `./BENCHMARKS.md`\n\n\n## Benchmark Flags\n\nIn order to be effective for developers and users wishing to verify\nresults, we do not enable all benchmarks by default.\n\nThe following class of flags are had\n\n1. Compiler flags\n2. Test flags\n3. Step Flags\n\n\nAll `Step flags` and `Test flags` are enabled by default. ALl `Step flags`\non may be too slow for your workflow.\n\n| Parent Flag   | Flag             | default status | Notes            |\n|:--------------|:-----------------|:---------------|------------------|\n| all           | all              | off            |                  |\n| all_compilers | miden            | off            |                  |\n| all_compilers | triton           | off            |                  |\n| all_compilers | halo2            | off            |                  |\n| all_compilers | plonk            | off            |                  |\n| all_compilers | risc             | off            | Fails to compile |\n| all_compilers | vampir_p         | off            |                  |\n| all_compilers | vampir_halo2     | off            |                  |\n| all_tests     | sudoku           | on             |                  |\n| all_tests     | fib              | on             |                  |\n| all_tests     | blake            | on             |                  |\n| all_steps     | compile          | on             |                  |\n| all_steps     | prove            | on             |                  |\n| all_steps     | verify           | on             |                  |\n| all_steps     | prove_and_verify | on             |                  |\n\n- Any compilers that have `Fails to compile`, have to be enabled by\n  hand in arg, as they are excluded from their parent\n\n\nSo if one wants to run everything one can run\n\n```bash\nmake table arg=\"all\"\n```\n\nIn fact if one is fine with the default `on` features then one just\nneeds to run\n\n```bash\nmake table arg=\"miden\"\nmake bench arg=\"miden\"\n```\n\nwhere you can replace miden with any compiler_flag, such as `risc`.\n\nHowever if one wishes to turn off a feature like various `steps` or\n`tests`\n\nthen one has to run a command like\n\n```bash\ncd shootout\n\ncargo bench --no-default-features --features \"all_tests prove_and_verify miden\"\n```\n\nHere, we turn off all the steps, running only the `prove_and_verify`\npart of the benchmark.\n\n\n## Contributing\n\nThis repository serves as a base for contributions and so\ncontributions should hopefully be easy, and deficiencies on the\nimplementations well known.\n\nHopefully the process is rather straight forward, however we give a\nguide below to more quickly get familiar with the structure of the\nproject.\n\n\nTo get a quick `TL;DR` feeling of this you could alternatively read\nhow the `miden` backend plugs in both in the `miden.rs`, `main.rs`,\n`bench.rs`, and the `miden` sub directory for project layout. That\nshould hopefully be readable enough without having to read the full\nguide below.\n\n### Layout Structure: SRC\n\nThe project has two components of organization, the `shootout`\ndirectory has an `src` folder that hosts a file per backend. These\nfiles (`miden.rs`, `risc.rs`, etc.) are rather basic, just hosting an\nimplementation of the given backend `struct` that was created.\n\n\n```bash\n.\n├── Cargo.lock\n├── Cargo.toml\n├── src\n│   ├── bench.rs\n│   ├── halo.rs\n│   ├── main.rs\n│   ├── miden.rs\n│   ├── plonk.rs\n│   └── risc.rs\n```\n\nWe can see other files as well such as `bench.rs` and `main.rs`\n\nFor creating a new backend to test, one will have to touch\n[`bench.rs`](shootout/src/bench.rs) to have their new backend be\ntested. Since the code added is just a way to get around Rust's lack\nof typing of existentials this should be an easy task.\n\nOne should also add a feature flag for one's compiler in `Cargo.toml`.\n\nFinally, in [`main.rs`](shootout/src/main.rs) we hook up our tests to\na certain benchmark we care about.\n\n```rust\npub fn bench_sudoku(c: \u0026mut Criterion) {\n    let to_bench = vec![\n        #[cfg(feature = \"miden\")]\n        ZKP::Miden(miden::sudoku()),\n        #[cfg(feature = \"plonk\")]\n        ZKP::Plonk(plonk::sudoku()),\n        #[cfg(feature = \"risc\")]\n        ZKP::Risc0(risc::sudoku()),\n        #[cfg(feature = \"halo2\")]\n        ZKP::Halo2(halo::sudoku()),\n    ];\n    bench_zkp(c, String::from(\"Sudoku\"), to_bench)\n}\n```\n\n\nAs we can see it's just adding the backend's struct to a vector list\nand wrapped with the enum found in `bench.rs`.\n\nThis structure should also make it easy to add new benchmark programs,\nplease feel free to add new kinds of programs we wish to test here\nwith whatever backend you are interested in! Hopefully others,\nincluding myself, can contribute to other backends for your test!\n\n### Layout Structure: sub directories\n\nThe layout structures of the other directories are as follows:\n\n```bash\n.\n├── Cargo.lock\n├── Cargo.toml\n├── miden\n├── notes.org\n├── risc\n├── sudoku-halo2\n├── sudoku-plonk\n└── zero-knowledge\n    ├── Cargo.toml\n    └── src\n```\n\nThe most important folder is `zero-knowledge` as this has the trait\nneeded to implement to have the benchmarker work.\n\n```rust\npub trait ZeroKnowledge {\n    type C;\n    type R;\n    fn name(\u0026self) -\u003e String;\n    fn compile(\u0026self) -\u003e Self::C;\n    fn prove(\u0026self, setup: \u0026mut Self::C) -\u003e Self::R;\n    fn verify(\u0026self, receipt: Self::R, program: \u0026mut Self::C) -\u003e ();\n    fn prove_and_verify(\u0026self) -\u003e () {\n        let circuit = self.compile();\n        let receipt = self.prove(\u0026circuit);\n        self.verify(receipt, \u0026circuit);\n    }\n}\n```\n\nHere for a custom backend one just needs to implement `compile`,\n`prove`, `verify`, and a `name`, and your new zero knowledge backend\ncan be tested!\n\nThe other folders like `miden` and `risc`, represent stand alone\nbackends. Adding a new program to benchmark or improve a current\nbenchmark should be quite easy.\n\nthe `sudoku-halo2` and `sudoku-plonk` are similar, but represent a\nsingle program/project style structure (In the future we will likely\ntransform these to fit the mold of what `risc` and `miden` have).\n\nLastly, the `notes.org` file lays out notes about the backends and\ndeficiencies in any particular implementation, hopefully over time all\nthe programs will be optimal for their specific backend to have a more\naccurate and fair results.\n\n### Project Structure\n\nFor adding a new backend or adding new programs for backends like\n`halo2` or `plonk`. One may have to create a new sub directory.\n\nthis can be done with `cargo new`, or if you already have working\ncode, just copy and paste your code into the directory! each of the\nsub-directories are standalone independent projects with their own\n`workspace`. The only new dependency you'll have to add is:\n\n```rust\n[dependencies]\nzero-knowledge = { path = \"../zero-knowledge\" }\n```\n\nand make sure the top level `Cargo.toml` excludes the project and\nbrings it in as a dependency.\n\nIf you are importing existing code for a new backend, make sure you\nimplement the `ZeroKnowledge` trait, and you're all set!\n\nWe will now talk about backend specific considerations, feel free to\nskip these if they aren't a backend you care about contributing to\n\n### Project Structure: Miden\n\nThe miden backend can be found in the `miden` sub directory. Really\none does not have to alter this code at all, just add your program to\n`src/miden.rs` to mainly fill in the starting advice tape or the\nstarting stack, along with the file path where the miden program is\nstored.\n\nΙ personally keep my miden code in\n[miden-assembler](miden-assembler/miden), as Ι generate out miden\ncode from my Common lisp DSL in\n[programs](miden-assembler/src/programs.lisp) sub directory. All one\nneeds to run this code is\n[quick-lisp](https://www.quicklisp.org/beta/) and just write\n\n```lisp\n;; if you need to load the file by hand\n;; I will assume the repl is in the miden-assembler directory\n(load \"miden-assembler.asd\")\n;; load the project\n(ql:quickload :miden-assembler)\n;; switch to the project\n(in-package :miden)\n;; dump the code changes\n(dump)\n```\n\nHowever you may keep it wherever you feel, just make sure path is\nproperly given in the `miden.rs` file.\n\nthe rest is just hooking up your new program to `main.rs` which should\nbe straight forward.\n\n### Project Structure: Risc0\n\nRisc0 can compile straight rust code, which is nice, however this\nmeans the structure is a bit different from `miden` or other ZKVMs.\n\n```bash\n.\n├── Cargo.toml\n├── methods\n│   ├── build.rs\n│   ├── Cargo.toml\n│   ├── guest\n│   └── src\n├── README.md\n├── src\n│   ├── lib.rs\n│   └── main.rs\n├── sudoku-core\n│   ├── Cargo.toml\n│   └── src\n└── target\n```\n\nthe `src` directory contains `lib.rs` which you will only need to\ntouch the imports from the `methods` directory. the `Risc` struct\nshould be general enough to handle any program you wish to compile.\n\n```bash\n.\n├── build.rs\n├── Cargo.toml\n├── guest\n│   ├── build.rs\n│   ├── Cargo.lock\n│   ├── Cargo.toml\n│   ├── src\n│   │   └── bin\n│   │       ├── fib_fifty.rs\n│   │       ├── fib_ninty_two.rs\n│   │       ├── fib.rs\n│   │       └── sudoku.rs\n└── src\n    └── lib.rs\n```\n\ninside `methods`, you should place your code in the `bin` of\n`guest`. As you can see the names of the current programs, these\ncorrespond to the imports in `lib.rs`\n\n```rust\npub use methods_fib::{FIB_ID,           FIB_PATH,\n                      FIB_FIFTY_ID,     FIB_FIFTY_PATH,\n                      FIB_NINTY_TWO_ID, FIB_NINTY_TWO_PATH,\n                      SUDOKU_ID,        SUDOKU_PATH};\n```\n\nNote that any dependency you want to be compiled with these should be\nplaced in the `Cargo.toml` in the guest directory. This includes\nshared code like we have for `sudoku-core`, which is linked both to\nthe `guest` and to the native rust code in `src`.\n\nMuch like miden, all that is now required is to make the `Risc` struct\nin `shootout/src/risc.rs` for your new program and tell `main.rs` to\nbenchmark it!\n\n### Project Structure: Stand alone.\n\nHopefully `miden` shows a good way to create standalone projects. So\nif one has existing `halo2` or `plonk` code, one could just copy and\npaste it into a directory and implement the `ZeroKnowledge` trait. for\ntheir program.\n\n## Alucard/VAMP-IR\nPlease see the official [Alucard repository](https://github.com/anoma/juvix-circuits).\n\n## Geb\n\nSee the [Geb repository](https://github.com/anoma/geb).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanoma%2Fzkp-compiler-shootout","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanoma%2Fzkp-compiler-shootout","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanoma%2Fzkp-compiler-shootout/lists"}