{"id":25493719,"url":"https://github.com/blocksense-network/zkvms-benchmarks","last_synced_at":"2025-11-09T02:30:37.151Z","repository":{"id":278019224,"uuid":"914231368","full_name":"blocksense-network/zkVMs-benchmarks","owner":"blocksense-network","description":"Setups and programs for zero-knowledge virtual machine benchmarking","archived":false,"fork":false,"pushed_at":"2025-02-17T14:46:27.000Z","size":376,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-17T15:33:53.662Z","etag":null,"topics":["benchmark-framework","benchmark-problems","benchmark-suite","benchmarking","contribute-here","jolt","nexus","nix","nix-flake","performance","reproducible","risc0","rust","sp1","zero-knowledge","zero-knowledge-proofs","zero-knowledge-virtual-machines","zkm","zkvm","zkwasm"],"latest_commit_sha":null,"homepage":"https://blocksense.network","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/blocksense-network.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-09T07:41:06.000Z","updated_at":"2025-02-13T11:05:18.000Z","dependencies_parsed_at":"2025-02-17T15:48:09.949Z","dependency_job_id":null,"html_url":"https://github.com/blocksense-network/zkVMs-benchmarks","commit_stats":null,"previous_names":["blocksense-network/zkvms-benchmarks"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blocksense-network%2FzkVMs-benchmarks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blocksense-network%2FzkVMs-benchmarks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blocksense-network%2FzkVMs-benchmarks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blocksense-network%2FzkVMs-benchmarks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/blocksense-network","download_url":"https://codeload.github.com/blocksense-network/zkVMs-benchmarks/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239565662,"owners_count":19660158,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark-framework","benchmark-problems","benchmark-suite","benchmarking","contribute-here","jolt","nexus","nix","nix-flake","performance","reproducible","risc0","rust","sp1","zero-knowledge","zero-knowledge-proofs","zero-knowledge-virtual-machines","zkm","zkvm","zkwasm"],"created_at":"2025-02-18T23:17:11.005Z","updated_at":"2025-11-09T02:30:37.110Z","avatar_url":"https://github.com/blocksense-network.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# zkVMs-benchmarks\n\nThis is a repository with setups and programs for zero-knowledge virtual machine benchmarking.\nIts ultimate goal is to deliver reproducible and accurate performance metrics across many zkVMs, so **you** can choose which technology suits your needs!\n\nBeing made with reproducibility in mind, this project also serves as a good framework for running programs across zkVMs without the complicated and ever-changing setups required to do so.\n\n## Usage\n\nThe backbones of this entire codebase are [Nix](https://nixos.org/) and Linux.\nMacOS is not supported!\nWindows is supported via WSL.\n\n**First**, install the Nix package manager, follow their [download instructions](https://nixos.org/download/).\nGenerally it should be enough to:\n\n```sh\nsh \u003c(curl -L https://nixos.org/nix/install) --daemon\n```\n\n\u003e [!WARNING]\n\u003e It is preferable to use the nixos.org script, as shown above!\n\u003e Certain systems provide Nix with their native package managers, however practice has shown those do not always lead to working setups!\n\nNow, what follows depends on your use case.\n\n### Run/benchmark a \"built-in\" program\n\nThe `guests` directory provides a variety of programs, all of which are proven to work with their default inputs.\nIt is advisable to first try and run one of them, so you can make sure Nix is installed and working properly.\n\nThe smallest one is `fibonacci`.\nTo make *all* zkVMs generate a proof for it just run:\n\n```sh\nnix run github:blocksense-network/zkVMs-benchmarks#fibonacci -- prove\n```\n\n\u003e [!NOTE]\n\u003e Take notice of the space between `--` and `prove`!\n\u003e It marks an \"end of options\", as specified by [the POSIX specification](https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html#tag_12_02).\n\nOr to make, for example, [SP1](https://docs.succinct.xyz/docs/sp1/introduction) generate a proof and verify it, you may run:\n\n```sh\nnix run github:blocksense-network/zkVMs-benchmarks#sp1/fibonacci -- verify\n```\n\nAs you can tell, you may issue `...#ZKVM/PROGRAM ...` to execute/prove/verify a single program on the chosen zkVMs or `...#PROGRAM ...` to do the same on *all* zkVMs.\nThe format `...#ZKVM ...` is also supported, pairing the chosen zkVM with the **default** guest program (currently `graph_coloring`).\n\nThe possible values for `ZKVM` correspond to the directory names inside `zkvms` folder at the root of the repository.\nConsequently, `PROGRAM` values are connected to directory names inside `guests`.\n\n### Run/benchmark your own program\n\n1. Clone the [git repository](https://github.com/blocksense-network/zkVMs-benchmarks)\n\n   ```sh\n   git clone git@github.com:blocksense-network/zkVMs-benchmarks.git\n   ```\n\n2. Navigate to the `guests` directory\n\n   ```sh\n   cd zkVMs-benchmarks/guests\n   ```\n\n3. Follow the instructions inside the `guests/README.md` file to setup your program\n\n  \u003e [!NOTE]\n  \u003e Remember to `git add` your project!\n  \u003e Nix only sees files which are tracked by git.\n\n4. Use the `.` path as the `nix run` source.\n   So, for example, if you want to create a proof for your program with [Jolt](https://jolt.a16zcrypto.com/), you can run:\n\n   ```sh\n   nix run .#jolt/NAME -- prove\n   ```\n\n   Where `NAME` is the name of your program (inside `guests`).\n\n## Command arguments\n\nThe general format for `nix run` is:\n\n```sh\nnix run github:blocksense-network/zkVMs-benchmarks#BINARY -- PARAMETERS\n```\n\n`BINARY` is either in the form `ZKVM`, `PROGRAM` or `ZKVM/PROGRAM`.\n\nAs already discussed in \"[Run/benchmark a \"built-in\" program](#runbenchmark-a-built-in-program)\", the possible values for `ZKVM` are the subdirectory names inside `zkvms` and for `PROGRAM` are the subdirectory names inside `guests`.\nThe first form executes/proves/verifies the **default** program (currently `graph_coloring`) with the selected zkVM, the second selects a given program to be ran across **all** zkVMs and the third chooses a specific zkVM and program to act upon.\n\nAll command parameters after `--` are passed to it.\nAs a start, you should look at the built-in help message.\nFurther in this section there are some common configurations of arguments you may want to use.\n\n```sh\nnix run github:blocksense-network/zkVMs-benchmarks#sp1/fibonacci -- --help\n```\n\n```\nA CLI tool for running and benchmarking guest programs inside a zkVM environment.\nThis binary has been built with a single zkVM and guest program in mind. If you\nwant to run or benchmark your own guest program inside a zkVM, head on over to\nhttps://github.com/blocksense-network/zkVMs-benchmarks\n\nUsage: host-sp1 [OPTIONS] \u003cRUN_TYPE\u003e [PRIVATE_INPUT] [PUBLIC_INPUT]\n\nArguments:\n  \u003cRUN_TYPE\u003e       What should the zkVM do with the guest\n                   [possible values: execute, prove, verify]\n  [PRIVATE_INPUT]  Path to private input file (in TOML format)\n  [PUBLIC_INPUT]   Path to public input file (in TOML format)\n\nOptions:\n  -b, --benchmark\n          Enable benchmark timer and formatted output\n  -r, --repeat \u003cREPEAT\u003e\n          Benchmark the given action multiple times\n  -m, --millis\n          Output timings as milliseconds instead of seconds\n  -o, --metrics-output \u003cMETRICS_OUTPUT\u003e\n          Put the benchmark's formatted output into a file of the given path\n  -a, --append\n          Append the benchmark formatted output to the given file, instead of\n          replacing it\n  -h, --help\n          Print help\n```\n\n### Example: benchmark a single program\n\nAs already mentioned, if you omit a zkVM when issuing `nix run`, all zkVMs will be ran for the given program.\nHowever, when benchmarking, to get a useable output, you need to use `--metrics-output` with `--append`:\n\n```sh\nnix run github:blocksense-network/zkVMs-benchmarks#fibonacci -- prove --benchmark --metrics-output result.csv --append\n```\n\n### Example: benchmark a single program with custom input and millisecond precision\n\nExtending on the previous example, we can pass public and private input TOML files as positional arguments, after `prove`:\n\n```sh\nnix run github:blocksense-network/zkVMs-benchmarks#fibonacci -- prove ./private.toml ./public.toml -bamo result.csv\n```\n\nInput cannot be fed through stdin and no other format, except TOML, is supported.\n\n## Benchmark metrics output format\n\nWhen running with the `--benchmark` attribute, additional data is emitted - the metrics output.\nThis is a very simple CSV content, with two columns: the first a name and the second a value.\n\nHere is a table with the currently available pairs:\n\n| Name     | Value type | Shown  |\n| -------- | ---------- | ------ |\n| zkvm     | String     | Always |\n| guest    | String     | Always |\n| duration | Integer    | Always |\n| repeats  | Integer    | Always |\n| average  | Integer    | Always |\n\n### Example output\n\n```csv\nzkvm,sp1\nguest,fibonacci\nduration,4\nrepeats,1\naverage,4\n```\n\n## Limitations\n\nDue to the complicated ways in which Nix ([craneLib](https://crane.dev/)) and cargo interact, most of the packages in this repository do **not** compile without Nix.\nThis also means that incremental/debug builds are not really possible.\n\nAs of writing, user-defined input types (argument types to the entrypoint function) are not supported.\nThis also includes types defined by any libraries you may use.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblocksense-network%2Fzkvms-benchmarks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblocksense-network%2Fzkvms-benchmarks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblocksense-network%2Fzkvms-benchmarks/lists"}