{"id":13687327,"url":"https://github.com/scipr-lab/dizk","last_synced_at":"2025-12-30T09:26:33.882Z","repository":{"id":41308901,"uuid":"141190586","full_name":"scipr-lab/dizk","owner":"scipr-lab","description":"Java library for distributed zero knowledge proof systems","archived":false,"fork":false,"pushed_at":"2022-04-13T19:55:42.000Z","size":179,"stargazers_count":230,"open_issues_count":9,"forks_count":66,"subscribers_count":34,"default_branch":"master","last_synced_at":"2024-08-03T15:06:27.159Z","etag":null,"topics":["cryptography","distributed-computing","zero-knowledge-proofs","zksnarks"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scipr-lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-07-16T20:25:23.000Z","updated_at":"2024-07-22T23:43:34.000Z","dependencies_parsed_at":"2022-08-10T01:53:59.881Z","dependency_job_id":null,"html_url":"https://github.com/scipr-lab/dizk","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scipr-lab%2Fdizk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scipr-lab%2Fdizk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scipr-lab%2Fdizk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scipr-lab%2Fdizk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scipr-lab","download_url":"https://codeload.github.com/scipr-lab/dizk/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224257588,"owners_count":17281744,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cryptography","distributed-computing","zero-knowledge-proofs","zksnarks"],"created_at":"2024-08-02T15:00:52.577Z","updated_at":"2025-12-30T09:26:33.848Z","avatar_url":"https://github.com/scipr-lab.png","language":"Java","funding_links":[],"categories":["SNARKs","安全"],"sub_categories":["Tools","Scaling the prover"],"readme":"\u003ch1 align=\"center\"\u003eDIZK\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://travis-ci.org/scipr-lab/dizk\"\u003e\u003cimg src=\"https://travis-ci.org/scipr-lab/dizk.svg?branch=master\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/scipr-lab/dizk/blob/master/AUTHORS\"\u003e\u003cimg src=\"https://img.shields.io/badge/authors-SCIPR%20Lab-orange.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/scipr-lab/dizk/blob/master/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-MIT-blue.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003ch4 align=\"center\"\u003eJava library for distributed zero knowledge proof systems\u003c/h4\u003e\n\n___DIZK___ (pronounced */'dizək/*) is a Java library for distributed zero knowledge proof systems. The library implements distributed polynomial evaluation/interpolation, computation of Lagrange polynomials, and multi-scalar multiplication. Using these scalable arithmetic subroutines, the library provides a distributed zkSNARK proof system that enables verifiable computations of up to billions of logical gates, far exceeding the scale of previous state-of-the-art solutions.\n\nThe library is developed by [SCIPR Lab](http://www.scipr-lab.org/) and contributors (see [AUTHORS](AUTHORS) file) and is released under the MIT License (see [LICENSE](LICENSE) file).\n\nThe library is developed as part of a paper called *\"[DIZK: A Distributed Zero Knowledge Proof System](https://eprint.iacr.org/2018/691)\"*.\n\n**WARNING:** This is an academic proof-of-concept prototype. This implementation is not ready for production use. It does not yet contain all the features, careful code review, tests, and integration that are needed for a deployment!\n\n## Table of contents\n\n- [Directory structure](#directory-structure)\n- [Overview](#overview)\n- [Build guide](#build-guide)\n- [Profiler](#profiler)\n- [Benchmarks](#benchmarks)\n- [References](#references)\n- [License](#license)\n\n## Directory structure\n\nThe directory structure is as follows:\n\n* [__src__](src): Java directory for source code and unit tests\n  * [__main/java__](src/main/java): Java source code, containing the following modules:\n    * [__algebra__](src/main/java/algebra): fields, groups, elliptic curves, FFT, multi-scalar multiplication\n    * [__bace__](src/main/java/bace): batch arithmetic circuit evaluation\n    * [__common__](src/main/java/common): standard arithmetic and Spark computation utilities\n    * [__configuration__](src/main/java/configuration): configuration settings for the Spark cluster\n    * [__profiler__](src/main/java/profiler): profiling infrastructure for zero-knowledge proof systems\n    * [__reductions__](src/main/java/reductions): reductions between languages (used internally)\n    * [__relations__](src/main/java/relations): interfaces for expressing statement (relations between instances and witnesses) as various NP-complete languages\n    * [__zk_proof_systems__](src/main/java/zk_proof_systems): serial and distributed implementations of zero-knowledge proof systems\n  * [__test/java__](src/test/java): Java unit tests for the provided modules and infrastructure\n\n## Overview\n\nThis library implements a distributed zero knowledge proof system, enabling scalably proving (and verifying) the integrity of computations, in zero knowledge.\n\nA prover who knows the witness for an NP statement (i.e., a satisfying input/assignment) can produce a short proof attesting to the truth of the NP statement. This proof can then be verified by anyone, and offers the following properties.\n\n- **Zero knowledge** - the verifier learns nothing from the proof besides the truth of the statement.\n- **Succinctness** - the proof is small in size and cheap to verify.\n- **Non-interactivity** - the proof does not require back-and-forth interaction between the prover and the verifier.\n- **Soundness** - the proof is computationally sound (such a proof is called an *argument*).\n- **Proof of knowledge** - the proof attests not just that the NP statement is true, but also that the prover knows why.\n\nThese properties comprise a **zkSNARK**, which stands for *Zero-Knowledge Succinct Non-interactive ARgument of Knowledge*.\nFor formal definitions and theoretical discussions about these, see [BCCT12] [BCIOP13] and the references therein.\n\n**DIZK** provides Java-based implementations using Apache Spark [Apa17] for:\n\n1. Proof systems\n    - A serial and distributed preprocessing zkSNARK for *R1CS* (Rank-1 Constraint Systems), an NP-complete language that resembles arithmetic circuit satisfiability. The zkSNARK is the protocol in [Gro16].\n    - A distributed Merlin-Arthur proof system for evaluating arithmetic circuits on batches of inputs; see [Wil16].\n2. Scalable arithmetic\n    - A serial and distributed radix-2 fast Fourier transform (FFT); see [Sze11].\n    - A serial and distributed multi-scalar multiplication (MSM); see [BGMW93] [Pip76] [Pip80].\n    - A serial and distributed Lagrange interpolation (Lag); see [BT04].\n3. Applications using the above zkSNARK for\n    - Authenticity of photos on three transformations (crop, rotation, blur); see [NT16].\n    - Integrity of machine learning models with support for linear regression and covariance matrices; see [Bis06] [Can69] [LRF97] [vW97].\n\n## Build guide\n\nThe library has the following dependencies:\n\n- [Java SE 8+](http://www.oracle.com/technetwork/java/javase/overview/index.html)\n- [Apache Maven](https://maven.apache.org/)\n- Fetched from `pom.xml` via Maven:\n    - [Spark Core 2.10](https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.10/1.0.0)\n    - [Spark SQL 2.10](https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.10/2.1.0)\n    - [JUnit 4.11](https://mvnrepository.com/artifact/junit/junit/4.11)\n    - [Google Java Format](https://github.com/google/google-java-format)\n- Fetched via Git submodules:\n    - [spark-ec2](https://github.com/amplab/spark-ec2/tree/branch-2.0)\n    \n### Why Java?\n\nThis library uses Apache Spark, an open-source cluster-computing framework that natively supports Java, Scala, and Python. Among these, we found Java to fit our goals because we could leverage its rich features for object-oriented programming and we could control execution in a (relatively) fine-grained way.\n\nWhile other libraries for zero knowledge proof systems are written in low-level languages (e.g., [libsnark](https://github.com/scipr-lab/libsnark) is written in C++ and [bellman](https://github.com/zkcrypto/bellman) in Rust), harnessing the speed of such languages in our setting is not straightforward. For example, we evaluated the possibility of interfacing with C (using native binding approaches like JNI and JNA), and concluded that the cost of memory management and process inferfacing resulted in a slower performance than from purely native Java execution.\n\n### Installation\n\nStart by cloning this repository and entering the repository working directory:\n```$xslt\ngit clone https://github.com/scipr-lab/dizk.git\ncd dizk\n```\n\nNext, fetch the dependency modules:\n```$xslt\ngit submodule init \u0026\u0026 git submodule update\n```\n\nFinally, compile the source code:\n```$xslt\nmvn compile\n```\n\n### Docker\n\n```\ncd your_dizk_project_directory\n\ndocker build -t dizk-container .\ndocker run -it dizk-container bash\n```\n\n### Testing\n\nThis library comes with unit tests for each of the provided modules. Run the tests with:\n```$xslt\nmvn test\n``` \n\n## Profiler\n\nUsing Amazon EC2, the profiler benchmarks the performance of serial and distributed zero-knowledge proof systems, as well as its underlying primitives.\nThe profiler uses `spark-ec2` to manage the cluster compute environment and a set of provided scripts for launch, profiling, and shutdown.\n\n### Spark EC2\n\nTo manage the cluster compute environment, DIZK uses [`spark-ec2@branch-2.0`](https://github.com/amplab/spark-ec2/tree/branch-2.0).\n`spark-ec2` is a tool to launch, maintain, and terminate [Apache Spark](https://spark.apache.org/docs/latest/) clusters on Amazon EC2.\n\nTo setup `spark-ec2`, run the following commands:\n```$xslt\ngit clone https://github.com/amplab/spark-ec2.git\ncd spark-ec2\ngit checkout branch-2.0\npwd\n```\n\nRemember where the directory for `spark-ec2` is located, as this will need to be provided as an environment variable for the scripts as part of the next step.\n\n### Profiling scripts\n\nTo begin, set the environment variables required to initialize the profiler in [init.sh](src/main/java/profiler/scripts/init.sh).\nThe profiling infrastructure will require access to an AWS account access key and secret key, which can be created with\nthe [instructions provided by AWS](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).\n\n```$xslt\nexport AWS_ACCESS_KEY_ID={Insert your AWS account access key}\nexport AWS_SECRET_ACCESS_KEY={Insert your AWS account secret key}\n\nexport AWS_KEYPAIR_NAME=\"{Insert your AWS keypair name, e.g. spark-ec2-oregon}\"\nexport AWS_KEYPAIR_PATH=\"{Insert the path to your AWS keypair .pem file, e.g. /Users/johndoe/Downloads/spark-ec2-oregon.pem}\"\n\nexport AWS_REGION_ID={Insert your AWS cluster region choice, e.g. us-west-2}\nexport AWS_CLUSTER_NAME={Insert your AWS cluster name, e.g. spark-ec2}\n\nexport SPOT_PRICE={Insert your spot price for summoning an EC2 instance, e.g. 0.1}\nexport SLAVES_COUNT={Insert the number of EC2 instances to summon for the cluster, e.g. 2}\nexport INSTANCE_TYPE={Insert the instance type you would like to summon, e.g. r3.large}\n\nexport DIZK_REPO_PATH=\"{Insert the path to your local DIZK repository, e.g. /Users/johndoe/dizk}\"\nexport SPARK_EC2_PATH=\"{Insert the path to your local spark-ec2 repository, e.g. /Users/johndoe/dizk/depends/spark-ec2}\"\n```\n\nNext, start the profiler by running:\n```$xslt\n./launch.sh\n```\n\nThe launch script uses `spark-ec2` and the environment variables to setup the initial cluster environment.\nThis process takes around 20-30 minutes depending on the choice of cluster configuration.\n\nAfter the launch is complete, upload the DIZK JAR file to the master node and SSH into the cluster with the following command:\n```$xslt\n./upload_and_login.sh\n```\n\nOnce you have successfully logged in to the cluster, navigate to the uploaded `scripts` folder and setup the initial cluster environment.\n\n```$xslt\ncd ../scripts\n./setup_environment.sh\n```\n\nThis creates a logging directory for Spark events and installs requisite dependencies, such as Java 8.\n\nLastly, with the cluster environment fully setup, set the desired parameters for benchmarking in [profile.sh](src/main/java/profiler/scripts/profile.sh) and run the following command to begin profiling:\n```$xslt\n./profile.sh\n```\n\n## Benchmarks\n\nWe evaluate the distributed implementation of the zkSNARK setup and prover.\nBelow we use *instance size* to denote the number of constraints in an R1CS instance.\n\n### libsnark *vs* DIZK\n\nWe measure the largest instance size (as a power of 2) that is supported by:\n\n- the serial implementation of PGHR’s protocol in [libsnark](https://github.com/scipr-lab/libsnark)\n- the serial implementation of Groth’s protocol in [libsnark](https://github.com/scipr-lab/libsnark)\n- the distributed implementation of Groth's protocol in **DIZK**\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://user-images.githubusercontent.com/9260812/43099291-9203db9a-8e76-11e8-8d68-528d903500e1.png\" width=\"68%\"\u003e\u003c/p\u003e\n\nWe see that using more executors allows us to support larger instance sizes,\nin particular supporting billions of constraints with sufficiently many executors.\nInstances of this size are much larger than what was previously possible via serial techniques.\n\n### Distributed zkSNARK\n\nWe benchmark the running time of the setup and the prover on an increasing number of constraints and with an increasing number of executors.\nNote that we do not need to evaluate the zkSNARK verifier as it is a simple and fast algorithm that can be run even on a smartphone.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://user-images.githubusercontent.com/9260812/43099290-91ec40c0-8e76-11e8-8391-c30fbddc4acd.png\" width=\"67%\"\u003e\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://user-images.githubusercontent.com/9260812/43099289-91d1d2b2-8e76-11e8-9a25-f06103903290.png\" width=\"59%\"\u003e\u003c/p\u003e\n\nOur benchmarks of the setup and the prover show us that:\n \n1. For a given number of executors, running times increase nearly linearly as expected, demonstrating scalability over a wide range of instance sizes.\n\n2. For a given instance size, running times decrease nearly linearly as expected, demonstrating parallelization over a wide range of number of executors.\n\n## References\n\n[Apa17] [_Apache Spark_](http://spark.apache.org/),\nApache Spark,\n2017\n\n[Bis06] [_Pattern recognition and machine learning_](https://www.springer.com/us/book/9780387310732),\nChristopher M. Bishop,\n*Book*, 2006\n\n[BCCT12] [_From extractable collision resistance to succinct non-interactive arguments of knowledge, and back again_](http://eprint.iacr.org/2011/443),\nNir Bitansky, Ran Canetti, Alessandro Chiesa, Eran Tromer,\n*Innovations in Theoretical Computer Science* (ITCS), 2012\n\n[BCIOP13] [_Succinct non-interactive arguments via linear interactive proofs_](http://eprint.iacr.org/2012/718),\nNir Bitansky, Alessandro Chiesa, Yuval Ishai, Rafail Ostrovsky, Omer Paneth,\n*Theory of Cryptography Conference* (TCC), 2013\n\n[BGMW93] [_Fast exponentiation with precomputation_](https://link.springer.com/chapter/10.1007/3-540-47555-9_18),\nErnest F. Brickell, Daniel M. Gordon, Kevin S. McCurley, and David B. Wilson,\n*International Conference on the Theory and Applications of Cryptographic Techniques* (EUROCRYPT), 1992\n\n[BT04] [_Barycentric Lagrange interpolation_](https://people.maths.ox.ac.uk/trefethen/barycentric.pdf),\nJean-Paul Berrut and Lloyd N. Trefethen,\n*SIAM Review*, 2004\n\n[Can69] [_A cellular computer to implement the Kalman filter algorithm_](https://dl.acm.org/citation.cfm?id=905686),\nLynn E Cannon,\n*Doctoral Dissertation*, 1969\n\n[Gro16] [_On the size of pairing-based non-interactive arguments_](https://eprint.iacr.org/2016/260.pdf),\nJens Groth,\n*International Conference on the Theory and Applications of Cryptographic Techniques (EUROCRYPT)*, 2016\n\n[LRF97] [_Generalized cannon’s algorithm for parallel matrix multiplication_](https://dl.acm.org/citation.cfm?id=263591),\nHyuk-Jae Lee, James P. Robertson, and Jose A. B. Fortes,\n*International Conference on Supercomputing*, 1997\n\n[NT16] [_Photoproof: Cryptographic image authentication for any set of permissible transformations_](https://www.cs.tau.ac.il/~tromer/papers/photoproof-oakland16.pdf),\nAssa Naveh and Eran Tromer,\n*IEEE Symposium on Security and Privacy*, 2016\n\n[Pip76] [_On the evaluation of powers and related problems_](https://ieeexplore.ieee.org/document/4567910/),\nNicholas Pippenger,\n*Symposium on Foundations of Computer Science* (FOCS), 1976\n\n[Pip80] [_On the evaluation of powers and monomials_](https://pdfs.semanticscholar.org/7d65/53e185fd90a855717ee915992e17f38c99ae.pdf),\nNicholas Pippenger,\n*SIAM Journal on Computing*, 1980\n\n[Sze11] [_Schönhage-Strassen algorithm with MapReduce for multiplying terabit integers_](https://people.apache.org/~szetszwo/ssmr20110429.pdf),\nTsz-Wo Sze,\n*International Workshop on Symbolic-Numeric Computation*, 2011\n\n[vW97] [_SUMMA: scalable universal matrix multiplication algorithm_](https://dl.acm.org/citation.cfm?id=899248),\nRobert A. van de Geijn and Jerrell Watts,\n*Technical Report*, 1997\n\n[Wil16] [_Strong ETH breaks with Merlin and Arthur: short non-interactive proofs of batch evaluation_](https://arxiv.org/pdf/1601.04743.pdf),\nRyan Williams,\n*Conference on Computational Complexity*, 2016\n\n## Acknowledgements\n\nThis work was supported by Intel/NSF CPS-Security grants,\nthe [UC Berkeley Center for Long-Term Cybersecurity](https://cltc.berkeley.edu/),\nand gifts to the [RISELab](https://rise.cs.berkeley.edu/) from Amazon, Ant Financial, CapitalOne, Ericsson, GE, Google, Huawei, IBM, Intel, Microsoft, and VMware.\nThe authors thank Amazon for donating compute credits to RISELab, which were extensively used in this project.\n\n## License\n\n[MIT License](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscipr-lab%2Fdizk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscipr-lab%2Fdizk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscipr-lab%2Fdizk/lists"}