{"id":22049088,"url":"https://github.com/qdrvm/erasure-coding-crust","last_synced_at":"2025-05-08T23:06:33.718Z","repository":{"id":94220561,"uuid":"565746637","full_name":"qdrvm/erasure-coding-crust","owner":"qdrvm","description":"Designing a Better World Through Decentralized Technologies","archived":false,"fork":false,"pushed_at":"2024-11-04T10:15:30.000Z","size":665,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-04T11:24:11.185Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/qdrvm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-14T08:40:04.000Z","updated_at":"2024-11-04T10:15:32.000Z","dependencies_parsed_at":"2024-11-04T11:33:39.382Z","dependency_job_id":null,"html_url":"https://github.com/qdrvm/erasure-coding-crust","commit_stats":null,"previous_names":["qdrvm/erasure-coding-crust"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qdrvm%2Ferasure-coding-crust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qdrvm%2Ferasure-coding-crust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qdrvm%2Ferasure-coding-crust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qdrvm%2Ferasure-coding-crust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/qdrvm","download_url":"https://codeload.github.com/qdrvm/erasure-coding-crust/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227351633,"owners_count":17768412,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-30T14:14:15.152Z","updated_at":"2024-11-30T14:14:15.707Z","avatar_url":"https://github.com/qdrvm.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Erasure-coding-crust\n\nC bindings over RUST implementations of [reed-solomon-novelpoly](https://github.com/paritytech/reed-solomon-novelpoly) that implements [Novel Polynomial Basis and Its Application to\nReed-Solomon Erasure Codes](https://www.citi.sinica.edu.tw/papers/whc/4454-F.pdf).\n\n## Build\n\n1. Install rust compiler (nightly), `cargo`:\n    ```bash\n    curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain nightly\n    source $HOME/.cargo/env\n    rustup install nightly\n    rustup default nightly\n    ```\n2. `mkdir build \u0026\u0026 cd build`\n3. Options;\n   - `-DTESTING=[ON|OFF]` - enable or disable build of tests.\n   - `-DCMAKE_BUILD_TYPE=[Release|Debug]` - select build type.\n   - `-DBUILD_SHARED_LIBS=[TRUE|FALSE]` - build shared/static library.\n   - `-DBENCHMARK=[ON|OFF]` - build benchmark/benchmark project\n   \n   ```bash\n   cmake .. -DCMAKE_BUILD_TYPE=Release\n   ```\n4. Build and install library: \n   ```\n   sudo make install\n   ```\n\n## Docs\n\nHeader with comments will be generated in `build/include/erasure_coding/erasure_coding.h`.\n\n## Performance (Rust vs. C++)\n\nDuring performance benchmarks we generate arbitrary data of various sizes and check its encoding and decoding durations comparing Rust and C++ implementations\n\n### Command\n\n```\ncd build\nmake benchmark\n./benchmark/benchmark\n```\n\n### Results\n#### Hardware\nAMD Ryzen 5950x, 32Gb RAM, under VM\n```\n~~~ [ Benchmark case: 15 bytes ] ~~~\nEncode RUST (100 cycles): 4 us\nDecode RUST (100 cycles): 78.775 ms\nEncode C++ (100 cycles): 4 us\nDecode C++ (100 cycles): 41.968 ms\n\n~~~ [ Benchmark case: 300 bytes ] ~~~\nEncode RUST (100 cycles): 735 us\nDecode RUST (100 cycles): 80.181 ms\nEncode C++ (100 cycles): 277 us\nDecode C++ (100 cycles): 42.517 ms\n\n~~~ [ Benchmark case: 5000 bytes ] ~~~\nEncode RUST (100 cycles): 12.815 ms\nDecode RUST (100 cycles): 97.648 ms\nEncode C++ (100 cycles): 4782 us\nDecode C++ (100 cycles): 51.095 ms\n\n~~~ [ Benchmark case: 100000 bytes ] ~~~\nEncode RUST (100 cycles): 251.146 ms\nDecode RUST (100 cycles): 422.756 ms\nEncode C++ (100 cycles): 100.051 ms\nDecode C++ (100 cycles): 220.24 ms\n\n~~~ [ Benchmark case: 1000000 bytes ] ~~~\nEncode RUST (100 cycles): 2526.03 ms\nDecode RUST (100 cycles): 3501.28 ms\nEncode C++ (100 cycles): 987.493 ms\nDecode C++ (100 cycles): 1770.36 ms\n\n~~~ [ Benchmark case: 10000000 bytes ] ~~~\nEncode RUST (100 cycles): 25.1685 s\nDecode RUST (100 cycles): 34.4632 s\nEncode C++ (100 cycles): 9.71128 s\nDecode C++ (100 cycles): 17.5619 s\n```\n\nApple M1 Pro, 16Gb Ram, native\n\n```\n~~~ [ Benchmark case: 22 bytes ] ~~~\nEncode RUST (100 cycles): 124 us\nDecode RUST (100 cycles): 66.506 ms\nEncode C++ (100 cycles): 31 us\nDecode C++ (100 cycles): 26.844 ms\n\n~~~ [ Benchmark case: 303 bytes ] ~~~\nEncode RUST (100 cycles): 847 us\nDecode RUST (100 cycles): 59.614 ms\nEncode C++ (100 cycles): 612 us\nDecode C++ (100 cycles): 27.749 ms\n\n~~~ [ Benchmark case: 5007 bytes ] ~~~\nEncode RUST (100 cycles): 14.157 ms\nDecode RUST (100 cycles): 78.362 ms\nEncode C++ (100 cycles): 10.553 ms\nDecode C++ (100 cycles): 43.531 ms\n\n~~~ [ Benchmark case: 100015 bytes ] ~~~\nEncode RUST (100 cycles): 282.544 ms\nDecode RUST (100 cycles): 470.959 ms\nEncode C++ (100 cycles): 211.462 ms\nDecode C++ (100 cycles): 364.47 ms\n\n~~~ [ Benchmark case: 1000015 bytes ] ~~~\nEncode RUST (100 cycles): 2835.5 ms\nDecode RUST (100 cycles): 4204.71 ms\nEncode C++ (100 cycles): 2133.94 ms\nDecode C++ (100 cycles): 3418.58 ms\n\n~~~ [ Benchmark case: 10000015 bytes ] ~~~\nEncode RUST (100 cycles): 28.5276 s\nDecode RUST (100 cycles): 41.6977 s\nEncode C++ (100 cycles): 21.6447 s\nDecode C++ (100 cycles): 34.2825 s\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqdrvm%2Ferasure-coding-crust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqdrvm%2Ferasure-coding-crust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqdrvm%2Ferasure-coding-crust/lists"}