{"id":13546077,"url":"https://github.com/loiclec/fuzzcheck-rs","last_synced_at":"2025-04-02T17:32:33.233Z","repository":{"id":43794106,"uuid":"202733183","full_name":"loiclec/fuzzcheck-rs","owner":"loiclec","description":"Modular, structure-aware, and feedback-driven fuzzing engine for Rust functions","archived":false,"fork":false,"pushed_at":"2025-02-24T13:55:24.000Z","size":3130,"stargazers_count":441,"open_issues_count":15,"forks_count":10,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-03-27T03:38:50.305Z","etag":null,"topics":["coverage-guided-fuzzing","fuzzer","fuzzing","grammar-fuzzer","property-based-testing","rust","testing"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/loiclec.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":null},"created_at":"2019-08-16T13:33:15.000Z","updated_at":"2025-02-24T13:55:27.000Z","dependencies_parsed_at":"2022-08-25T20:22:08.819Z","dependency_job_id":"ef0c68d0-b3c7-4370-a43f-9a7ca0f22f9f","html_url":"https://github.com/loiclec/fuzzcheck-rs","commit_stats":{"total_commits":896,"total_committers":6,"mean_commits":"149.33333333333334","dds":0.4029017857142857,"last_synced_commit":"cd8ab38b31d757165108f1c9140a7294bfa5f281"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loiclec%2Ffuzzcheck-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loiclec%2Ffuzzcheck-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loiclec%2Ffuzzcheck-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loiclec%2Ffuzzcheck-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/loiclec","download_url":"https://codeload.github.com/loiclec/fuzzcheck-rs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246860247,"owners_count":20845634,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coverage-guided-fuzzing","fuzzer","fuzzing","grammar-fuzzer","property-based-testing","rust","testing"],"created_at":"2024-08-01T12:00:31.149Z","updated_at":"2025-04-02T17:32:32.221Z","avatar_url":"https://github.com/loiclec.png","language":"Rust","funding_links":[],"categories":["Vulnerability Assessment","Rust","Dynamic Checkers"],"sub_categories":["Fuzzing"],"readme":"# Fuzzcheck\n\n[![CI](https://github.com/loiclec/fuzzcheck-rs/actions/workflows/cargo.yml/badge.svg)](https://github.com/loiclec/fuzzcheck-rs/actions/workflows/cargo.yml)\n[![Docs](https://img.shields.io/docsrs/fuzzcheck?color=blueviolet)](https://docs.rs/fuzzcheck)\n[![MIT licensed](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE.txt)\n[![crates.io](https://img.shields.io/crates/v/fuzzcheck)](https://crates.io/crates/fuzzcheck)\n\nFuzzcheck is a modular, structure-aware, and feedback-driven fuzzing engine for Rust \nfunctions. \n\nGiven a function `test: (T) -\u003e bool`, you can use fuzzcheck to find a value of\ntype `T` that fails the test or leads to a crash. \n\nThe tool [`fuzzcheck-view`](https://github.com/loiclec/fuzzcheck-view) is available\nto visualise the code coverage of each/all test cases generated by fuzzcheck. It is still just\na prototype though.\n\nFollow the [guide at fuzzcheck.neocities.org](https://fuzzcheck.neocities.org) to get \nstarted or read [the documentation on docs.rs](https://docs.rs/fuzzcheck).\n\n## Setup\n\nLinux or macOS is required. [Windows support is planned but I need help with it](https://github.com/loiclec/fuzzcheck-rs/issues/8).\n\nRust nightly is also required. You can install it with:\n```\nrustup toolchain install nightly\n```\n\nWhile it is not strictly necessary, installing the `cargo-fuzzcheck` \nexecutable will make it easier to run fuzzcheck.\n```bash\ncargo install cargo-fuzzcheck\n```\n\nIn your `Cargo.toml` file, add `fuzzcheck` as a dev-dependency:\n```toml\n[dev-dependencies]\nfuzzcheck = \"0.12\"\n```\n\nThen, we need a way to serialize values. By default, fuzzcheck uses `serde_json`\nfor that purpose (but it can be changed). That means our data types should \nimplement serde's traits. In `Cargo.toml`, add:\n```\n[dependencies]\nserde = { version = \"1.0\", features = [\"derive\"] }\n```\n\n## Usage\n\nBelow is an example of how to use fuzz test. Note:\n1. every code related to fuzzcheck is conditional on `#[cfg(test)]` because we \ndon't want to carry the fuzzcheck dependency in normal builds\n2. the `#![cfg_attr(test, feature(no_coverage))]` is required by fuzzcheck’s procedural macros\n3. the use of `derive(fuzzcheck::DefaultMutator)` makes a custom type fuzzable\n\n```rust\n#![cfg_attr(fuzzing, feature(no_coverage))]\nuse serde::{Deserialize, Serialize};\n\n#[cfg_attr(fuzzing, derive(fuzzcheck::DefaultMutator))]\n#[derive(Clone, Serialize, Deserialize)]\nstruct SampleStruct\u003cT, U\u003e {\n    x: T,\n    y: U,\n}\n\n#[cfg_attr(fuzzing, derive(fuzzcheck::DefaultMutator))]\n#[derive(Clone, Serialize, Deserialize)]\nenum SampleEnum {\n    A(u16),\n    B,\n    C { x: bool, y: bool },\n}\n\nfn should_not_crash(xs: \u0026[SampleStruct\u003cu8, SampleEnum\u003e]) {\n    if xs.len() \u003e 3\n        \u0026\u0026 xs[0].x == 100\n        \u0026\u0026 matches!(xs[0].y, SampleEnum::C { x: false, y: true })\n        \u0026\u0026 xs[1].x == 55\n        \u0026\u0026 matches!(xs[1].y, SampleEnum::C { x: true, y: false })\n        \u0026\u0026 xs[2].x == 87\n        \u0026\u0026 matches!(xs[2].y, SampleEnum::C { x: false, y: false })\n        \u0026\u0026 xs[3].x == 24\n        \u0026\u0026 matches!(xs[3].y, SampleEnum::C { x: true, y: true })\n    {\n        panic!()\n    }\n}\n\n// fuzz tests reside along your other tests and have the #[test] attribute\n#[cfg(all(fuzzing, test))]\nmod tests {\n    #[test]\n    fn test_function_shouldn_t_crash() {\n        let result = fuzzcheck::fuzz_test(super::should_not_crash) // the test function to fuzz\n            .default_mutator() // the mutator to generate values of \u0026[SampleStruct\u003cu8, SampleEnum\u003e]\n            .serde_serializer() // save the test cases to the file system using serde\n            .default_sensor_and_pool() // gather observations using the default sensor (i.e. recording code coverage)\n            .arguments_from_cargo_fuzzcheck() // take arguments from the cargo-fuzzcheck command line tool\n            .stop_after_first_test_failure(true) // stop the fuzzer as soon as a test failure is found\n            .launch();\n        assert!(!result.found_test_failure);\n    }\n}\n```\n\nWe can now use `cargo-fuzzcheck` to launch the test, using Rust nightly:\n```sh\nrustup override set nightly\n# the argument is the *exact* path to the test function\ncargo fuzzcheck tests::test_function_shouldn_t_crash\n```\n\nThis starts a loop that will stop when a failing test has been found. After about ~50ms of fuzz-testing on my machine, \nthe following line is printed:\n```\nFailing test case found. Saving at \"fuzz/tests::test_function_shouldn_t_crash/artifacts/59886edc1de2dcc1.json\"\n```\nThe file `59886edc1de2dcc1.json` contains the JSON-encoded input that failed the test.\n\n```json\n[\n  {\n    \"x\": 100,\n    \"y\": {\n      \"C\": {\n        \"x\": false,\n        \"y\": true\n      }\n    }\n  },\n  {\n    \"x\": 55,\n    \"y\": {\n      \"C\": {\n        \"x\": true,\n        \"y\": false\n      }\n    }\n  },\n  ..\n]\n```\n\n## Minifying failing test inputs\n\nFuzzcheck can also be used to *minify* a large input that fails a test.\nIf the failure is recoverable (i.e. it is not a segfault/stack overflow), and \nthe fuzzer is not instructed to stop after the first failure, then the failing\ntest cases will be minified automatically. Otherwise, you can use the `minify`\ncommand.\n\nLet's say you have a file `crash.json` containing an input that you would like\nto minify. Launch `cargo fuzzcheck \u003cexact name of fuzz test\u003e` with the `minify` command\nand an `--input-file` option.\n\n```bash\ncargo fuzzcheck \"tests::test_function_shouldn_t_crash\" --command minify --input-file \"crash.json\"\n```\n\nThis will repeatedly launch the fuzzer in “minify” mode and save the\nartifacts in the folder `artifacts/crash.minified`. The name of each artifact \nwill be prefixed with the complexity of its input. For example,\n`crash.minified/800--fe958d4f003bd4f5.json` has a complexity of `8.00`.\n\nYou can stop the minifying fuzzer at any point and look for the least complex\ninput in the `crash.minified` folder.\n\n## Alternatives\n\nOther crates with the same goal are [`quickcheck`](https://crates.io/crates/quickcheck) \nand [`proptest`](https://crates.io/crates/proptest). Fuzzcheck can be more powerful \nthan these because it guides the generation of test cases based on feedback\ngenerated from running the test function. This feedback is most often code coverage,\nbut can be different.\n\nAnother similar crate is [`cargo-fuzz`](https://crates.io/crates/cargo-fuzz), often paired \nwith [`arbitrary`](https://crates.io/crates/arbitrary). In this case, \nfuzzcheck has an advantage by being easier to use, more modular, and being more \nfundamentally structure-aware and thus potentially more efficient.\n\n## Previous work on fuzzing engines\n\nAs far as I know, evolutionary, coverage-guided fuzzing engines were\npopularized by [American Fuzzy Lop (AFL)](http://lcamtuf.coredump.cx/afl/).  \nFuzzcheck is also evolutionary and coverage-guided.\n\nLater on, LLVM released its own fuzzing engine, \n[libFuzzer](https://www.llvm.org/docs/LibFuzzer.html), which is based on the\nsame ideas as AFL, but it uses Clang’s \n[SanitizerCoverage](https://clang.llvm.org/docs/SanitizerCoverage.html) and is\nin-process (it lives in the same process as the program being fuzz-tested.  \nFuzzcheck is also in-process. It uses rustc’s `-Z instrument-coverage` option \ninstead of SanitizerCoverage for code coverage instrumentation.\n\nBoth AFL and libFuzzer work by manipulating bitstrings (e.g. `1011101011`).\nHowever, many programs work on structured data, and mutations at the\nbitstring level may not map to meaningful mutations at the level of the\nstructured data. This problem can be partially addressed by using a compact\nbinary encoding such as protobuf and providing custom mutation functions to\nlibFuzzer that work on the structured data itself. This is a way to perform\n“structure-aware fuzzing” ([talk](https://www.youtube.com/watch?v=U60hC16HEDY),\n[tutorial](https://github.com/google/fuzzer-test-suite/blob/master/tutorial/structure-aware-fuzzing.md)).\n\nAn alternative way to deal with structured data is to use generators just like\nQuickCheck’s `Arbitrary` trait. And then to “treat the raw byte buffer input \nprovided by the coverage-guided fuzzer as a sequence of random values and\nimplement a “random” number generator around it.” \n([cited blog post by @fitzgen](https://fitzgeraldnick.com/2019/09/04/combining-coverage-guided-and-generation-based-fuzzing.html)). \nThe tool `cargo-fuzz` has\n[recently](https://fitzgeraldnick.com/2020/01/16/better-support-for-fuzzing-structured-inputs-in-rust.html) \nimplemented that approach.\n\nFuzzcheck is also structure-aware, but unlike previous attempts at\nstructure-aware fuzzing, it doesn't use an intermediary binary encoding such as\nprotobuf nor does it use Quickcheck-like generators.\nInstead, it directly mutates the typed values in-process.\nThis is better many ways. First, it is faster because there is no\nneed to encode and decode inputs at each iteration. Second, the complexity of\nthe input is given by a user-defined function, which will be more accurate than\ncounting the bytes of the protobuf encoding.\nFinally, and most importantly, the mutations are faster and more meaningful \nthan those done on protobuf or `Arbitrary`’s byte buffer-based RNG.\nA detail that I particularly like about fuzzcheck, and that is possible only \nbecause it mutates typed values, is that every mutation is done **in-place**\nand is reversable. That means that generating a new test case is super fast, \nand can often even be done with zero allocations.\n\nAs I was developing Fuzzcheck for Swift, a few researchers developed Fuzzchick\nfor Coq ([paper](https://www.cs.umd.edu/~mwh/papers/fuzzchick-draft.pdf)). It \nis a coverage-guided property-based testing tool implemented as an extension to\nQuickchick. As far as I know, it is the only other tool with the same philosophy\nas fuzzcheck. The similarity between the names `fuzzcheck` and `Fuzzchick` is a \ncoincidence.\n\n[LibAFL](https://github.com/AFLplusplus/LibAFL) is another modular fuzzer written\nin Rust. It was released relatively recently.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floiclec%2Ffuzzcheck-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Floiclec%2Ffuzzcheck-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floiclec%2Ffuzzcheck-rs/lists"}