{"id":13535389,"url":"https://github.com/BurntSushi/quickcheck","last_synced_at":"2025-04-02T01:30:33.568Z","repository":{"id":14837012,"uuid":"17559913","full_name":"BurntSushi/quickcheck","owner":"BurntSushi","description":"Automated property based testing for Rust (with shrinking).","archived":false,"fork":false,"pushed_at":"2025-03-08T07:18:39.000Z","size":864,"stargazers_count":2526,"open_issues_count":32,"forks_count":155,"subscribers_count":18,"default_branch":"master","last_synced_at":"2025-03-28T13:56:21.310Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BurntSushi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-03-09T07:29:09.000Z","updated_at":"2025-03-27T18:11:57.000Z","dependencies_parsed_at":"2023-01-13T18:08:55.003Z","dependency_job_id":"e83741b4-8eda-4843-9597-14f59134fab7","html_url":"https://github.com/BurntSushi/quickcheck","commit_stats":{"total_commits":526,"total_committers":87,"mean_commits":6.045977011494253,"dds":0.3060836501901141,"last_synced_commit":"aa968a94650b5d4d572c4ef581a7f5eb259aa0d2"},"previous_names":[],"tags_count":111,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurntSushi%2Fquickcheck","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurntSushi%2Fquickcheck/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurntSushi%2Fquickcheck/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BurntSushi%2Fquickcheck/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BurntSushi","download_url":"https://codeload.github.com/BurntSushi/quickcheck/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246490778,"owners_count":20786054,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T08:00:55.418Z","updated_at":"2025-04-02T01:30:33.547Z","avatar_url":"https://github.com/BurntSushi.png","language":"Rust","funding_links":[],"categories":["代码","Rust","Vulnerability Assessment","Development tools","others"],"sub_categories":["测试","Property-Based Testing","Testing"],"readme":"# quickcheck\n\nQuickCheck is a way to do property based testing using randomly generated\ninput. This crate comes with the ability to randomly generate and shrink\nintegers, floats, tuples, booleans, lists, strings, options and results.\nAll QuickCheck needs is a property function—it will then randomly generate\ninputs to that function and call the property for each set of inputs. If the\nproperty fails (whether by a runtime error like index out-of-bounds or by not\nsatisfying your property), the inputs are \"shrunk\" to find a smaller\ncounter-example.\n\nThe shrinking strategies for lists and numbers use a binary search to cover\nthe input space quickly. (It should be the same strategy used in\n[Koen Claessen's QuickCheck for\nHaskell](https://hackage.haskell.org/package/QuickCheck).)\n\n[![Build status](https://github.com/BurntSushi/quickcheck/workflows/ci/badge.svg)](https://github.com/BurntSushi/quickcheck/actions)\n[![crates.io](https://img.shields.io/crates/v/quickcheck.svg)](https://crates.io/crates/quickcheck)\n\nDual-licensed under MIT or the [UNLICENSE](https://unlicense.org/).\n\n## Documentation\n\nThe API is fully documented:\n[https://docs.rs/quickcheck](https://docs.rs/quickcheck).\n\n## Simple example\n\nHere's an example that tests a function that reverses a vector:\n\n```rust\nfn reverse\u003cT: Clone\u003e(xs: \u0026[T]) -\u003e Vec\u003cT\u003e {\n    let mut rev = vec![];\n    for x in xs.iter() {\n        rev.insert(0, x.clone())\n    }\n    rev\n}\n\n#[cfg(test)]\nmod tests {\n    use quickcheck::quickcheck;\n    use super::reverse;\n\n    quickcheck! {\n        fn prop(xs: Vec\u003cu32\u003e) -\u003e bool {\n            xs == reverse(\u0026reverse(\u0026xs))\n        }\n  }\n}\n```\n\nThis example uses the `quickcheck!` macro, which is backwards compatible with\nold versions of Rust.\n\n## The `#[quickcheck]` attribute\n\nTo make it easier to write QuickCheck tests, the `#[quickcheck]` attribute\nwill convert a property function into a `#[test]` function.\n\nTo use the `#[quickcheck]` attribute, you must import the `quickcheck` macro\nfrom the `quickcheck_macros` crate:\n\n```rust\nfn reverse\u003cT: Clone\u003e(xs: \u0026[T]) -\u003e Vec\u003cT\u003e {\n    let mut rev = vec![];\n    for x in xs {\n        rev.insert(0, x.clone())\n    }\n    rev\n}\n\n#[cfg(test)]\nmod tests {\n    use quickcheck_macros::quickcheck;\n    use super::reverse;\n\n    #[quickcheck]\n    fn double_reversal_is_identity(xs: Vec\u003cisize\u003e) -\u003e bool {\n        xs == reverse(\u0026reverse(\u0026xs))\n    }\n}\n```\n\n## Installation\n\n`quickcheck` is on `crates.io`, so you can include it in your project like so:\n\n```toml\n[dependencies]\nquickcheck = \"1\"\n```\n\nIf you're only using `quickcheck` in your test code, then you can add it as a\ndevelopment dependency instead:\n\n```toml\n[dev-dependencies]\nquickcheck = \"1\"\n```\n\nIf you want to use the `#[quickcheck]` attribute, then add `quickcheck_macros`\n\n```toml\n[dev-dependencies]\nquickcheck = \"1\"\nquickcheck_macros = \"1\"\n```\n\nN.B. When using `quickcheck` (either directly or via the attributes),\n`RUST_LOG=quickcheck` enables `info!` so that it shows useful output\n(like the number of tests passed). This is **not** needed to show\nwitnesses for failures.\n\nCrate features:\n\n- `\"use_logging\"`: (Enabled by default.) Enables the log messages governed\n  `RUST_LOG`.\n- `\"regex\"`: (Enabled by default.) Enables the use of regexes with\n  `env_logger`.\n\n## Minimum Rust version policy\n\nThis crate's minimum supported `rustc` version is `1.71.0`.\n\nThe current policy is that the minimum Rust version required to use this crate\ncan be increased in minor version updates. For example, if `crate 1.0` requires\nRust 1.20.0, then `crate 1.0.z` for all values of `z` will also require Rust\n1.20.0 or newer. However, `crate 1.y` for `y \u003e 0` may require a newer minimum\nversion of Rust.\n\nIn general, this crate will be conservative with respect to the minimum\nsupported version of Rust.\n\nWith all of that said, currently, `rand` is a public dependency of\n`quickcheck`. Therefore, the MSRV policy above only applies when it is more\naggressive than `rand`'s MSRV policy. Otherwise, `quickcheck` will defer to\n`rand`'s MSRV policy.\n\n## Compatibility\n\nIn general, this crate considers the `Arbitrary` implementations provided as\nimplementation details. Strategies may or may not change over time, which may\ncause new test failures, presumably due to the discovery of new bugs due to a\nnew kind of witness being generated. These sorts of changes may happen in\nsemver compatible releases.\n\n## Alternative Rust crates for property testing\n\nThe [`proptest`](https://docs.rs/proptest) crate is inspired by the\n[Hypothesis](https://hypothesis.works/) framework for Python.\nYou can read a comparison between `proptest` and `quickcheck`\n[here](https://github.com/proptest-rs/proptest/blob/main/proptest/README.md#differences-between-quickcheck-and-proptest)\nand\n[here](https://github.com/proptest-rs/proptest/issues/15#issuecomment-348382287).\nIn particular, `proptest` improves on the concept of shrinking. So if you've\never had problems/frustration with shrinking in `quickcheck`, then `proptest`\nmight be worth a try!\n\n## Alternatives for fuzzing\n\nPlease see the\n[Rust Fuzz Book](https://rust-fuzz.github.io/book/introduction.html)\nand the\n[`arbitrary`](https://crates.io/crates/arbitrary) crate.\n\n## Discarding test results (or, properties are polymorphic!)\n\nSometimes you want to test a property that only holds for a *subset* of the\npossible inputs, so that when your property is given an input that is outside\nof that subset, you'd discard it. In particular, the property should *neither*\npass nor fail on inputs outside of the subset you want to test. But properties\nreturn boolean values—which either indicate pass or fail.\n\nTo fix this, we need to take a step back and look at the type of the\n`quickcheck` function:\n\n```rust\npub fn quickcheck\u003cA: Testable\u003e(f: A) {\n    // elided\n}\n```\n\nSo `quickcheck` can test any value with a type that satisfies the `Testable`\ntrait. Great, so what is this `Testable` business?\n\n```rust\npub trait Testable {\n    fn result(\u0026self, \u0026mut Gen) -\u003e TestResult;\n}\n```\n\nThis trait states that a type is testable if it can produce a `TestResult`\ngiven a source of randomness. (A `TestResult` stores information about the\nresults of a test, like whether it passed, failed or has been discarded.)\n\nSure enough, `bool` satisfies the `Testable` trait:\n\n```rust\nimpl Testable for bool {\n    fn result(\u0026self, _: \u0026mut Gen) -\u003e TestResult {\n        TestResult::from_bool(*self)\n    }\n}\n```\n\nBut in the example, we gave a *function* to `quickcheck`. Yes, functions can\nsatisfy `Testable` too!\n\n```rust\nimpl\u003cA: Arbitrary + Debug, B: Testable\u003e Testable for fn(A) -\u003e B {\n    fn result(\u0026self, g: \u0026mut Gen) -\u003e TestResult {\n        // elided\n    }\n}\n```\n\nWhich says that a function satisfies `Testable` if and only if it has a single\nparameter type (whose values can be randomly generated and shrunk) and returns\nany type (that also satisfies `Testable`). So a function with type `fn(usize)\n-\u003e bool` satisfies `Testable` since `usize` satisfies `Arbitrary` and `bool`\nsatisfies `Testable`.\n\nSo to discard a test, we need to return something other than `bool`. What if we\njust returned a `TestResult` directly? That should work, but we'll need to\nmake sure `TestResult` satisfies `Testable`:\n\n```rust\nimpl Testable for TestResult {\n    fn result(\u0026self, _: \u0026mut Gen) -\u003e TestResult { self.clone() }\n}\n```\n\nNow we can test functions that return a `TestResult` directly.\n\nAs an example, let's test our reverse function to make sure that the reverse of\na vector of length 1 is equal to the vector itself.\n\n```rust\nfn prop(xs: Vec\u003cisize\u003e) -\u003e TestResult {\n    if xs.len() != 1 {\n        return TestResult::discard()\n    }\n    TestResult::from_bool(xs == reverse(\u0026xs))\n}\nquickcheck(prop as fn(Vec\u003cisize\u003e) -\u003e TestResult);\n```\n\n(A full working program for this example is in\n[`examples/reverse_single.rs`](https://github.com/BurntSushi/quickcheck/blob/master/examples/reverse_single.rs).)\n\nSo now our property returns a `TestResult`, which allows us to encode a bit\nmore information. There are a few more\n[convenience functions defined for the `TestResult`\ntype](https://docs.rs/quickcheck/*/quickcheck/struct.TestResult.html).\nFor example, we can't just return a `bool`, so we convert a `bool` value to a\n`TestResult`.\n\n(The ability to discard tests allows you to get similar functionality as\nHaskell's `==\u003e` combinator.)\n\nN.B. Since discarding a test means it neither passes nor fails, `quickcheck`\nwill try to replace the discarded test with a fresh one. However, if your\ncondition is seldom met, it's possible that `quickcheck` will have to settle\nfor running fewer tests than usual. By default, if `quickcheck` can't find\n`100` valid tests after trying `10,000` times, then it will give up.\nThese parameters may be changed using\n[`QuickCheck::tests`](https://docs.rs/quickcheck/*/quickcheck/struct.QuickCheck.html#method.tests)\nand [`QuickCheck::max_tests`](https://docs.rs/quickcheck/*/quickcheck/struct.QuickCheck.html#method.max_tests),\nor by setting the `QUICKCHECK_TESTS` and `QUICKCHECK_MAX_TESTS`\nenvironment variables.\nThere is also `QUICKCHECK_MIN_TESTS_PASSED` which sets the minimum number of\nvalid tests that need pass (defaults to `0`) in order for it to be considered a\nsuccess.\n\n## Shrinking\n\nShrinking is a crucial part of QuickCheck that simplifies counter-examples for\nyour properties automatically. For example, if you erroneously defined a\nfunction for reversing vectors as: (my apologies for the contrived example)\n\n```rust\nfn reverse\u003cT: Clone\u003e(xs: \u0026[T]) -\u003e Vec\u003cT\u003e {\n    let mut rev = vec![];\n    for i in 1..xs.len() {\n        rev.insert(0, xs[i].clone())\n    }\n    rev\n}\n```\n\nAnd a property to test that `xs == reverse(reverse(xs))`:\n\n```rust\nfn prop(xs: Vec\u003cisize\u003e) -\u003e bool {\n    xs == reverse(\u0026reverse(\u0026xs))\n}\nquickcheck(prop as fn(Vec\u003cisize\u003e) -\u003e bool);\n```\n\nThen without shrinking, you might get a counter-example like:\n\n```text\n[quickcheck] TEST FAILED. Arguments: ([-17, 13, -12, 17, -8, -10, 15, -19,\n-19, -9, 11, -5, 1, 19, -16, 6])\n```\n\nWhich is pretty mysterious. But with shrinking enabled, you're nearly\nguaranteed to get this counter-example every time:\n\n```text\n[quickcheck] TEST FAILED. Arguments: ([0])\n```\n\nWhich is going to be much easier to debug.\n\n## More Thorough Checking\n\nQuickcheck uses random input to test, so it won't\nalways find bugs that could be uncovered with a particular\nproperty. You can improve your odds of finding these latent\nbugs by spending more CPU cycles asking quickcheck to find\nthem for you. There are a few different ways to do this, and\nwhich one you choose is mostly a matter of taste.\n\nIf you are finding yourself doing this sort of thing a\nlot, you might also be interested in trying out\n[`cargo fuzz`](https://github.com/rust-fuzz/cargo-fuzz),\nwhich runs in a loop by default.\n\n### Running in a Loop\n\nOne approach is to run your quickcheck properties in a loop that\njust keeps going until you tell it to stop or it finds a bug.\nFor example, you could use a bash script such as the following\none.\n\n```bash\n#!/usr/bin/bash\n\nwhile true\ndo\n    cargo test qc_\n    if [[ x$? != x0 ]] ; then\n        exit $?\n    fi\ndone\n```\n\nOne thing to note is that this script passes the `qc_` filter to\n`cargo test`. This assumes that you've prefixed all your quickcheck\nproperties with `qc_`. You could leave off the filter, but then\nyou would be running all your deterministic tests as well, which\nwould take time away from quickcheck!\n\nChecking the return code and exiting is also important. Without that\ntest, you won't ever notice when a failure happens.\n\n### Cranking the Number of Tests\n\nAnother approach is to just ask quickcheck to run properties more\ntimes. You can do this either via the\n[tests()](https://docs.rs/quickcheck/*/quickcheck/struct.QuickCheck.html#method.tests)\nmethod, or via the `QUICKCHECK_TESTS` environment variable.\nThis will cause quickcheck to run for a much longer time. Unlike,\nthe loop approach this will take a bounded amount of time, which\nmakes it more suitable for something like a release cycle that\nwants to really hammer your software.\n\n### Making Arbitrary Smarter\n\nThis approach entails spending more time generating interesting\ninputs in your implementations of Arbitrary. The idea is to\nfocus on the corner cases. This approach can be tricky because\nprogrammers are not usually great at intuiting corner cases,\nand the whole idea of property checking is to take that burden\noff the programmer. Despite the theoretical discomfort, this\napproach can turn out to be practical.\n\n## Generating Structs\n\nIt is very simple to generate structs in QuickCheck. Consider the following\nexample, where the struct `Point` is defined:\n\n```rust\nstruct Point {\n    x: i32,\n    y: i32,\n}\n```\n\nIn order to generate a random `Point` instance, you need to implement\nthe trait `Arbitrary` for the struct `Point`:\n\n```rust\nuse quickcheck::{Arbitrary, Gen};\n\nimpl Arbitrary for Point {\n    fn arbitrary(g: \u0026mut Gen) -\u003e Point {\n        Point {\n            x: i32::arbitrary(g),\n            y: i32::arbitrary(g),\n        }\n    }\n}\n```\n\n## Case study: The Sieve of Eratosthenes\n\nThe [Sieve of Eratosthenes](https://en.wikipedia.org/wiki/Sieve_of_Eratosthenes)\nis a simple and elegant way to find all primes less than or equal to `N`.\nBriefly, the algorithm works by allocating an array with `N` slots containing\nbooleans. Slots marked with `false` correspond to prime numbers (or numbers\nnot known to be prime while building the sieve) and slots marked with `true`\nare known to not be prime. For each `n`, all of its multiples in this array\nare marked as true. When all `n` have been checked, the numbers marked `false`\nare returned as the primes.\n\nAs you might imagine, there's a lot of potential for off-by-one errors, which\nmakes it ideal for randomized testing. So let's take a look at my\nimplementation and see if we can spot the bug:\n\n```rust\nfn sieve(n: usize) -\u003e Vec\u003cusize\u003e {\n    if n \u003c= 1 {\n        return vec![];\n    }\n\n    let mut marked = vec![false; n+1];\n    marked[0] = true;\n    marked[1] = true;\n    marked[2] = true;\n    for p in 2..n {\n        for i in (2*p..n).filter(|\u0026n| n % p == 0) {\n            marked[i] = true;\n        }\n    }\n    marked.iter()\n          .enumerate()\n          .filter_map(|(i, \u0026m)| if m { None } else { Some(i) })\n          .collect()\n}\n```\n\nLet's try it on a few inputs by hand:\n\n```text\nsieve(3) =\u003e [2, 3]\nsieve(5) =\u003e [2, 3, 5]\nsieve(8) =\u003e [2, 3, 5, 7, 8] # !!!\n```\n\nSomething has gone wrong! But where? The bug is rather subtle, but it's an\neasy one to make. It's OK if you can't spot it, because we're going to use\nQuickCheck to help us track it down.\n\nEven before looking at some example outputs, it's good to try and come up with\nsome *properties* that are always satisfiable by the output of the function. An\nobvious one for the prime number sieve is to check if all numbers returned are\nprime. For that, we'll need an `is_prime` function:\n\n```rust\nfn is_prime(n: usize) -\u003e bool {\n    n != 0 \u0026\u0026 n != 1 \u0026\u0026 (2..).take_while(|i| i*i \u003c= n).all(|i| n % i != 0)\n}\n```\n\nAll this is doing is checking to see if any number in `[2, sqrt(n)]` divides\n`n` with base cases for `0` and `1`.\n\nNow we can write our QuickCheck property:\n\n```rust\nfn prop_all_prime(n: usize) -\u003e bool {\n    sieve(n).into_iter().all(is_prime)\n}\n```\n\nAnd finally, we need to invoke `quickcheck` with our property:\n\n```rust\nfn main() {\n    quickcheck(prop_all_prime as fn(usize) -\u003e bool);\n}\n```\n\nA fully working source file with this code is in\n[`examples/sieve.rs`](https://github.com/BurntSushi/quickcheck/blob/master/examples/sieve.rs).\n\nThe output of running this program has this message:\n\n```text\n[quickcheck] TEST FAILED. Arguments: (4)\n```\n\nWhich says that `sieve` failed the `prop_all_prime` test when given `n = 4`.\nBecause of shrinking, it was able to find a (hopefully) minimal counter-example\nfor our property.\n\nWith such a short counter-example, it's hopefully a bit easier to narrow down\nwhere the bug is. Since `4` is returned, it's likely never marked as being not\nprime. Since `4` is a multiple of `2`, its slot should be marked as `true` when\n`p = 2` on these lines:\n\n```rust\nfor i in (2*p..n).filter(|\u0026n| n % p == 0) {\n    marked[i] = true;\n}\n```\n\nAh! But does the `..` (range) operator include `n`? Nope! This particular\noperator is a half-open interval.\n\nA `2*p..n` range will never yield `4` when `n = 4`. When we change this to\n`2*p..n+1`, all tests pass.\n\nIn addition, if our bug happened to result in an index out-of-bounds error,\nthen `quickcheck` can handle it just like any other failure—including\nshrinking on failures caused by runtime errors.\n\nBut hold on... we're not done yet. Right now, our property tests that all\nthe numbers returned by `sieve` are prime but it doesn't test if the list is\ncomplete. It does not ensure that all the primes between `0` and `n` are found.\n\nHere's a property that is more comprehensive:\n\n```rust\nfn prop_prime_iff_in_the_sieve(n: usize) -\u003e bool {\n    sieve(n) == (0..(n + 1)).filter(|\u0026i| is_prime(i)).collect::\u003cVec\u003c_\u003e\u003e()\n}\n```\n\nIt tests that for each number between 0 and n, inclusive, the naive primality test\nyields the same result as the sieve.\n\nNow, if we run it:\n\n```rust\nfn main() {\n    quickcheck(prop_all_prime as fn(usize) -\u003e bool);\n    quickcheck(prop_prime_iff_in_the_sieve as fn(usize) -\u003e bool);\n}\n```\n\nwe see that it fails immediately for value n = 2.\n\n```text\n[quickcheck] TEST FAILED. Arguments: (2)\n```\n\nIf we inspect `sieve()` once again, we see that we mistakenly mark `2` as\nnon-prime. Removing the line `marked[2] = true;` results in both properties\npassing.\n\n## What's not in this port of QuickCheck?\n\nI think I've captured the key features, but there are still things missing:\n\n- Only functions with 8 or fewer parameters can be quickchecked. This\nlimitation can be lifted to some `N`, but requires an implementation for each\n`n` of the `Testable` trait.\n- Functions that fail because of a stack overflow are not caught by QuickCheck.\nTherefore, such failures will not have a witness attached\nto them. (I'd like to fix this, but I don't know how.)\n- `Coarbitrary` does not exist in any form in this package. It's unlikely that\nit ever will.\n- `Arbitrary` is not implemented for closures. See\n[issue #56](https://github.com/BurntSushi/quickcheck/issues/56)\nfor more details on why.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBurntSushi%2Fquickcheck","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBurntSushi%2Fquickcheck","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBurntSushi%2Fquickcheck/lists"}