{"id":18016993,"url":"https://github.com/fitzgen/preduce","last_synced_at":"2025-10-17T20:12:08.296Z","repository":{"id":66008106,"uuid":"80450384","full_name":"fitzgen/preduce","owner":"fitzgen","description":"A parallel, language agnostic, automatic test case reducer","archived":false,"fork":false,"pushed_at":"2019-09-16T17:22:43.000Z","size":359,"stargazers_count":8,"open_issues_count":5,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-10-05T00:32:11.371Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fitzgen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-01-30T18:32:13.000Z","updated_at":"2022-06-03T12:51:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"b64be8f0-07e8-4576-9475-8b1674f4f01f","html_url":"https://github.com/fitzgen/preduce","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fitzgen/preduce","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fitzgen%2Fpreduce","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fitzgen%2Fpreduce/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fitzgen%2Fpreduce/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fitzgen%2Fpreduce/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fitzgen","download_url":"https://codeload.github.com/fitzgen/preduce/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fitzgen%2Fpreduce/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279001293,"owners_count":26083058,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-09T02:00:07.460Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-30T04:19:52.443Z","updated_at":"2025-10-09T11:15:26.857Z","avatar_url":"https://github.com/fitzgen.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"**⚠ ⚠ ⚠**\n\n**This is very much an incomplete and work-in-progress prototype!! Any claims\nmade below are likely false.**\n\n**⚠ ⚠ ⚠**\n\n# `preduce`\n\n[![Build Status](https://travis-ci.org/fitzgen/preduce.png?branch=master)](https://travis-ci.org/fitzgen/preduce)\n\nA parallel, language-agnostic, automatic test case reducer.\n\n`preduce` takes a large test case that exhibits some interesting behavior (such\nas crashing your program) and finds a smaller test case that exhibits the same\ninteresting behavior.\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n\n\n- [What? Why?](#what-why)\n- [Installing `preduce`](#installing-preduce)\n- [Using `preduce`](#using-preduce)\n  - [Writing an Is-Interesting? Predicate Script](#writing-an-is-interesting-predicate-script)\n  - [Using `preduce` as a Libary](#using-preduce-as-a-libary)\n- [How `preduce` Works](#how-preduce-works)\n  - [Reducers](#reducers)\n  - [Is-Interesting? Predicates](#is-interesting-predicates)\n  - [Traversal of the Search Space](#traversal-of-the-search-space)\n    - [Reframing the Problem](#reframing-the-problem)\n- [Why Not Merge `preduce` with C-Reduce?](#why-not-merge-preduce-with-c-reduce)\n- [Areas of Future Exploration](#areas-of-future-exploration)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n## What? Why?\n\nMinimal test cases are the most useful: they contain only the bits necessary to\ntrigger the bug. Everything else is actively detrimental to understanding and\nfixing the bug because it must be tediously considered and disregarded by the\nprogrammer, wasting her valuable time.\n\nAt the same time, creating minimal test cases is often difficult. Requiring bug\nreporters to provide a minimal test case results in valuable bugs not getting\nreported. Even for a program's developers, who are intimately familiar with its\ninner workings, isolating the relevant bits of a huge test case can be like\nfinding a needle in a haystack.\n\nSo why not automate the test case candidate process? This is exactly what\n`preduce`, and programs that inspired it, provide.\n\nFor more background:\n\n* [Delta Debugging on Wikipedia](https://en.wikipedia.org/wiki/Delta_Debugging)\n\n* [C-Reduce](http://embed.cs.utah.edu/creduce/), an *excellent* automatic test\n  case reducer geared mostly towards C and C++ test cases, but also usable with\n  other curly brace languages. `preduce` is directly inspired by C-Reduce, but\n  has the goal of better utilizing many-core machines to reduce test cases\n  faster. For more details, see the implementation notes further below.\n\n## Installing `preduce`\n\nFirst, [install Rust and its package manager `cargo`](https://rustup.rs/) if you\ndon't already have them installed.\n\nThen, use `cargo` to install `preduce`:\n\n```\n$ cargo install preduce\n```\n\n`preduce` should now be installed at `$HOME/.cargo/bin/preduce`, so make sure\nthat `$HOME/.cargo/bin` is on your `$PATH`.\n\n## Using `preduce`\n\nTo use `preduce`, invoke it with the script that implements the \"is this test\ncase interesting?\" predicate and the test case.\n\n```\n$ preduce [options] ./path/to/test/case ./path/to/predicate ./path/to/reducer...\n```\n\nFor information about the various options available, run\n\n```\n$ preduce --help\n```\n\n### Writing an Is-Interesting? Predicate Script\n\nPredicate scripts are invoked with a single argument: a relative path the\npotentially-interesting test case file it should judge. The predicate script\nmust communicate whether the test case is interesting or not by exiting `0` for\ninteresting, or non-zero for uninteresting.\n\nThe predicate script **must** be deterministic. If it isn't, `preduce`'s results\nwon't be useful, and they won't likely be very reduced either. Garbage in,\ngarbage out.\n\nHere are some tips for shell scripts:\n\n* Start with `set -eu` to exit non-zero if any subcommand fails, or an undefined\n  variable is used.\n* Leverage `grep`'s non-zero exit when it doesn't find any matches. This can be\n  used to hunt for a particular error message from your program, or early exit\n  when you know the grepped-for pattern must appear in the test case input file\n  to trigger your bug.\n\n### Using `preduce` as a Libary\n\nFor programmatic control over candidate strategies and is-interesting\npredicates, you can use the `preduce` crate as a library.\n\nCreate a new executable crate with `cargo`:\n\n```\n$ cargo new --bin my-custom-test-case-reducer\n$ cd my-custom-test-case-reducer/\n```\n\nThen, add `preduce` as a dependency in your `Cargo.toml`:\n\n```toml\n[dependencies]\npreduce = \"\u003cinsert latest preduce version here\u003e\"\n```\n\nFor library usage documentation, see\nthe [`preduce` library documentation on docs.rs](https://docs.rs/preduce).\n\n## How `preduce` Works\n\n`preduce` is comprised of three parts:\n\n1. A test case reducer, which generates smaller test cases of unknown\n   interesting-ness from an initial, larger, known-interesting test case.\n\n2. A predicate to test whether a given test case is interesting or not.\n\n3. Traversal of the reduced test case search space, which leverages (1) and (2).\n\n### Reducers\n\nA test case reducer is given a known-interesting test case, and yields a series\nof potentially interesting candidates of the known-interesting test case. Test\ncase reducers are typically implemented as scripts invoked as subprocesses of\n`preduce`. This design is mimicked from C-Reduce, and allows `preduce` to reuse\nall of C-Reduce's test case reducers.\n\n`preduce`'s builtin test case candidate strategies include:\n\n* Removing a line of text\n* Removing a comment\n* Removing a contiguous, indented chunk of text\n* And all of C-Reduce's other candidate strategies\n\nUsers may also provide their own reducers to be used alongside or instead of the\nbuiltin set of reducers.\n\nCustom reducers **must** produce *candidates* of their seed test case: each new\ntest case must be smaller in size than the seed. If this property is not held,\n`preduce` is not guaranteed to terminate.\n\n### Is-Interesting? Predicates\n\nThe \"is this test case interesting?\" predicates are wholly supplied by the user,\nand are opaque to `preduce`. `preduce` simply invokes them. Additionally, it is\n`preduce` that manages all concurrency and parallelism, and both the predicate\nscripts and the reducers need not be privy.\n\nOnce again, this mimics C-Reduce's design.\n\n### Traversal of the Search Space\n\nTo best describe `preduce`'s search strategy is to contrast it with C-Reduce's\nsearch strategy, and describe the motivations that led me to implementing an\nalternative. What follows is largely an adaptation and summarization of John\nRegehr's blog post about making C-Reduce parallel in the first\nplace: [Parallelizing Delta Debugging](http://blog.regehr.org/archives/749).\n\nEmpirically, most candidates are not interesting. C-Reduce is most often used\nwith C/C++ compilers: whether they crash when compiling the test case, whether\nthey miscompile it, etc. Indeed, I was using C-Reduce for a compiler as well,\njust not a typical one: `rust-bindgen`, which takes C/C++ headers and emits\nextern FFI declaration boiler plate for Rust programs that want to use the C/C++\nheader's library. In such a scenario, it is easy to see that most candidates\nwill change the semantics of the C/C++ program in ways such that the buggy code\npath in the compiler is not triggered, or, even more likely, produce an invalid\nC/C++ program.\n\nWith that in mind, let's discuss C-Reduce's approach to parallelizing delta\ndebugging.\n\nC-Reduce has an initial, known-interesting test case, and spawns off\nN=number-of-cores workers to test each one's interesting-ness.\n\n          version O\n             /|\\\n            / | \\\n           /  |  \\\n          /   |   \\\n         ?    ?    ?\n\nTwo of the workers complete, and find their test cases uninteresting, so\nC-Reduce spawns two new workers with the next candidates to be judged for\ninteresting-ness.\n\n        version O ----.\n           /|\\      \\  \\\n          / | \\      \\  \\\n         /  |  \\      \\  \\\n        /   |   \\      \\  \\\n       X    X    ?      ?  ?\n\nThen a worker finds its test case interesting. All active workers' test cases\nare abandoned, and C-Reduce begins generating and testing candidates of the new,\nsmaller test case.\n\n         version O -------.\n           /|  \\        \\  \\\n          / |   \\        \\  \\\n         /  |    \\        \\  \\\n        /   |     \\        \\  \\\n       X    X   version 1   X  X\n                   /|\\\n                  / | \\\n                 /  |  \\\n                /   |   \\\n               ?    ?    ?\n\nThis greedy search is repeated to a fixpoint where no further candidates are\ninteresting.\n\nC-Reduce doesn't worry about the work that is abandoned whenever a new candidate\nis judged interesting because most candidates aren't interesting.\n\nBut many candidates are independent of each other, for example removing one\nfunction is often independent from removing another function. We can generate\nthe \"same\" candidate once again for the new, reduced test case: same\nfunction-removal diff, different test case to apply the diff to. And then we\nbegin testing that \"same\" candidate again, only to lose the race to be the first\nworker to confirm interesting-ness once again, and we repeat this race-losing\nprocess potentially many times. The more workers we have, and the more\nparallelization we introduce, the more likely each worker is to lose the race,\nand the more likely we are to repeat the same work over and over.\n\nJohn Regehr found that performance with four workers was an improvement over one\nworker, but that five workers was worse than one, and as even more are added\nperformance falls off a cliff. On the machine I'm working with, I have 48\nlogical cores, and I **really** want to utilize them, because C-Reduce runs\noften take a day and half when minimizing test cases for `rust-bindgen`.\n\nJohn Regehr describes in his blog post how he initially investigated each core\nhaving its own copy of the smallest interesting test case found so far, and\nperiodically merging them. He ran into these problems:\n\n* The `merge` tool is very conservative, and will often fail with conflicts.\n* He was unsure how often to merge.\n\nLike C-Reduce, `preduce` maintains a global most-reduced known-interesting test\ncase. Unlike C-Reduce, `preduce` does not abandon the work that workers are\nperforming when a new most-reduced interesting test case is discovered, and\ninstead let's them finish. If their test case is smaller than the current\nmost-reduced interesting test case, then it becomes the new most-reduced\ninteresting test case, and we start searching candidates generated from this new\ntest case. `preduce` also generates a merge between the new test case and what\nwas (or still is, if this test case is smaller) the most-reduced interesting\ntest case. This is inserted into the queue of candidates to test for\ninteresting-ness the same as any other generated candidate. If the merge fails,\nthen it is discarded.\n\nThis approach provides a definite answer to how often to merge: whenever an\ninteresting test case is found. As for conflicts and merge failures, `preduce`\nbuilds a `git` history of the test case's candidates and uses `git`'s awareness\nof history to make more merges succeed. It also does dumb little tricks like\nshuffling the order of generated candidates so that a reducer which generates\ncandidates in order (eg remove the first line, remove the second line, ...)\ndoesn't accidentally make every candidate conflict with each other.\n\nHere is an illustration of `preduce`'s merging approach. A candidate is found\ninteresting while two more are still being judged. This spawns off new workers\nfor new candidate of the new test case. Some of these are judged uninteresting\nquickly:\n\n         version O ------------.\n           /|  \\          \\     \\\n          / |   \\          \\     \\\n         /  |    \\          \\     \\\n        /   |     \\          \\     \\\n       X    X   version 1.0   ?     ?\n                   /|\\\n                  / | \\\n                 /  |  \\\n                /   |   \\\n               ?    X    X\n\nMeanwhile, one of the candidates of the initial test case is also judged\ninteresting. This kicks off a new worker, attempting to do a merge between the\ninteresting candidates:\n\n         version O ------------.\n           /|  \\          \\     \\\n          / |   \\          \\     \\\n         /  |    \\          \\     \\\n        /   |     \\          \\     \\\n       X    X   version 1.0   ?  version 1.1\n                   /|\\     \\       /\n                  / | \\     \\     /\n                 /  |  \\     \\   /\n                /   |   \\     \\ /\n               ?    X    X     ?\n\nNext, the last direct candidate of version 0 is judged uninteresting. We start\ntesting the next candidate of the current most-reduced interesting test\ncase. For illustrative purposes, let's say that is version 1.0 (it could be\neither 1.0 or 1.1, it makes no difference).\n\n         version O ------------.\n           /|  \\          \\     \\\n          / |   \\          \\     \\\n         /  |    \\          \\     \\\n        /   |     \\          \\     \\\n       X    X   version 1.0   X  version 1.1\n                /  /|\\     \\       /\n               /  / | \\     \\     /\n              /  /  |  \\     \\   /\n             /  /   |   \\     \\ /\n            ?  ?    X    X     ?\n\nThe merge between 1.0 and 1.1 is found interesting, and becomes the smallest .\n\n         version O ------------.\n           /|  \\          \\     \\\n          / |   \\          \\     \\\n         /  |    \\          \\     \\\n        /   |     \\          \\     \\\n       X    X   version 1.0   X  version 1.1\n                /  /|\\     \\       /\n               /  / | \\     \\     /\n              /  /  |  \\     \\   /\n             /  /   |   \\     \\ /\n            ?  ?    X    X  version 2.0\n                                \\\n                                 \\\n                                  \\\n                                   \\\n                                    ?\n\nThis process continues until we reach a fixed point where there are no more\ncandidates to make, or all candidates are found to be uninteresting.\n\nLike C-Reduce's approach, this merge approach is also greedy. Just \"slightly\nless\" greedy, in that we try to avoid throwing away work we've aready started to\ndo with the merges. Fundamentally, at each step we still only consider\ncandidates of the current best choice.\n\nIn order for this merging approach to be an improvement over C-Reduce's greedy,\nwork-abandoning approach, the following assumptions must hold:\n\n* Merging two interesting reduced test cases should usually result in another\n  interesting test case.\n\n* The cost of letting all is-interesting tests run to completion does not\n  outweigh the benefit we get from merging. This is usually the case because\n  uninteresting cases are usually judged quicker, so any tests that are still\n  running when the first completes are more likely to be interesting as well.\n\n#### Reframing the Problem\n\n\u003e How do we know `preduce` will reach a fixed point and terminate? This just\n\u003e sounds like a minor heuristics-based tweak to C-Reduce's searching, isn't\n\u003e there something with a big-O improvement? I'm not satisfied!\n\nPatience. I'm not satisfied either! But I don't have anything better yet.\n\nReframing the problem might help get the creative juices going, so here is a\nlittle brain dump. Lots of handwaving, simplifications, and assumptions\nincoming.\n\nWe are given:\n\n* An is-interesting predicate function *f* mapping test cases to boolean.\n\n* An initial test case *T*.\n\n* The finite set of all the candidates we can make, *R = { r\u003csub\u003e0\u003c/sub\u003e,\n  r\u003csub\u003e1\u003c/sub\u003e, ..., r\u003csub\u003en\u003c/sub\u003e }*. Applying a set of candidates *r ⊆ R* to\n  *T* produces the reduced test case *T\u003csub\u003er\u003c/sub\u003e*. That is, *r* are patch\n  operations producing reduced test cases, not reduced test cases themselves.\n  For simplicity, we assume that either candidates are associative and\n  commutative, or can be sorted meaningfully before applying them to *T*.\n\nWith this in hand, we define the search space as the lattice produced by the\npowerset of *R*, with a partial ordering defined by set inclusion. There are\n2\u003csup\u003e*n*\u003c/sup\u003e elements in the search space.\n\nWe begin our search from the bottom element: the empty set of candidates, ie the\ninitial test case. We are ostensibly searching for the maximal candidate set\n*r\u003csub\u003emax\u003c/sub\u003e* for which *f* returns true: *f(r\u003csub\u003emax\u003c/sub\u003e(T)) =\ntrue*. However, searching 2\u003csup\u003e*n*\u003c/sup\u003e elements is much too many to be\npractical, so we are searching for a *maximal enough* candidate set.\n\nC-Reduce makes a greedy depth first search up the lattice. It tests each\ncandidate set *|r| = 1* until *f* returns true, and then tests each candidate\nset *|r| = 2* reachable by adding a single new candidate to the last candidate\nset *|r| = 1* for which *f* returned true, etc... until it reaches a fixed\npoint. It only adds candidates to the candidate set at any given step, and so\nthe process is monotone. The fixed point is reached either because it walks to\nthe top of the lattice (all candidates applied, the empty test case) or *f*\nreturns false for all candidate sets with one more item than the current\ncandidate set. There are *n* candidate sets *|r| = 1*, *n - 1* candidate sets\n*|r| = 2*, ..., 1 candidate set *|r| = n*. Therefore, it has running time of\n*O(n\u003csup\u003e2\u003c/sup\u003e)*.\n\nMerging fits right in with this reframing of the problem: it is the lattice's\njoin operation: union of candidate sets. This preserves the monotone property of\nthe process. It also runs in *O(n\u003csup\u003e2\u003c/sup\u003e)*.\n\nC-Reduce will go up the first path it finds where *f* is returning true, and may\nmiss paths that ultimately go further up the lattice to a more larger candidate\nset, but begin \"after\" the entry point to the current candidate set. The merging\napproach will find these paths *if* it began testing their entries for\ninteresting-ness before the current path's entry was found interesting. The more\nparallel workers testing interesting-ness, the more likely to find such paths'\nentry points. This means that more parallel workers are a boon in a merging\ntraversal, while in C-Reduce's current approach they are more likely to lose the\nrace to first-interesting-candidate and do useless work. This is a nice\nimprovement, if nothing else.\n\nYet another alternative is to perform a depth-first search over the entire\nlattice space, but formalize the *maximal enough* property into a function of\nrunning time or size of the reduced test case. Then, if we ever find a candidate\nset that is both interesting and satisfies the *maximal enough* function, we\nstop.\n\nMaybe there is some more clever property of the lattice we can exploit to\ndiscover a *maximal enough* candidate set in much faster time, or with much\nbetter parallelism. If you have ideas, please let me know!\n\nThe frustration with this lattice perspective of the problem is that our\nis-interesting predicate function *f* itself is not guaranteed to be\nmonotone. That is, it is possible that there is some interesting candidate set\nthat is not reachable starting from the empty candidate set. This means that the\ncombination of the candidates power set lattice with *f* is not itself a\nlattice.\n\n## Why Not Merge `preduce` with C-Reduce?\n\n\u003e Since `preduce` already shares much of its design with C-Reduce, and only\n\u003e improves on the orchestration of the traversal of the search space, why not\n\u003e merge it with C-Reduce?\n\nYes! That would be lovely!\n\n`preduce` exists as a prototype to better parallelize C-Reduce, and (assuming\nthat it meets that goal) it would be great to either replace C-Reduce's search\norchestration with `preduce`'s or port the algorithm to C-Reduce.\n\n## Areas of Future Exploration\n\nAllow `preduce` to distribute work across many different machines, not just\ncores on one machine.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffitzgen%2Fpreduce","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffitzgen%2Fpreduce","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffitzgen%2Fpreduce/lists"}