{"id":18010185,"url":"https://github.com/p-e-w/tree-saw","last_synced_at":"2025-04-04T12:44:28.246Z","repository":{"id":150094329,"uuid":"121115548","full_name":"p-e-w/tree-saw","owner":"p-e-w","description":"tree-sitter and compilers testing each other","archived":false,"fork":false,"pushed_at":"2018-02-17T07:06:50.000Z","size":7,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-09T22:41:18.176Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/p-e-w.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-11T11:13:50.000Z","updated_at":"2023-09-08T17:36:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"53d29ebd-bfd6-479c-8e27-fee64669f7c0","html_url":"https://github.com/p-e-w/tree-saw","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/p-e-w%2Ftree-saw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/p-e-w%2Ftree-saw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/p-e-w%2Ftree-saw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/p-e-w%2Ftree-saw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/p-e-w","download_url":"https://codeload.github.com/p-e-w/tree-saw/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247182338,"owners_count":20897379,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-30T02:13:17.051Z","updated_at":"2025-04-04T12:44:28.227Z","avatar_url":"https://github.com/p-e-w.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003etree-saw\u003c/h1\u003e\n\u003ch3 align=\"center\"\u003etree-sitter and compilers testing each other\u003c/h3\u003e\n\u003cbr\u003e\n\ntree-saw generates random programs from [tree-sitter](https://github.com/tree-sitter/tree-sitter) grammars and feeds them back to tree-sitter, and/or to a compiler in syntax-checking mode. This allows it to automatically find grammar problems, tree-sitter bugs, and potentially also bugs in the compiler.\n\ntree-saw precomputes the expected recursion depth for each rule in the grammar and uses that information to guide weighted sampling of `CHOICE` rules. `REPEAT` lengths are drawn from a [Poisson distribution](https://en.wikipedia.org/wiki/Poisson_distribution) and regular expression matches are generated using [randexp.js](https://github.com/fent/randexp.js). The resulting probability distribution favors outputs of reasonable size that are well-suited for automated testing.\n\nWhen encountering a program that causes a compiler or grammar error, tree-saw \"prunes\" the syntax tree in accordance with the grammar's rules to find the smallest version of the program that still generates an error. The final output is a compact failure example that lets a human quickly identify (and hopefully fix) the underlying bug.\n\n\n## Bug categories\n\ntree-saw can expose the following types of issues:\n\n* **[\"Type II\"](https://en.wikipedia.org/wiki/Type_I_and_type_II_errors) grammar bugs:** Bugs where a syntactically invalid program is accepted by a grammar. These manifest themselves as errors when compiling the generated output. Running tree-saw on any existing tree-sitter grammar rapidly reveals hundreds of such bugs.\n* **tree-sitter core bugs:** Bugs inside tree-sitter itself. These can show themselves as the grammar being unable to correctly parse the output generated by itself. This occurs with some frequency, and I'm still investigating whether such failures are caused by tree-sitter or by bugs in tree-saw's generator.\n* **Compiler bugs:** Bugs where a syntactically valid generated program results in a compiler syntax error. Since most compilers have parsers that are of much higher quality than their tree-sitter equivalents, such finds are likely to be extremely rare, and in almost all cases indicate a grammar problem instead.\n\ntree-saw can *not* automatically detect **\"type I\" grammar bugs**, where a syntactically valid program is incorrectly rejected by a grammar. However, the generated programs often reveal edge cases that lead to manual identification of such bugs.\n\n\n## Usage\n\n```\n$ tree-saw --grammar tree-sitter-c --compiler 'gcc -fsyntax-only -xc -' c.json\n\u003e\u003e\u003e\u003e\u003eRESULT\n\u003e\u003e\u003e\u003e\u003eSOURCE\nextern \" \" { }\n\u003c\u003c\u003c\u003c\u003cSOURCE\n\u003e\u003e\u003e\u003e\u003eAST\n(translation_unit (linkage_specification (string_literal) (declaration_list)))\n\u003c\u003c\u003c\u003c\u003cAST\n\u003e\u003e\u003e\u003e\u003eERROR\nCompiler error:\n\u003cstdin\u003e:1:8: error: expected identifier or ‘(’ before string constant\n\u003c\u003c\u003c\u003c\u003cERROR\n\u003c\u003c\u003c\u003c\u003cRESULT\n```\n\nFor a full list of options, run `tree-saw` without arguments.\n\n\n## License\n\nCopyright \u0026copy; 2018 Philipp Emanuel Weidmann (\u003cpew@worldwidemann.com\u003e)\n\nReleased under the terms of the [MIT License](https://opensource.org/licenses/MIT)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fp-e-w%2Ftree-saw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fp-e-w%2Ftree-saw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fp-e-w%2Ftree-saw/lists"}