{"id":16664211,"url":"https://github.com/jmitchell/tree-sitter-dhall","last_synced_at":"2025-07-03T13:07:52.047Z","repository":{"id":57379553,"uuid":"172831731","full_name":"jmitchell/tree-sitter-dhall","owner":"jmitchell","description":"(WIP) Dhall grammar for Tree-sitter","archived":false,"fork":false,"pushed_at":"2019-03-02T23:44:14.000Z","size":7745,"stargazers_count":8,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-07-03T13:07:50.973Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jmitchell.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-27T02:57:01.000Z","updated_at":"2024-02-26T21:19:37.000Z","dependencies_parsed_at":"2022-09-02T20:22:15.870Z","dependency_job_id":null,"html_url":"https://github.com/jmitchell/tree-sitter-dhall","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jmitchell/tree-sitter-dhall","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmitchell%2Ftree-sitter-dhall","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmitchell%2Ftree-sitter-dhall/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmitchell%2Ftree-sitter-dhall/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmitchell%2Ftree-sitter-dhall/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jmitchell","download_url":"https://codeload.github.com/jmitchell/tree-sitter-dhall/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmitchell%2Ftree-sitter-dhall/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263331780,"owners_count":23450157,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T10:44:16.814Z","updated_at":"2025-07-03T13:07:52.024Z","avatar_url":"https://github.com/jmitchell.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"tree-sitter-dhall\n=================\n\n[![Build Status](https://travis-ci.org/jmitchell/tree-sitter-dhall.svg?branch=master)](https://travis-ci.org/jmitchell/tree-sitter-dhall)\n\n[Dhall][] grammar for [tree-sitter][].\n\n\nStatus\n------\n\n_Work in progress (unstable)_\n\nThere are some consequential [bugs][]. However, 187/190 Dhall files\n[found](./script/test.js) in [dhall-haskell][] and its git submodules\n(including [dhall-lang][]) parse as expected.\n\nTree-sitter unit tests in [`./corpus`](./corpus) and\n[`./corpus-broken`](./corpus-broken) don't yet cover the full\n[grammar][]. Corpus test coverage is currently tracked in\n[TODO.md][corpus-coverage].\n\n\nUsage\n-----\n\nSee tree-sitter's [using parsers][] documentation.\n\n\nApplications\n------------\n\nExisting:\n\n- [Atom editor support][atom-language-dhall]\n\nPotential:\n\n- Parser for Dhall ports to languages which support C FFI.\n- Portable Dhall syntax libraries and tools.\n  - Validation\n  - Formatting\n  - Linting and refactoring\n  - Querying\n  - Syntactic diffing/patching\n  - Conversion\n    - user-friendly AST\n    - programming languages\n\t- data formats, e.g. [Dhall's binary representation][dhall-binary]\n\t- syntax highlighting for a document format, e.g. HTML+CSS\n  - More editor integrations\n    - simplest in editors which support tree-sitter\n\n\nApproach\n--------\n\nFrom the [project plan][]:\n\n\u003e My hope is as [dhall.abnf][grammar] evolves the process of updating\n\u003e tree-sitter-dhall, and any of its future dependencies, can be mostly\n\u003e automated.\n\n[abnf-to-tree-sitter][] compiled the standard dhall.abnf grammar from\n[dhall-lang][] into [generated_grammar.js](./generated_grammar.js), a\nstarting point for a tree-sitter input grammar. Adjustments were\nneeded, so manual changes were implemented in\n[`grammar.js`](./grammar.js).\n\nRealizing full automation requires:\n\n1. Manually resolving bugs in [`grammar.js`](./grammar.js) through\n   changes that could conceivably be automated, and would ideally make\n   sense for all ABNF grammars.\n2. Improving [abnf-to-tree-sitter][] so it can deterministically\n   generate the working `grammar.js` based on\n   - [dhall.abnf][grammar]\n   - initial production rule (unspecified in [RFC 5234][])\n   - production rules to hide in the parse tree\n   - set of pure heuristics to transform the grammar into an\n     equivalent one that's preferable for some reason\n   - parameters for deterministic search which is informed by\n     - feedback from `tree-sitter generate`\n\t - feedback from `tree-sitter parse` on a set of known valid inputs\n\t - associativity and relative production precedence hints in case\n       it's ambiguous in the ABNF\n     - specification defining semantic reductions in terms of\n       syntactic productions\n   - ... (hopefully not much more)\n\n### Tradeoffs\n\nDepending on your objectives this might not be the right approach.\n\n#### Incremental and ambiguous parsing are overkill\n\nThis _could_ be an issue when optimizing for one-shot parsing\ntime/space.\n\n_Word to the wise: performance optimizations should be informed by\nbenchmarking and profiling realistic input, and also balanced against\npotential performance regressions elsewhere._\n\n[Tree-sitter][tree-sitter] is designed to incrementally parse code as\nit is modified (common in code editors). The data structures and\nalgorithms to support this may incur unnecessary overhead. I have not\nyet had a reason to compare performance of tree-sitter parsers against\noptimized parsers. NB: `tree-sitter parse --time` (used in CI tests)\nmay be convenient for initial benchmarking.\n\nAdditionally, tree-sitter generates [GLR parsers][] which support\nambiguous grammars and therefore may search for many potential parses\nconcurrently. The semantics of ABNF's `/` operator [aren't fully\nspecified](https://stackoverflow.com/a/54951806/204305),\n[dhall.abnf][grammar] relies on backtracking, and it's generally\n[undecidable whether a given context-free grammar is inherently\nambiguous](https://en.wikipedia.org/wiki/Greibach%27s_theorem#Applications)\nso automation efforts must account for ambiguity by default. I also\nhaven't had a reason to analyze which ambiguous parse paths are\nguaranteed to reach a dead-end and could therefore be safely pruned\nfrom the search.\n\n#### Parse trees are verbose and subject to change according to the standard grammar\n\nConcise abstract syntax trees or other high-level interfaces for Dhall\nsyntax may be preferred for their relative ease of use.\n\nHowever, if implementing these interfaces requires any steps which\ncan't be derived automatically, future changes to the standard may\nconflict with those interfaces. As a result conflicts would require\n\n- introducing a breaking change,\n- implementing an unclean workaround, or\n- cutting off support for current and future versions of the standard\n\nNone of the options are ideal. The next part suggests how to manage\nthis.\n\n#### Automating parts that have so far been manual may cause regressions\n\nOver the long-term permitting breaking changes is sometimes for the\nbest. It's the only way to avoid introducing and maintaining hacks for\nthe sake of short-term convenience and long-term backwards\nincompatiblity.\n\nExplicitly dividing projects into two categories should help strike a\nbalance.\n\n1. Expand the set of tooling that can be deterministically generated\n   from declarative specifications. Breaking changes to the\n   specifications automatically translate into corresponding breaking\n   changes in these tools. Open standards development and version\n   control should make it simple to trace these breaking changes to\n   the original source.\n2. Manually build useful tools on top of this frontier of generated\n   tooling. Take a best-effort approach to designing stable,\n   user-friendly interfaces. Offer a vision for how future\n   incompatible specification changes will be handled, for example\n   support the latest N major versions of the specification\n   concurrently.\n\nThose who want to optimize for stability should primarily use the\ndeterministically generated tooling, and those who are willing to\nsacrifice some stability for UX can instead opt for the manually\ncrafted interfaces.\n\nThe goal is for this project and others like [abnf-to-tree-sitter][]\nto abstract any special cases particular to Dhall and automate as much\nas possible. In other words, they are currently in category 2 with\naspirations of being in category 1.\n\n### Contributing\n\nIssues with the [help\nwanted](https://github.com/jmitchell/tree-sitter-dhall/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22)\nlabel are a good place to start.\n\n\nReferences\n----------\n\n* [Dhall's ABNF grammar][grammar]\n* [RFC 5234: Augmented BNF for Syntax Specifications: ABNF][RFC 5234]\n\n\n[Dhall]: https://dhall-lang.org/\n[tree-sitter]: https://github.com/tree-sitter/tree-sitter\n[bugs]: https://github.com/jmitchell/tree-sitter-dhall/issues?q=is%3Aissue+is%3Aopen+label%3Abug\n[dhall-haskell]: https://github.com/dhall-lang/dhall-haskell\n[dhall-lang]: https://github.com/dhall-lang/dhall-lang\n[./corpus]: https://github.com/jmitchell/tree-sitter-dhall/tree/master/corpus\n[./corpus-broken]: https://github.com/jmitchell/tree-sitter-dhall/tree/master/corpus-broken\n[grammar]: https://github.com/dhall-lang/dhall-lang/blob/master/standard/dhall.abnf\n[corpus-coverage]: https://github.com/jmitchell/tree-sitter-dhall/blob/master/TODO.md#corpus-coverage\n[using parsers]: http://tree-sitter.github.io/tree-sitter/using-parsers\n[atom-language-dhall]: https://github.com/jmitchell/atom-language-dhall\n[dhall-binary]: https://github.com/dhall-lang/dhall-lang/blob/master/standard/binary.md\n[project plan]: https://github.com/dhall-lang/dhall-lang/issues/323\n[abnf-to-tree-sitter]: https://github.com/jmitchell/abnf-to-tree-sitter\n[RFC 5234]: https://tools.ietf.org/html/rfc5234\n[GLR parsers]: https://en.wikipedia.org/wiki/GLR_parser\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmitchell%2Ftree-sitter-dhall","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjmitchell%2Ftree-sitter-dhall","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmitchell%2Ftree-sitter-dhall/lists"}