{"id":15067625,"url":"https://github.com/nikhilm/ninja-rs","last_synced_at":"2025-04-10T14:23:16.819Z","repository":{"id":57645119,"uuid":"239345829","full_name":"nikhilm/ninja-rs","owner":"nikhilm","description":"An educational implementation of the ninja build system, based on ideas from the Build Systems a la Carte paper.","archived":false,"fork":false,"pushed_at":"2020-10-07T03:17:01.000Z","size":560,"stargazers_count":57,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-10T13:57:59.690Z","etag":null,"topics":["build-system","ninja","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nikhilm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-09T17:38:45.000Z","updated_at":"2025-01-09T17:23:08.000Z","dependencies_parsed_at":"2022-09-26T16:40:36.043Z","dependency_job_id":null,"html_url":"https://github.com/nikhilm/ninja-rs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikhilm%2Fninja-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikhilm%2Fninja-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikhilm%2Fninja-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikhilm%2Fninja-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nikhilm","download_url":"https://codeload.github.com/nikhilm/ninja-rs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248233939,"owners_count":21069493,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["build-system","ninja","rust"],"created_at":"2024-09-25T01:25:17.573Z","updated_at":"2025-04-10T14:23:16.804Z","avatar_url":"https://github.com/nikhilm.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ninja-rs\n\nninja-rs (Like ninjas, but pronounced with an intrusive \"r\" ;-)) is a clone of\nthe [ninja](https://ninja-build.org) build system.\n\n## Motivation\n\nninja-rs was created for several different reasons:\n\n1. Education - [Build Systems a la Carte][bsp] was a hugely influential paper\n   for me as it formalized and distilled build systems. This was an attempt to\n   implement a real build system using the ideas in that paper. Read the notes\n   on design below for more about this. Evan Martin, the original author of\n   ninja, also [wishes][reflection] they had this paper around.\n\n2. An exercise in using Rust outside of my day job for a real project.\n\n3. I had never written a proper parser before.\n\nninja is a fairly simple build system (it was explicitly designed to be the\n\"assembly language\" of build systems), so I figured it would be a\ngood place to start. It is relatively small, and the original code is quite\nreadable. It doesn't concern itself with any networking or packaging. As Evan\nMartin [says][reflection]:\n\n\u003e Ninja is pretty easy to implement for the fun 20% of it and the remaining 80%\n\u003e is \"just\" some fiddly details.\n\nIt is a work in progress. As my understanding of Ninja and Rust evolved,\nvarious things changed and continue to change.\n\n[bsp]: https://www.microsoft.com/en-us/research/publication/build-systems-a-la-carte/\n\n## Feature complete-ness\n\nAt this point the parser is fairly feature complete and ninja-rs is capable of\nbuilding the simple CMake hello-world in this repository. It has a long way to\ngo to fully support everything ninja supports.\n\n- [X] Working parser and topological sort based builder\n- [X] mtime based rebuilding\n- [X] Basic command-line compatibility with Ninja\n- [X] Implicit and ordered dependencies\n- [X] Variables and scoping\n- [X] Handling failed commands correctly\n- [ ] Pools\n- [ ] Path canonicalization\n- [ ] Windows support (Nothing intentionally stopping it, but not tested either.)\n- [ ] Ninja log\n- [ ] build file regeneration\n- [ ] C compiler include parsing (`-M` for GCC/clang, `/showIncludes` for MSVC) and dependency log\n- [ ] Dynamic dependencies\n- [ ] Better pretty-printing\n- [ ] [Extra tools](https://ninja-build.org/manual.html#_extra_tools)\n- [ ] Provide releases\n- [ ] Make `ninjars` available as a `cargo install`-able command.\n\n## Usage\n\nMake sure you have at least Rust 1.46-nightly (that is the only one I've tested with). Clone this repository, then:\n\n```\ncargo build --release -p ninjars --bin ninja\n```\n\nThis will result in a binary `./target/release/ninja` which is statically linked and can be copied wherever you need.\n\n## Contributing\n\nContributions are welcome. Please understand that I have limited time to look\nat pull requests, so I may not reply promptly.  For bug fixes, simply submit a\nPR, ideally with a regression test for that bug.\n\nIf you are considering implementing a major feature, please [file an issue](https://github.com/nikhilm/ninja-rs/issues)\ndiscussing the feature and how you intend to implement it. There may be\nmultiple solutions and trade-offs. A thousand-line PR with a major feature is\nlikely to get rejected if you do not discuss it first.\n\n### Testing\n\nRunning this in the workspace directory will run all tests. Please make sure\nall tests pass when you submit a PR. Please add a test for any new code\nintroduced. PRs with tests are easier to review and accept.\n\n```\nRUST_BACKTRACE=full cargo test\n```\n\n### Formatting\n\nRun `cargo fmt` over the code before submitting.\n\n## Design\n\nThe [DESIGN.md](DESIGN.md) and [parse/DESIGN.md](parse/DESIGN.md) documents\nwere written while I was thinking of various things and are not meant to make\nsense to anyone else. This section summarizes the actual design.\n\nThis design falls out of the following goals:\n\n### Try to stick to the primitives of the Build Systems a la Carte paper\n\nMost production build systems have been around before this paper, and most of\nthem tend not to have strict boundaries between the various stages. I've tried\nto make the primitives shine through when possible. For example, the rebuilder\nand scheduler very explicitly use graphs, talk in terms of Keys and Tasks and\ntry to avoid direct I/O (or thinking in terms of real-world things like unix\ntimes) as much as possible. There are of course limitations on this, since Rust\ndoesn't have the full generic/higher-kinded types support that Haskell does.\nWriting excessively generic code in Rust can get old quickly if one has to\nstart putting trait bounds or generics everywhere. There is a first stage that\nparses ninja files and translates these to a build description. This is then\ntransformed to a set of Keys and Tasks as in the paper. Then, there is an\nimplementation of the Scheduler (_topological_) and Rebuilder (_mtime based_)\nthat implements ninja semantics.\n\nIn some sense, this is almost a compiler pipeline for the ninja language, but\nthe final stage is an interpreter for the action graph instead of a code\ngenerator.\n\n### Try to have usable stand-alone crates\n\nThe implementation is a collection of crates. The parser can be used by itself.\nThe lexer preserves relatively complete information about tokens (unlike\nninja), which could allow things like a `ninja-fmt`. I originally envisioned\nthe parser yielding per-file ASTs, but the way ninja handles scoping across\n`include` and `subninja` rules, makes this intractable. Specifically,\nvariables in the included file are evaluated immediately within the current\nenvironment, so this cannot be parallelized and a per-file AST is meaningless\nas soon as inclusions happen.\n\nThe main `ninja` crate simply assembles all these pieces together.\nTheoretically, one could use the `tasks` and `build` crates to create another\nbuild system. There are a bunch of unclean API boundaries right now, but those\nare amenable to cleaning up.\n\n### Enable optimizing for performance\n\nI started off really focusing on trying to avoid allocations and copies, but\nthis quickly got intractable with Rust's strong ownership requirements. So I've\ncurrently gone to the other extreme, with liberal uses of `clone()`. The lexer\nis still \"zero-copy\", dishing out references to a single `\u0026[u8]`.\nCanonicalization and string-interning can take care of the paths and it should\nbe possible to go from the lexer to the task description without copying, as\nlong as one is willing to propagate more lifetimes around. Of course, variable\nevaluation will always require allocation of new bytes.\n\nIt isn't clear yet whether this can achieve ninja's levels of performance while\nmaintaining readability.\n\n### Be readable/documented\n\nThis generally falls out of sticking to the primitives, but I've also called\nout bits and pieces more obviously, when it isn't clear how to translate\nsomething from \"how does ninja do it\" to \"how do I express this in the\nprimitives from the paper\". A particular instance is the rebuilder's handling\nof phony rules and dirtiness. Similarly, the scheduler does a very intentional\ndepth-first topological sort, unlike ninja, where something similar is done but\nthe graph building is coupled with the parsing and handling dynamic\ndependencies.\n\n### Be testable\n\nNinja doesn't really have a spec. It has a manual that explains behavior\nbroadly, but the original implementation is the only source of truth. This\nmeans a lot of the canonical behavior can only be determined by running\nspecific build files through ninja. I've tried to add tests for as much of the\ncode base as possible. Particularly, the parser has a bunch of acceptance tests\nto ensure compliance with ninja and also to act as regression tests. These\nacceptance tests are just `.ninja` files, which can be quickly run with `ninja`\nto determine if they are following \"the spec\".\n\n[reflection]: http://neugierig.org/software/blog/2020/05/ninja.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikhilm%2Fninja-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnikhilm%2Fninja-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikhilm%2Fninja-rs/lists"}