{"id":17789534,"url":"https://github.com/ehwan/rustylr","last_synced_at":"2025-10-19T18:48:04.285Z","repository":{"id":248332176,"uuid":"828069590","full_name":"ehwan/RustyLR","owner":"ehwan","description":"GLR, LR(1) LALR(1) parser generator for Rust with custom reduce action","archived":false,"fork":false,"pushed_at":"2024-10-26T14:42:38.000Z","size":1214,"stargazers_count":10,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-07T13:54:31.178Z","etag":null,"topics":["automata","automaton","bison","code-generation","compiler","compiler-compiler","deterministic-finite-automata","dfa","glr","glr-parser","lalr-parser","lalr1","lr1","lr1-parser","non-deterministic-finite-automaton","parser","parser-generator","rust","rust-lang","yacc"],"latest_commit_sha":null,"homepage":"https://docs.rs/rusty_lr","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ehwan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"ehwan"}},"created_at":"2024-07-13T03:30:16.000Z","updated_at":"2024-11-20T05:00:50.000Z","dependencies_parsed_at":"2024-07-31T18:19:40.282Z","dependency_job_id":"159174c8-85ad-4061-9855-4b10b18bba5e","html_url":"https://github.com/ehwan/RustyLR","commit_stats":null,"previous_names":["ehwan/rustylr"],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ehwan%2FRustyLR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ehwan%2FRustyLR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ehwan%2FRustyLR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ehwan%2FRustyLR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ehwan","download_url":"https://codeload.github.com/ehwan/RustyLR/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243814880,"owners_count":20352037,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automata","automaton","bison","code-generation","compiler","compiler-compiler","deterministic-finite-automata","dfa","glr","glr-parser","lalr-parser","lalr1","lr1","lr1-parser","non-deterministic-finite-automaton","parser","parser-generator","rust","rust-lang","yacc"],"created_at":"2024-10-27T10:34:14.390Z","updated_at":"2025-10-19T18:48:04.264Z","avatar_url":"https://github.com/ehwan.png","language":"Rust","funding_links":["https://github.com/sponsors/ehwan"],"categories":[],"sub_categories":[],"readme":"# rusty_lr\n[![crates.io](https://img.shields.io/crates/v/rusty_lr.svg)](https://crates.io/crates/rusty_lr)\n[![docs.rs](https://docs.rs/rusty_lr/badge.svg)](https://docs.rs/rusty_lr)\n\n***A Bison-like Parser generator \u0026 Compiler frontend for Rust generating optimised IELR(1), LALR(1) parser tables, with deterministic LR and non-deterministic LR (GLR) parsing.***\n\nRustyLR is a parser generator that converts context-free grammars into IELR(1)/LALR(1) tables and supporting deterministic LR and non-deterministic GLR parsing strategies. It supports custom reduce actions in Rust, with beautiful diagnostics.\nHighly inspired by tools like *bison*, it uses a similar syntax while integrating seamlessly with Rust's ecosystem.\nIt constructs optimized state machines, ensuring efficient and reliable parsing.\n\n![title](images/title.png)\n\n## Features\n - **Custom Reduce Actions:** Define custom actions in Rust, allowing you to build custom data structures easily.\n - **Automatic Optimization:** Reduces parser table size and improves performance by grouping terminals with identical behavior across parser states.\n - **Multiple Parsing Strategies:** Supports minimal-LR(1), LALR(1) parser tables, and GLR parsing strategy.\n - **Detailed Diagnostics:** Detects grammar conflicts, verbose conflict resolution stages, and optimization stages.\n - **Static \u0026 Runtime Conflict Resolution:** Provides mechanisms to resolve conflicts at compile time or runtime.\n - **Location Tracking:** Tracks the location of every token in the parse tree, useful for error reporting and debugging.\n - **State Machine Debugging:** The `rustylr` executable provides a `--state` option that allows you to debug and visualize the generated state machine. This is useful for understanding how the parser will behave and for identifying potential issues in the grammar.\n\n## Quick Start: Using the `rustylr` Executable\n\nThe recommended way to use RustyLR is with the standalone `rustylr` executable. It's faster, provides richer grammar diagnostics, and includes commands for debugging state machines directly.\n\nHere is a step-by-step guide to get you started.\n\n**1. Add `rusty_lr` to your dependencies**\n\nFirst, add the `rusty_lr` runtime library to your project's `Cargo.toml`. The generated parser code will depend on it.\n\n```toml\n[dependencies]\nrusty_lr = \"...\" # Use the same version as the executable\n```\n\n**2. Install the `rustylr` executable**\n\nYou can install the executable from crates.io using `cargo`:\n\n```bash\ncargo install rustylr\n```\n\n**3. Create a grammar file**\n\nCreate a file named `src/grammar.rs`. This file will contain your token definitions and grammar rules. Any Rust code above the `%%` separator will be copied directly to the generated output file.\n\n```rust\n// src/grammar.rs\n// This code is copied to the generated file.\npub enum MyToken {\n    Num(i32),\n    Plus,\n}\n\n%% // Grammar rules start here.\n\n%tokentype MyToken;\n%start E;\n%left plus; // Specify left-associativity for the 'plus' token.\n\n// Define tokens and how they map to MyToken variants.\n%token num MyToken::Num(_);\n%token plus MyToken::Plus;\n\n// Define grammar rules and their return types.\n// E(i32) means the non-terminal E returns an i32.\n// In the action blocks `{ ... }`, you can refer to the values of symbols\n// on the right-hand side by their names (e.g., `e1`, `e2`, `num`).\nE(i32): e1=E plus e2=E { e1 + e2 }\n      | num { let MyToken::Num(num) = num else { unreachable!(); };\n        num\n      }\n      ;\n```\n\n**4. Generate the parser code**\n\nRun the `rustylr` executable to process your grammar file. This command will generate `src/parser.rs` from `src/grammar.rs`.\n\n```bash\nrustylr src/grammar.rs src/parser.rs\n```\n\n**5. Use the generated parser in your code**\n\nFinally, include the newly generated `src/parser.rs` as a module in your `main.rs` or `lib.rs` and use it to parse a token stream.\n\n```rust\n// In src/main.rs\n\n// Include the generated parser module.\nmod parser;\n// Bring the token enum into scope.\nuse parser::MyToken;\n\nfn main() {\n    // Example token stream for \"1 + 2\"\n    let tokens = vec![MyToken::Num(1), MyToken::Plus, MyToken::Num(2)]; \n\n    let parser = parser::EParser::new();        // Assumes 'E' is your start symbol\n    let mut context = parser::EContext::new();\n    let mut userdata = (); // No userdata in this example.\n\n    for token in tokens {\n        match context.feed(\u0026parser, token, \u0026mut userdata) {\n            Ok(_) =\u003e {}\n            Err(e) =\u003e {\n                eprintln!(\"Parse error: {}\", e);\n                return;\n            }\n        }\n    }\n\n    // Get the final parsed result.\n    match context.accept(\u0026parser) {\n        Ok(result) =\u003e {\n            let final_result: i32 = result;\n            println!(\"Parsed result: {}\", final_result); // Should print \"3\"\n        },\n        Err(e) =\u003e {\n            eprintln!(\"Failed to produce a final result: {}\", e);\n        }\n    }\n}\n```\n\n**Important:** Ensure the version of the `rustylr` executable you run matches the version of the `rusty_lr` crate in your `Cargo.toml`. Mismatched versions can lead to build errors.\n\n\n\n## Generated Code Structure\n\nThe generated code will include several structs and enums:\n - `\u003cStart\u003eParser`: A struct that holds the parser table. [(LR docs)](https://docs.rs/rusty_lr/latest/rusty_lr/lr/trait.Parser.html) [(GLR docs)](https://docs.rs/rusty_lr/latest/rusty_lr/glr/trait.Parser.html)\n - `\u003cStart\u003eContext`: A struct that maintains the current parsing state and symbol values. [(LR docs)](https://docs.rs/rusty_lr/latest/rusty_lr/lr/struct.Context.html) [(GLR docs)](https://docs.rs/rusty_lr/latest/rusty_lr/glr/struct.Context.html)\n - `\u003cStart\u003eState`: A type representing a parser state and its associated table. \n - `\u003cStart\u003eRule`: A type representing a production rule. [(docs)](https://docs.rs/rusty_lr/latest/rusty_lr/struct.ProductionRule.html)\n - `\u003cStart\u003eNonTerminals`: An enum representing all non-terminal symbols in the grammar. [(docs)](https://docs.rs/rusty_lr/latest/rusty_lr/trait.NonTerminal.html)\n\n\n### Working with Context\nYou can also get contextual information from the `\u003cStart\u003eContext` struct:\n```rust\nlet mut context = \u003cStart\u003eContext::new();\n\n// ... parsing ...\n\ncontext.expected_token();    // Get expected (terminal, non-terminal) symbols for current state\ncontext.can_feed(\u0026term);     // Check if a terminal symbol can be fed\ncontext.trace();             // Get all `%trace` non-terminals currently being parsed\nprintln!(\"{}\", context.backtrace()); // Print backtrace of the parser state\nprintln!(\"{}\", context);     // Print tree structure of the parser state (`tree` feature)\n```\n\n### The Feed Method\nThe generated code includes a `feed` method that processes tokens:\n\n```rust\ncontext.feed(\u0026parser, term, \u0026mut userdata); // Feed a terminal symbol and update the state machine\ncontext.feed_location(\u0026parser, term, \u0026mut userdata, term_location); // Feed a terminal symbol with location tracking\n```\n\nThis method returns `Ok(())` if the token was successfully parsed, or an `Err` if there was an error.\n\n**Note:** The actual method signatures differ slightly when building a GLR parser.\n\n## GLR Parsing\nRustyLR offers built-in support for Generalized LR (GLR) parsing, enabling it to handle ambiguous or nondeterministic grammars that traditional LR(1) or LALR(1) parsers cannot process.\nSee [GLR.md](GLR.md) for details.\n\n## Error Handling and Conflict Resolution\nRustyLR provides multiple mechanisms for handling semantic errors and resolving conflicts during parsing:\n - **Panic Mode Error Recovery:** Use the `error` token for panic-mode error recovery\n - **Operator Precedence:** Set precedence with `%left`, `%right`, `%precedence` for terminals\n - **Reduce Rule Priority:** Set priority with `%dprec` for production rules\n - **Runtime Errors:** Return `Err` from reduce actions to handle semantic errors\n\nSee [SYNTAX.md - Resolving Conflicts](SYNTAX.md#resolving-conflicts) for detailed information.\n\n## Location Tracking\nTrack the location of tokens and non-terminals for better error reporting and debugging:\n\n```rust\nExpr: exp1=Expr '+' exp2=Expr {\n    println!(\"Location of exp1: {:?}\", @exp1);\n    println!(\"Location of exp2: {:?}\", @exp2);\n    println!(\"Location of this expression: {:?}\", @$); // @$ is the location of the non-terminal itself\n    exp1 + exp2\n}\n| Expr error Expr {\n    println!(\"Error at: {:?}\", @error); // @error is the location of the error token\n    0 // Return a default value\n}\n```\n\nSee [SYNTAX.md - Location Tracking](SYNTAX.md#location-tracking) for detailed information.\n\n## State Machine Debugging\nThe `rustylr` executable includes a powerful `--state` option for debugging the generated parser's state machine. This feature allows you to inspect the details of each state, including its production rules, expected tokens, and transitions to other states. It is an invaluable tool for diagnosing grammar ambiguities, understanding shift/reduce conflicts, and verifying that the parser behaves as expected.\n\nTo use it, run `rustylr` with the `--state` flag, followed by your grammar file:\n\n```bash\nrustylr --state src/grammar.rs\n```\n\nThis will output a detailed, color-coded representation of the state machine directly in your terminal, making it easy to trace the parser's logic.\n\n![State Machine Debug](images/state_option.png)\n\nThis visualization helps you understand the parsing process step-by-step and is particularly useful for debugging complex grammars.\n\n## Examples\n - [Calculator (enum version)](examples/calculator/src/parser.rs): A numeric expression parser using custom token enums\n - [Calculator (u8 version)](examples/calculator_u8/src/parser.rs): A numeric expression parser using byte tokens\n - [JSON Validator](examples/json/src/parser.rs): A JSON syntax validator\n - [Lua 5.4 syntax parser](https://github.com/ehwan/lua_rust/blob/main/parser/src/parser.rs): A complete Lua language parser\n - [C language parser](https://github.com/ehwan/C-language-Parser-In-Rust/blob/main/src/ast/parser_lr.rs): A C language parser\n - [Bootstrap parser](rusty_lr_parser/src/parser/parser.rs): RustyLR's own syntax parser is written in RustyLR itself\n\n## Cargo Features\n - `build`: Enables build script tools for generating parsers at compile time.\n - `tree`: Enables automatic syntax tree construction for debugging purposes. Makes `Context` implement `Display` for pretty-printing.\n\n## Grammar Syntax\nRustyLR's grammar syntax is inspired by traditional Yacc/Bison formats.\nSee [SYNTAX.md](SYNTAX.md) for detailed grammar definition syntax.\n\n## Contributing\nContributions are welcome! Please feel free to open an issue or submit a pull request.\n\n### Project Structure\nThis project is organized as a Cargo workspace with the following crates:\n\n - **`rusty_lr/`**: The main end-user library that provides the public API. This is what users add to their `Cargo.toml`.\n - **`rusty_lr_core/`**: Core parsing engine containing the fundamental data structures, algorithms, and runtime components for both deterministic (`src/parser/deterministic`) and non-deterministic (`src/parser/nondeterministic`) parsing.\n - **`rusty_lr_parser/`**: The main code generation engine that parses RustyLR's grammar syntax, builds parser tables, and generates the actual parser code. This is the core of the parser generation process.\n - **`rusty_lr_derive/`**: Procedural macro interface that wraps `rusty_lr_parser` to provide the `lr1!` macro for inline grammar definitions.\n - **`rusty_lr_buildscript/`**: Build script interface that wraps `rusty_lr_parser` for generating parser code at compile time when using the `build` feature.\n - **`rusty_lr_executable/`**: Standalone `rustylr` executable for command-line parser generation.\n - **`scripts/`**: Development and testing scripts\n\nThe crates have the following dependency relationships:\n- `rusty_lr` depends on `rusty_lr_core`, `rusty_lr_derive`, and `rusty_lr_buildscript` (optional)\n- `rusty_lr_derive` and `rusty_lr_buildscript` depend on `rusty_lr_parser`\n- `rusty_lr_parser` depends on `rusty_lr_core`\n- `rusty_lr_executable` depends on `rusty_lr_buildscript`\n\n```mermaid\ngraph TD;\n    subgraph User Facing\n        rusty_lr;\n        rusty_lr_executable;\n    end\n\n    subgraph Internal\n        rusty_lr_derive;\n        rusty_lr_buildscript;\n        rusty_lr_parser;\n        rusty_lr_core;\n    end\n\n    rusty_lr --\u003e rusty_lr_core;\n    rusty_lr --\u003e rusty_lr_derive;\n    rusty_lr --\u003e rusty_lr_buildscript;\n\n    rusty_lr_derive --\u003e rusty_lr_parser;\n    rusty_lr_buildscript --\u003e rusty_lr_parser;\n    \n    rusty_lr_executable --\u003e rusty_lr_buildscript;\n\n    rusty_lr_parser --\u003e rusty_lr_core;\n```\n\n\n### About the Versioning\nRustyLR consists of two big parts:\n  - executable (`rustylr`), the code generator\n  - runtime (`rusty_lr`), the main library\n\nSince the `cargo` automatically uses the latest patch in `major.minor.patch` version of a crate, we increase the patch number only if the generated code is compatible with the runtime. That is, for any user who is not using buildscript or proc-macro, and using the executable-generated code itself,\nany code change that could make compile errors with the previous generated code will result in a minor version bump.\n\n## License\nThis project is dual-licensed under either of the following licenses, at your option:\n\n - MIT License ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n - Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fehwan%2Frustylr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fehwan%2Frustylr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fehwan%2Frustylr/lists"}