{"id":18497564,"url":"https://github.com/csb6/libearley","last_synced_at":"2025-05-14T05:14:07.341Z","repository":{"id":219491694,"uuid":"749185318","full_name":"csb6/libearley","owner":"csb6","description":"Earley parser library","archived":false,"fork":false,"pushed_at":"2024-05-21T02:23:55.000Z","size":51,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-25T17:42:12.616Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/csb6.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-27T20:21:30.000Z","updated_at":"2024-05-21T02:23:58.000Z","dependencies_parsed_at":null,"dependency_job_id":"4c87c4b8-2edb-4bcd-b949-99f57f5fd76d","html_url":"https://github.com/csb6/libearley","commit_stats":null,"previous_names":["csb6/libearley"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csb6%2Flibearley","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csb6%2Flibearley/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csb6%2Flibearley/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/csb6%2Flibearley/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/csb6","download_url":"https://codeload.github.com/csb6/libearley/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239213595,"owners_count":19601006,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T13:34:56.223Z","updated_at":"2025-02-17T00:19:57.398Z","avatar_url":"https://github.com/csb6.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Libearley\n\nA C++ [Earley parser](https://en.wikipedia.org/wiki/Earley_parser) library.\n\n## Features\n\n- Includes a recognizer for matching grammar rules and functions for traversing the output to build a parse tree\n- Includes functions for pretty-printing rules and Earley items\n- Can be run on partial input\n- Written as a generic library\n    - Can parse any type conforming to the [std::ranges::input_range](https://en.cppreference.com/w/cpp/ranges/input_range) concept.\n        - Works with `std::string`, `std::ifstream`, `std::span\u003cCustomTokenType\u003e`, etc.\n    - Symbols used in the grammar can be any [regular type](https://en.cppreference.com/w/cpp/concepts/regular) that can be converted into an integer.\n        - This includes enums, characters, or even user-created classes.\n\n## Usage\n\n### Dependencies\n\n- A compiler that supports C++20\n- Boost (tested with 1.82, but should work with any version that supports C++20 and includes\n  the [Boost.STLInterfaces](https://www.boost.org/doc/libs/1_84_0/doc/html/stl_interfaces.html) library)\n\n### Building\n\nSince this is a template library, it is header only.\n\nThe `CMakeLists.txt` file in the root of this repository can be added to your project. Alternatively,\nthe files in the `lib` subdirectory can be added directly to your build system.\n\n### Example\n\nThe core functionality of the library is provided in `earley.hpp`. `earley_print.hpp` includes functions for\nprinting some of the library's data structures to iostreams, which might be useful during development.\n\nIn order to build a parser, the library requires the user to provide the following parameters:\n\n- A **token type**: This type represents one element of the input.\n    - This is often a character type (since many parsers parse character strings), but it can be any other\n      type, such as the type of tokens emitted by a lexer.\n- A **symbol type**: This type represents the terminal and non-terminal symbols of the grammar.\n    - Typically, an enum class type should be used.\n        - **Note:** For enum class types, you must provide a special enum member named `Symbol_Count`. This\n          will be used in the parser to determine the number of possible symbols.\n        - For symbol types that are not enum classes, a custom specialization of `earley::symbol_traits` must be\n          provided.\n    - A pair of helper functions with the signatures `bool is_terminal(Symbol)` and\n      `bool matches_terminal(Symbol, Token)` must also be provided. These are used to check if a symbol is a terminal\n      and if a given terminal symbol matches a given token, respectively.\n- A **grammar**: This is a collection of parsing rules.\n    - It must be a contiguous collection of `earley::Rule\u003cSymbol\u003e` objects (e.g. an array of `earley::Rule\u003cSymbol\u003e`)\n    - The grammar rules must get wrapped in an `earley::RuleSet\u003cSymbol\u003e` object. This is what is passed to the parser.\n    - Each rule consists of a \"left-hand side\", which is a non-terminal symbol, and a \"right-hand side\", which\n      is a sequence of zero or more terminal and/or non-terminal symbols.\n    - The parser assumes that all rules with the same left-hand side are located adjacently to each other in the grammar.\n- A **start symbol**: The goal of parsing is to the match the input to a rule that has this symbol as its left-hand\n  side.\n\nTo parse input, call the function `earley::parse` and pass the above information as parameters. This function\nwill return a sequence of *state sets*, each of which holds the Earley items considered against the\ninput at each position. For example, `state_sets[0]` represents the items considered at the\nfirst element of the input, `state_sets[1]` at the second element, and so on.\n\nHere is an example program that parses some input and prints the rule that matched the full input (if any).\nThis code can also be found in `test/example.cpp`.\n\n```cpp\n#include \u003ccstdint\u003e\n#include \u003ciostream\u003e\n#include \u003cspan\u003e\n#include \u003cstring_view\u003e\n#include \u003ccstddef\u003e\n#include \"earley.hpp\"\n#include \"earley_print.hpp\"\n\nenum class Symbol {\n    /* Terminals */\n    Plus, Minus, Mult, Div, LParen, RParen, Digit,\n    /* Nonterminals */\n    Number, Sum, Product, Factor,\n\n    Symbol_Count\n};\n\nstatic constexpr\nbool is_terminal(Symbol s) { return (uint8_t)s \u003c= (uint8_t)Symbol::Digit; }\n\nstatic constexpr\nbool matches_terminal(Symbol terminal, char input)\n{\n    using enum Symbol;\n    switch(terminal) {\n        case Plus:   return input == '+';\n        case Minus:  return input == '-';\n        case Mult:   return input == '*';\n        case Div:    return input == '/';\n        case LParen: return input == '(';\n        case RParen: return input == ')';\n        case Digit:  return std::isdigit(input);\n        default:     return false;\n    }\n}\n\nstatic\nstd::ostream\u0026 operator\u003c\u003c(std::ostream\u0026 out, Symbol s);\n\nint main()\n{\n    /*\n    This grammar represents the following EBNF grammar:\n\n    sum = sum \"+\" product;\n    sum = sum \"-\" product;\n    sum = product;\n    product = product \"/\" factor;\n    product = product \"*\" factor;\n    product = factor;\n    factor = \"(\" sum \")\";\n    factor = number;\n    number = digit;\n    number = digit number;\n    digit = \"0\" | \"1\" | \"2\" | \"3\" | \"4\" | \"5\" | \"6\" | \"7\" | \"8\" | \"9\";\n\n    Note: this grammar is the same as the grammar used in the excellent article series by\n    Loup Vaillant: https://loup-vaillant.fr/tutorials/earley-parsing/\n    */\n    using enum Symbol;\n    static const earley::Rule\u003cSymbol\u003e rules[] = {\n        { Sum,     { Sum, Plus, Product } },\n        { Sum,     { Sum, Minus, Product } },\n        { Sum,     { Product } },\n        { Product, { Product, Mult, Factor } },\n        { Product, { Product, Div, Factor } },\n        { Product, { Factor } },\n        { Factor,  { LParen, Sum, RParen } },\n        { Factor,  { Number } },\n        { Number,  { Digit } },\n        { Number,  { Digit, Number } }\n    };\n    constexpr auto start_symbol = Sum;\n    std::span\u003cconst earley::Rule\u003cSymbol\u003e\u003e rules_view = rules;\n    earley::RuleSet rule_set{rules_view};\n\n    std::string_view input = \"1+(8*9)\";\n    auto state_sets = parse\u003cchar\u003e(rule_set, start_symbol, input);\n\n    auto full_parse = earley::find_full_parse(rules_view, start_symbol, state_sets, input);\n    if(!full_parse) {\n        std::cerr \u003c\u003c \"Error: parse failed\\n\";\n        return 1;\n    }\n    // Use one of the printing functions to print the rule that was used to completely parse the input\n    std::cerr \u003c\u003c \"Full parse: \"; earley::print_item(std::cerr, rules_view, *full_parse.item) \u003c\u003c \"\\n\";\n\n    return 0;\n}\n\nstatic\nstd::ostream\u0026 operator\u003c\u003c(std::ostream\u0026 out, Symbol s)\n{\n    using enum Symbol;\n    switch(s) {\n        case Plus:\n            out \u003c\u003c \"'+'\";\n            break;\n        case Minus:\n            out \u003c\u003c \"'-'\";\n            break;\n        case Mult:\n            out \u003c\u003c \"'*'\";\n            break;\n        case Div:\n            out \u003c\u003c \"'/'\";\n            break;\n        case LParen:\n            out \u003c\u003c \"'('\";\n            break;\n        case RParen:\n            out \u003c\u003c \"')'\";\n            break;\n        case Digit:\n            out \u003c\u003c \"[0-9]\";\n            break;\n        case Number:\n            out \u003c\u003c \"Number\";\n            break;\n        case Sum:\n            out \u003c\u003c \"Sum\";\n            break;\n        case Product:\n            out \u003c\u003c \"Product\";\n            break;\n        case Factor:\n            out \u003c\u003c \"Factor\";\n            break;\n        case Symbol_Count:\n            break;\n    }\n    return out;\n}\n```\n\nThe output should be:\n\n```\nFull parse: Sum -\u003e Sum '+' Product . (0)\n```\n\nThis means that the entire input was matched with the grammar rule `Sum -\u003e Sum '+' Product`. \n\n## License\n\nThis library is licensed under the GPL-3.0 license. For more information,\nsee the LICENSE file in this directory\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsb6%2Flibearley","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcsb6%2Flibearley","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcsb6%2Flibearley/lists"}