{"id":13896658,"url":"https://github.com/meric/leftry","last_synced_at":"2025-07-17T13:30:43.274Z","repository":{"id":139107971,"uuid":"60184246","full_name":"meric/leftry","owner":"meric","description":"Leftry - A left-recursion enabled recursive-descent parser combinator library for Lua.","archived":false,"fork":false,"pushed_at":"2017-10-09T09:44:00.000Z","size":90,"stargazers_count":36,"open_issues_count":0,"forks_count":4,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-11-25T02:33:15.166Z","etag":null,"topics":["lua","parser-combinators","parser-generator","parser-library","parsers"],"latest_commit_sha":null,"homepage":"","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/meric.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2016-06-01T14:30:37.000Z","updated_at":"2024-09-21T11:36:14.000Z","dependencies_parsed_at":"2024-02-23T11:14:49.700Z","dependency_job_id":"55694dcd-83fb-434b-a219-8fd45b2dd73c","html_url":"https://github.com/meric/leftry","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/meric/leftry","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/meric%2Fleftry","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/meric%2Fleftry/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/meric%2Fleftry/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/meric%2Fleftry/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/meric","download_url":"https://codeload.github.com/meric/leftry/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/meric%2Fleftry/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265611071,"owners_count":23797807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lua","parser-combinators","parser-generator","parser-library","parsers"],"created_at":"2024-08-06T18:03:04.008Z","updated_at":"2025-07-17T13:30:42.932Z","avatar_url":"https://github.com/meric.png","language":"Lua","readme":"# Leftry - A left recursion enabled recursive-descent parser combinator library. #\n\nThis library is for creating and composing parsers.\n\n[Example Lua Parser](http://github.com/meric/l2l/blob/master/l2l/lua.lua#L410)\n\nFor example:\n\n```\nlocal grammar = require(\"leftry\")\nlocal factor = grammar.factor\nlocal span = grammar.span\n\n# Declaring a Non-Terminal, \"A\"\nlocal A = factor(\"A\", function(A) return\n  span(A, \"1\"), # 1st alternative, A := A \"1\"\n  \"1\"           # 2nd alternative, A := \"1\"\nend)\n\n# Declaring a Non-Terminal, \"B\"\nlocal B = factor(\"B\", function(B) return\n  span(B, \"2\"), # 1st alternative, B := B \"2\"\n  A             # 2nd alternative, B := A\nend)\n\n# Using the composed parser.\n# The first argument is the input string.\n# The second argument is the string index to start from.\nprint(B(\"111122222\", 1))\n```\n\nThis creates a parser `B` that can parse the string \"111122222\".\n\nThe purpose of the anonymous function in declaration of the non-terminal\nenables self-reference and referencing other non-terminals that are not fully \ninitialized yet.\n\n\n## Install ##\n\n`luarocks install --server=http://luarocks.org/dev leftry`\n\n## Algorithm ##\n\nFirst trace with a [left-factored](http://www.csd.uwo.ca/~moreno//CS447/Lectures/Syntax.html/node9.html)\nversion of the grammar, then apply the left-recursive grammar. Like how a human would intuitively do it.\n\n## Other Top-down left recursion enabled parser combinator implementations ##\n\n* http://hafiz.myweb.cs.uwindsor.ca/xsaiga/fullAg.html\n* https://github.com/djspiewak/gll-combinators\n\n## Running unit tests ##\n\n`lua test.lua` \n\n## Usage ##\n\n* A parser has the following function signature.\n\n ```\n rest, values = parser(invariant, position, [peek])\n ```\n\n 1. `rest` is the next index to parse. `rest-1` is the last index of the parsed\n    value. If `rest` is `nil`, it means the parse is invalid.\n 2. `values` is the data created as a result of a successful parse.\n 3. `invariant` is the Lua string that is being parsed.\n 4. `position` is the integer index to start the parse from.\n    It must be 1 or greater.\n 5. `peek` (optional). If true, validates only, and `values` will be `nil`.\n    Validating without creating the data is several times faster.\n\n* As iterator:\n\n  The function signature of a parser allows it to be used as a Lua iterator\n  in a for-loop.\n\n  ```\n  local actual = {}\n  for rest, values in span(\"1\", \"2\", \"3\"), \"123123123\", 1 do\n      table.insert(actual, rest)\n  end\n  -- actual == {4, 7, 10}\n  ```\n\n  This can be useful, for example in a programming language parser, to iterate\n  through each parsed statement.\n\n* Composition:\n\n  Parsers can be nested. Left recursion is allowed.\n\n  ```\n  local A = factor(\"A\", function(A) return\n    span(A, \"1\"), \"1\"\n  end)\n  local B = factor(\"B\", function(B) return\n    span(B, \"2\"), A\n  end)\n  local src = \"11112222\"\n  assert(B(src, 1, true) == #src + 1)\n  ```\n\n* Data initialization:\n\n  You can customise how the data is generated from a parse.\n\n  ```\n  local A = factor(\"A\", function(A) return\n    span(A, \"1\") % function(initial, value)\n      return (initial or \"\") .. value\n    end, \"1\"\n  end)\n  local src = \"111\"\n  A(src, 1) -- returns #src + 1, \"111\"\n  ```\n\n## Left recursion ##\n\nLeftry can handle some examples of left recursion.\n\n* Simple left recursion:\n\n  ```\n  local A = factor(\"A\", function(A) return\n    span(A, \"1\"), \"1\"\n  end)\n  ```\n\n* Nested left recursion:\n\n  ```\n  local A = factor(\"A\", function(A) return\n    span(A, \"1\"), \"1\"\n  end)\n  local B = factor(\"B\", function(B) return\n    span(B, \"2\"), A\n  end)\n  ```\n\n## Performance ##\n\nPerformance of the built-in Lua parser.\n\n* Macbook Pro 2.6 Ghz i7 with 16 GB RAM:\n  * Lua:\n    1. Validate the grammar of around 0.4 megabytes of Lua per second.\n    2. Parse 0.1 megabytes of Lua into abstract syntax tree representation per second.\n  * LuaJIT:\n    1. Validate the grammar of around 4 megabytes of Lua per second.\n    2. Parse 0.68 megabytes of Lua into abstract syntax tree representation per second.\n  * For comparison:\n    1. Lua interpreter can load a 15 megabyte Lua function in one second.\n    2. LuaJIT can load a 25 megabyte Lua function in one second.\n\n\n## Elements ##\n\n* `factor(name, generate, [initializer])`\n\n  Create a non-terminal element.\n\n  * `name` is the tostring value of the element.\n  * `generate` is the function that, when called with this element, returns the\n    definition of this non-terminal. The values returned with this function\n    will be wrapped in an `any`. You may optionally, explicitly return a single\n    `any` that contains all the alternatives. Strings literals are\n    automatically converted into `term` elements.\n  * `initializer` is the function that will be called with values parsed\n    from this element to let the user convert the parsed value into something\n    useful. See \"Data Constructors\" section.\n\n  Usage:\n\n  ```\n  local A = factor(\"A\", function(A) return\n    span(A, \"1\"), \"1\"\n  end)\n  ```\n\n* `rep(element, [reducer])`\n\n  Create a repeating element. It can be used only in an `any` or a `span`.\n\n  * `element` is the element that can appear 0 or more times (if this element\n    is in a `span`), or 1 or more times (if this element is in an `any`).\n    Strings literals are automatically converted into `term` elements.\n  * `reducer` is the function that will be called with values parsed\n    from this element to let the user convert the parsed values into something\n    useful. See \"Data Constructors\" section.\n\n  Usage:\n\n  ```\n  span(\n    \"1\",\n    rep(\"2\", function(a, b) return (a or \"\")..b end),\n    \"3\")\n  ```\n* `opt(element)`\n\n  Create an optional element. It can be used only in a `span`.\n\n  * `element` is the element that can appear 0 or 1 times.\n    Strings literals are automatically converted into `term` elements.\n\n  Usage:\n\n  ```\n  span(\"1\", opt(\"2\"), \"3\")\n  ```\n\n* `any(...)`\n\n  Create an any element. The any element contains a set of alternatives, that \n  will be attempted from left to right.\n\n  The `any` element is used internally to wrap the alternatives returned\n  by the `factor` \"generate\" function.\n\n  You do not need to worry about using `any`.\n\n* `span(...)`\n\n  Create an span element. The span element can be assigned a reducer with the\n  `%` operator. See the \"Data Constructors\" section.\n\n  * `...` the children of the span element. Each child must be encountered in\n    order to provide a valid parse, unless it is an `opt` or `rep` element.\n\n  Usage:\n\n  ```\n  local A = factor(\"A\", function(A) return\n    span(A, \"1\") % function(initial, value)\n      return (initial or \"\") .. value\n    end, \"1\"\n  end)\n  ```\n\n  The span element can also be assigned a spacing rule using the `^` operator:\n\n  ```\n  local function span(...)\n    -- Apply spacing rule to all spans we use in the Lua grammar.\n    return grammar.span(...) ^ {spacing=spacing, spaces=\" \\t\\r\\n\"}\n  end\n  ```\n\n  See built-in Lua parser for an example on what spacing function looks like.\n\n* `term(literal, [initializer])`\n\n  Create a literal element.\n\n  * `literal` the string literal that must be encountered to provide a valid\n    parse.\n  * `initializer` is the function that will be called with the literal whenever\n    there is a valid parse.\n\n  Usage:\n\n  ```\n  term(\"hello\")\n  ```\n\n## Data Constructors ##\n\n### Initializer ###\n\n* `factor`\n\n  ```\n  function(\n      value,    -- The parsed value to transform.\n      self,     -- The element responsible for parsing being transformed.\n      position, -- The index where `value` was found.\n      rest,     -- The next index after `value` ends.\n      choice)   -- The index of the alternative `value` was parsed from.\n\n    -- Default implementation\n    return value\n  end\n  ```\n\n* `term`\n\n  ```\n  function(\n      value,    -- The parsed value to transform. (The literal.)\n      self,     -- The element responsible for parsing being transformed.\n      position, -- The index where `value` was found.\n      rest)     -- The next index after `value` ends.\n\n    -- Default implementation\n    return value\n  end\n  ```\n\n### Reducer ###\n\nReducers are functions that will be folded over each value that will be parsed.\n\n* `span`\n\n  ```\n  function(\n    accumulated, -- The accumulated value. In the first iteration it is `nil`.\n    value,       -- The current value that is parsed.\n    self,        -- The element parsing the current value.\n    position,    -- The index where the current value begins.\n    rest,        -- The next index after `value` ends.\n    i)           -- `value` belongs to the `i`th element of this `span` element.\n\n    -- Default implementation\n    return rawset(initial or {}, i, value)\n  end\n  ```\n\n* `rep`\n\n  ```\n  function(\n    accumulated, -- The accumulated value. In the first iteration it is `nil`.\n    value,       -- The current value that is parsed.\n    self,        -- The element parsing the current value.\n    position,    -- The index where the current value begins.\n    rest,        -- The next index after `value` ends.\n    i)           -- The `i`th time the child element has been encountered.\n\n    -- Default implementation\n    return rawset(initial or {}, i, value)\n  end\n  ```\n\n## Caveats ##\n\nThe current implementation does not enforce the following rules properly.\n\n1. A `span` must have more than one child.\n2. In a left recursion alternative, only the first element may be the\n   left-recurring non-terminal. More than one consecutive left-recurring\n   non-terminal is not supported, even if it currently works.\n\n   ```\n   -- This is OK\n   local A = factor(\"A\", function(A) return\n                span(A, \"1\", A, \"2\"), \"1\"\n             end)\n   -- This is not OK\n   local A = factor(\"A\", function(A) return\n                span(A,  A, \"1\", \"2\"), \"1\"\n             end)\n   ```\n3. An `any` element must not have any `opt` children.\n4. A `rep` element that is a child of an `any` element requires 1 or more\n   elements to match.\n5. A `rep` element that is a child of an `span` element requires 0 or more\n   elements to match.\n6. The first nonterminal element of a span that is part of a left recursion\n   path, cannot be wrapped in `opt` or `rep`.\n\n## TODO ##\n\n* ~~Implement Lua grammar in Leftry to prove it can handle the grammar of a\nprogramming language.~~\n* ~~Implement whitespacing support.~~\n* ~~Add appropriate data initializers for the builtin Lua parser, to override the\n  default ones.~~\n\n* Test the following grammar:\n\n    ```\n    \"b\"\n    | s ~ s       ^^ { _ + _ }\n    | s ~ s ~ s   ^^ { _ + _ + _ }\n    https://github.com/djspiewak/gll-combinators\n    ```\n","funding_links":[],"categories":["Lua"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmeric%2Fleftry","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmeric%2Fleftry","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmeric%2Fleftry/lists"}