{"id":15692637,"url":"https://github.com/discretetom/retsac","last_synced_at":"2025-03-31T23:31:13.047Z","repository":{"id":44029393,"uuid":"502550723","full_name":"DiscreteTom/retsac","owner":"DiscreteTom","description":"Text lexer and parser. Compiler frontend framework.","archived":true,"fork":false,"pushed_at":"2025-01-31T03:06:23.000Z","size":3816,"stargazers_count":7,"open_issues_count":14,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-03-03T12:47:54.257Z","etag":null,"topics":["bison","compiler","flex","lexer","llvm","lr","parser"],"latest_commit_sha":null,"homepage":"https://discretetom.github.io/retsac/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DiscreteTom.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-06-12T07:42:26.000Z","updated_at":"2025-01-31T03:07:05.000Z","dependencies_parsed_at":"2023-12-15T02:00:32.030Z","dependency_job_id":"ca555029-04de-44fb-9202-d8c774444be0","html_url":"https://github.com/DiscreteTom/retsac","commit_stats":{"total_commits":785,"total_committers":2,"mean_commits":392.5,"dds":0.001273885350318471,"last_synced_commit":"7ea06e7449e3dbd19085f8036edaf398f941a659"},"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiscreteTom%2Fretsac","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiscreteTom%2Fretsac/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiscreteTom%2Fretsac/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiscreteTom%2Fretsac/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DiscreteTom","download_url":"https://codeload.github.com/DiscreteTom/retsac/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246557700,"owners_count":20796695,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bison","compiler","flex","lexer","llvm","lr","parser"],"created_at":"2024-10-03T18:36:53.553Z","updated_at":"2025-03-31T23:31:13.041Z","avatar_url":"https://github.com/DiscreteTom.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deprecated\n\nSee [whitehole](https://github.com/DiscreteTom/whitehole).\n\n# Retsac\n\n[![npm](https://img.shields.io/npm/v/retsac?style=flat-square)](https://www.npmjs.com/package/retsac)\n![coverage](https://img.shields.io/codecov/c/github/DiscreteTom/retsac?style=flat-square)\n![build](https://img.shields.io/github/actions/workflow/status/DiscreteTom/retsac/publish.yml?style=flat-square)\n![license](https://img.shields.io/github/license/DiscreteTom/retsac?style=flat-square)\n\n\n\n\u003e [!WARNING]\n\u003e This project is still in early development stage, the API may change frequently.\n\nText lexer and parser. Compiler frontend framework.\n\nThis can be used to **_fast prototype_** your own programming language compiler/translator frontend, or parse your domain specific language.\n\nTry it online in the [playground](https://dttk.discretetom.com/js-playground?crushed=%28%27dependencieU%27https%253A%252F%252Fcdn.jsdelivr.net%252Fnpm%252FM%25400.17.0%252Fdist%252FM.min.js%27%255D%7EcellUJPYpaY9ZLO%252C%2520ELRG6MNtrue%7Eid%210%29%252CJWrite%2520the%2520PKr9lO_LO.BQer%257BXaH%252F123%252FG*bQq%253B--IZpKrG_ELR.AdvancedBQerDlOGXVH%255C%27a%255C%27G*bQDVH%2522V%2522%252C%2520checkAllHtrueG%257D%253BC4418%29%252CJPK9Ys6pKr.pKAll%257B%2522123%2522%257D-Iroot6Ys.buffer%255B0%255D--console.log%257Broot.toTYeStringq%257DC5544%29%255D%7EpanelUF5544%252CF4418%255D%29*%257D-zz.-%255Cr%255Cn6%2520%253D%25209%27%7Ecode%21%27ICNfalse%7Eid%21FD%257BZF170372543G%2520%29H%253A%2520Iconst%2520J%28%27name%21%27KarseMYtsacN%27%7EYadonly%21OexerQuildUs%21%255BVentryX*defineDYreZ%28%2520_6new%2520q%257B%257Dz%2520%2520%2501zq_ZYXVUQONMKJIHGFDC96-*_).\n\n## Installation\n\n```bash\nyarn add retsac\n```\n\n## Features\n\n- The Lexer, yield [token](https://github.com/DiscreteTom/retsac/blob/main/src/lexer/model/token.ts) from the text input string.\n  - Regex support. See [examples](#examples) below.\n  - [Built-in util functions](https://github.com/DiscreteTom/retsac/tree/main/src/lexer/utils).\n    - JavaScript's string literal, numeric literal, integer literal, identifier, etc.\n    - JSON's string literal, numeric literal.\n  - Support custom functions.\n- The Parser, co-work with the lexer and produce an [AST (Abstract Syntax Tree)](https://github.com/DiscreteTom/retsac/blob/main/src/parser/ast).\n  - ELR (Expectational LR) parser.\n    - **_Meta characters_** like `+*?` when defining a grammar rule.\n    - **_Conflict detection_**, try to **_auto resolve conflicts_**.\n    - Query children nodes by using `$('name')` instead of `children[index]`.\n    - Top-down traverse the AST.\n    - Bottom-up reduce data.\n    - Expect lexer to yield specific token type and/or content.\n    - Try to **_re-lex_** the input if parsing failed.\n    - **_DFA serialization \u0026 hydration_** to accelerate future building.\n  - Serializable AST object to co-work with other tools (e.g. compiler backend libs like LLVM).\n- Strict type checking with TypeScript.\n  - _This is amazing, you'd better try this out by yourself._\n\n## Resources\n\n- [~~Documentation \u0026 API reference~~ (deprecated, working on a new one).](https://discretetom.github.io/retsac/)\n- [A demo programming language which compiles to WebAssembly.](https://github.com/DiscreteTom/dt0)\n- [Build tmLanguage.json file in TypeScript with `tmlb`.](https://github.com/DiscreteTom/tmlb)\n- [Compose `RegExp` in JavaScript in a readable and maintainable way with `r-compose`.](https://github.com/DiscreteTom/r-compose)\n\u003c!-- - [VSCode extension.](https://github.com/DiscreteTom/vscode-retsac) --\u003e\n\n## [Examples](https://github.com/DiscreteTom/retsac/tree/main/examples)\n\n### [JSON Parser](https://github.com/DiscreteTom/retsac/blob/main/examples/parser/json/json.ts)\n\nIn this example, we use `AdvancedBuilder` to define grammar rules with `+*?`, define top-down traversers using `traverser`, and query nodes in grammar rules using `$` and `$$`.\n\nAll conflicts are auto resolved.\n\n\u003cdetails open\u003e\u003csummary\u003eClick to Expand\u003c/summary\u003e\n\n```ts\nconst lexer = new Lexer.Builder()\n  .ignore(Lexer.whitespaces()) // ignore blank characters\n  .define({\n    // built-in support for JSON\n    string: Lexer.json.stringLiteral(),\n    number: Lexer.json.numericLiteral(),\n  })\n  .define(Lexer.wordKind(\"true\", \"false\", \"null\")) // token's kind name equals to the literal value\n  .anonymous(Lexer.exact(...\"[]{},:\")) // single char borders without a kind name\n  .build();\n\nexport const builder = new ELR.AdvancedBuilder({ lexer })\n  .data\u003cunknown\u003e()\n  .define(\n    { value: `string | number | true | false | null` },\n    // eval the only child's text to get the value\n    (d) =\u003e d.traverser(({ children }) =\u003e eval(children[0].text!)),\n  )\n  .define(\n    { value: `object | array` },\n    // call the only child's traverse method to get the object/array value\n    (d) =\u003e d.traverser(({ children }) =\u003e children[0].traverse()),\n  )\n  .define(\n    // `?` for zero or one, `*` for zero or more, use `()` to group\n    // quote literal values with `'` or `\"`\n    { array: `'[' (value (',' value)*)? ']'` },\n    // use `$$` to select all children with the given name\n    // traverse all values in the array and return the result as an array\n    (d) =\u003e d.traverser(({ $$ }) =\u003e $$(`value`).map((v) =\u003e v.traverse())),\n  )\n  .define(\n    // use `@` to rename a grammar\n    { object_item: `string@key ':' value` },\n    (d) =\u003e\n      // return an object\n      // use `$` to select the first child with the given name\n      d.traverser(({ $ }) =\u003e ({\n        // eval the key's value, then traverse child to get the entry's value\n        [eval($(`key`)!.text!)]: $(`value`)!.traverse(),\n      })),\n  )\n  .define({ object: `'{' (object_item (',' object_item)*)? '}'` }, (d) =\u003e\n    d.traverser(({ $$ }) =\u003e\n      // every object_item's traverse result is an object, we need to merge them.\n      Object.assign({}, ...$$(`object_item`).map((item) =\u003e item.traverse())),\n    ),\n  );\n```\n\n\u003c/details\u003e\n\n### [Calculator](https://github.com/DiscreteTom/retsac/blob/main/examples/parser/calculator/calculator.ts)\n\nIn this example, we use `reducer` to define bottom-up data reducers, so we can get the result when the AST is built.\n\nThere are conflicts introduced by those grammar rules, we use the high-level resolver API `priority` to resolve them.\n\n\u003cdetails\u003e\u003csummary\u003eClick to Expand\u003c/summary\u003e\n\n```ts\nconst lexer = new Lexer.Builder()\n  .ignore(Lexer.whitespaces()) // ignore blank characters\n  .define({ number: /[0-9]+(?:\\.[0-9]+)?/ })\n  .anonymous(Lexer.exact(...\"+-*/()\")) // operators\n  .build();\n\nexport const builder = new ELR.ParserBuilder({ lexer })\n  .data\u003cnumber\u003e()\n  .define({ exp: \"number\" }, (d) =\u003e\n    // the result of the reducer will be stored in the node's value\n    d.reducer(({ matched }) =\u003e Number(matched[0].text)),\n  )\n  .define({ exp: `'-' exp` }, (d) =\u003e d.reducer(({ values }) =\u003e -values[1]!))\n  .define({ exp: `'(' exp ')'` }, (d) =\u003e d.reducer(({ values }) =\u003e values[1]))\n  .define({ exp: `exp '+' exp` }, (d) =\u003e\n    d.reducer(({ values }) =\u003e values[0]! + values[2]!),\n  )\n  .define({ exp: `exp '-' exp` }, (d) =\u003e\n    d.reducer(({ values }) =\u003e values[0]! - values[2]!),\n  )\n  .define({ exp: `exp '*' exp` }, (d) =\u003e\n    d.reducer(({ values }) =\u003e values[0]! * values[2]!),\n  )\n  .define({ exp: `exp '/' exp` }, (d) =\u003e\n    d.reducer(({ values }) =\u003e values[0]! / values[2]!),\n  )\n  .priority(\n    { exp: `'-' exp` }, // highest priority\n    [{ exp: `exp '*' exp` }, { exp: `exp '/' exp` }],\n    [{ exp: `exp '+' exp` }, { exp: `exp '-' exp` }], // lowest priority\n  );\n```\n\n\u003c/details\u003e\n\n## Contribute\n\nAll issues and pull requests are highly welcomed.\n\n## [CHANGELOG](https://github.com/DiscreteTom/retsac/blob/main/CHANGELOG.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiscretetom%2Fretsac","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdiscretetom%2Fretsac","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiscretetom%2Fretsac/lists"}