{"id":15713551,"url":"https://github.com/homonoidian/synthax","last_synced_at":"2025-03-30T18:46:43.058Z","repository":{"id":207971701,"uuid":"720533192","full_name":"homonoidian/synthax","owner":"homonoidian","description":"Synthax is a simple parser synthesizer for Crystal","archived":false,"fork":false,"pushed_at":"2024-04-17T17:32:14.000Z","size":857,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"trunk","last_synced_at":"2025-02-05T21:25:59.574Z","etag":null,"topics":["crystal","parser-library"],"latest_commit_sha":null,"homepage":"https://homonoidian.github.io/synthax/","language":"Crystal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/homonoidian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-18T19:16:36.000Z","updated_at":"2024-04-12T20:59:36.000Z","dependencies_parsed_at":"2024-02-08T01:22:45.176Z","dependency_job_id":"ee36e8a4-303c-4c96-926a-d37c3f983694","html_url":"https://github.com/homonoidian/synthax","commit_stats":null,"previous_names":["homonoidian/synthax"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/homonoidian%2Fsynthax","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/homonoidian%2Fsynthax/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/homonoidian%2Fsynthax/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/homonoidian%2Fsynthax/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/homonoidian","download_url":"https://codeload.github.com/homonoidian/synthax/tar.gz/refs/heads/trunk","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246365637,"owners_count":20765546,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crystal","parser-library"],"created_at":"2024-10-03T21:32:04.581Z","updated_at":"2025-03-30T18:46:42.827Z","avatar_url":"https://github.com/homonoidian.png","language":"Crystal","funding_links":[],"categories":[],"sub_categories":[],"readme":"# synthax\n\nSynthax is a simple parser synthesizer for Crystal.\n\n```crystal\n# JSON grammar\n\nws = some(' ' | '\\r' | '\\n' | '\\t')\ndigit = '0'..'9'\ndigits = many(digit)\ninteger = maybe('-') \u0026 ('0' | ('1'..'9') \u0026 some(digit))\nfraction = '.' \u0026 digits\nexponent = ('E' | 'e') \u0026 ('+' | '-') \u0026 digits\nnumber = keep(integer \u0026 maybe(fraction) \u0026 maybe(exponent), \"number:value\")\nhex = digit | ('A'..'F') | ('a'..'f')\nescape = '\"' | '\\\\' | '/' | 'b' | 'f' | 'n' | 'r' | 't' | ('u' \u0026 hex \u0026 hex \u0026 hex \u0026 hex)\ncharacter = ((0x0020..0x10FFFF) - '\"' - '\\\\') | ('\\\\' \u0026 escape)\nstring = '\"' \u0026 keep(some(character), \"string:value\") \u0026 '\"'\nvalue = ahead\nelement = ws \u0026 value \u0026 ws\nelements = sep(element, by: ',')\narray = '[' \u0026 (elements | ws) \u0026 ']'\nmember = capture(ws \u0026 string \u0026 ':' \u0026 element, \"pair\")\nmembers = sep(member, by: ',')\nobject = '{' \u0026 (members | ws) \u0026 '}'\nvalue.put \\\n  capture(object) |\n  capture(array) |\n  capture(string) |\n  capture(number) |\n  lit(\"true\") |\n  lit(\"false\") |\n  lit(\"null\")\n\njson = element\n```\n\n## Installation\n\n1. Add the dependency to your `shard.yml`:\n\n   ```yaml\n   dependencies:\n     synthax:\n       github: homonoidian/synthax\n   ```\n\n2. Run `shards install`\n\n## Usage\n\n- Basic:\n\n  ```crystal\n  require \"synthax\"\n\n  include Sthx::DSL\n\n  # Write rules here...\n\n  top = your_toplevel_rule\n\n  \"my string\".apply(top)  # =\u003e Sthx::Ctx | Sthx::Err\n  \"my string\".apply?(top) # =\u003e Sthx::Tree?\n  \"my string\".apply!(top) # =\u003e Sthx::Tree\n  ```\n\n- A bit more sophisticated:\n\n  ```crystal\n  require \"synthax\"\n\n  module Foo\n    include Sthx::DSL\n\n    GRAMMAR = grammar\n\n    def self.grammar\n      # Write rules here...\n\n      your_toplevel_rule\n    end\n  end\n\n  \"my string\".apply(Foo::GRAMMAR)  # =\u003e Sthx::Ctx | Sthx::Err\n  \"my string\".apply?(Foo::GRAMMAR) # =\u003e Sthx::Tree?\n  \"my string\".apply!(Foo::GRAMMAR) # =\u003e Sthx::Tree\n  ```\n\n- Rules and `Tree` are persistent and immutable. Linked list + path copying\n  is used where appropriate, for storing children and attributes `Pf::Map` is\n  used (hence the dependency on `permafrost`).\n\n- For all else [see the docs](https://homonoidian.github.io/synthax/)\n\n- See examples in the `examples/` directory.\n\n## `capture` and `keep`\n\nA `Tree` has *children* (`0` to some `N` of them) and *attributes* (string to string pairs).\n\n`capture(other, id)` lets you enclose trees produced by *other* in a new parent tree\nwith the given *id*. All `keep`s directly in *other* will add attributes onto this new\nparent tree. There is always an implicit root tree. It is the parent of all top level\ncaptures and keeps.\n\n`keep(other, id)` takes *the fragment of source code* matched by *other* and extends\nthe current tree with an attribute called *id*, with the matched fragment of source\ncode as the value. The tree produced by *other* is discarded. It's a bit like named\ncapture in regex.\n\n## Performance\n\nIt's pretty horrible but okay for that phase where you don't have thousands upon\nthousands of lines of code / frequent reparsing thereof. Fast parsing is the least\nof concerns when you're prototyping a language/etc.\n\nIf you need to go through millions of characters routinely this is the worst shard\nto pick I guess. I think recursive descent \u0026 a state-machine-ish lexer is better\nfor that purpose.\n\n- No lexer means each character must be processed by rules on the heap. This also\n  means that backtracking to explore another branch is much more expensive, requiring\n  to repeatedly revisit same parts of the string within a different context. The\n  grammar driving the parsing instead of the string makes it a much more painful\n  process in general (because the string always knows better). But indexing ain't\n  quick too.\n\n- Nothing fancy or theoretical is done here. The thing is extremely simple. Take\n  a look at the source code yourself.\n\nFor 10mb JSON example (including `anify`):\n\n```text\n        JSON.parse  10.86  ( 92.08ms) (± 5.62%)  33.9MB/op        fastest\nSynthax JSON parse   1.68  (596.44ms) (± 2.88%)   225MB/op   6.48× slower\n```\n\nTo run the benchmark yourself use: `crystal run examples/json.cr -Dbenchmark --release`\n\n- Memory usage is horrible due to `Sthx::Tree` overhead and children array\n  overhead when converting to `JSON::Any`, plus `JSON::Any` itself of course.\n  The children array cna be eliminated if you visit the `Sthx::Tree` yourself,\n  without using the convenience `Sthx::Tree#map` methods. You always know more\n  than those methods, so make use of that.\n\n- Parsing itself does not consume any memory because it's just recursively\n  exploring a graph (well, if we don't count the call stack of course!) But\n  you can't opt out of `Sthx::Tree` generation so haha live with it :)\n\n- I don't think it's currently possible to build something like that with generics,\n  handling captures and all (to construct client AST directly); it gets too nasty\n  too soon. And generally, generics caused more Crystal language bugs than anything\n  else for me, so I try not to venture too far into that territory.\n\n## Development\n\nJust do it.\n\n## Contributing\n\n1. Fork it (\u003chttps://github.com/homonoidian/synthax/fork\u003e)\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create a new Pull Request\n\n## Contributors\n\n- [Alexey Yurchenko](https://github.com/homonoidian) - creator and maintainer\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhomonoidian%2Fsynthax","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhomonoidian%2Fsynthax","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhomonoidian%2Fsynthax/lists"}