{"id":15654920,"url":"https://github.com/rmosolgo/lingo","last_synced_at":"2025-10-08T19:08:15.290Z","repository":{"id":45230493,"uuid":"46237642","full_name":"rmosolgo/lingo","owner":"rmosolgo","description":"parser generator ","archived":false,"fork":false,"pushed_at":"2021-12-29T21:34:13.000Z","size":58,"stargazers_count":28,"open_issues_count":0,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-01T09:56:26.468Z","etag":null,"topics":["crystal","parser","parser-generator"],"latest_commit_sha":null,"homepage":"","language":"Crystal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rmosolgo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-11-15T21:54:18.000Z","updated_at":"2024-09-29T05:19:27.000Z","dependencies_parsed_at":"2022-08-12T11:50:46.624Z","dependency_job_id":null,"html_url":"https://github.com/rmosolgo/lingo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmosolgo%2Flingo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmosolgo%2Flingo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmosolgo%2Flingo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rmosolgo%2Flingo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rmosolgo","download_url":"https://codeload.github.com/rmosolgo/lingo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251856992,"owners_count":21655119,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crystal","parser","parser-generator"],"created_at":"2024-10-03T12:54:51.398Z","updated_at":"2025-10-08T19:08:10.264Z","avatar_url":"https://github.com/rmosolgo.png","language":"Crystal","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lingo [![Build Status](https://travis-ci.org/rmosolgo/lingo.svg)](https://travis-ci.org/rmosolgo/lingo)\n\nA parser generator for Crystal, inspired by [Parslet](https://github.com/kschiess/parslet).\n\nLingo provides text processing by:\n- parsing the string into a tree of nodes\n- providing a visitor to allow you to work from the tree\n\n## Installation\n\nAdd this to your application's `shard.yml`:\n\n```yaml\ndependencies:\n  lingo:\n    github: rmosolgo/lingo\n```\n\n## Usage\n\nLet's write a parser for highway names. The result will be a method for turning strings into useful objects:\n\n```ruby\ndef parse_road(input_str)\n  ast = RoadParser.new.parse(input_str)\n  visitor = RoadVisitor.new\n  visitor.visit(ast)\n  visitor.road\nend\n\nroad = parse_road(\"I-5N\")\n# \u003cRoad @interstate=true, @number=5, @direction=\"N\"\u003e\n```\n\n(See more examples in [`/examples`](https://github.com/rmosolgo/lingo/tree/master/examples).)\n\nIn the USA, we write highway names like this:\n\n```\n50    # Route 50\nI-64  # Interstate 64\nI-95N # Interstate 95, Northbound\n29B   # Business Route 29\n```\n\n### Parser\n\nThe general structure is `{interstate?}{number}{direction?}{business?}`. Let's express that with Lingo rules:\n\n```ruby\nclass RoadParser \u003c Lingo::Parser\n  # Match a string:\n  rule(:interstate) { str(\"I-\") }\n  rule(:business) { str(\"B\") }\n\n  # Match a regex:\n  rule(:digit) { match(/\\d/) }\n  # Express repetition with `.repeat`\n  rule(:number) { digit.repeat }\n\n  rule(:north) { str(\"N\") }\n  rule(:south) { str(\"S\") }\n  rule(:east) { str(\"E\") }\n  rule(:west) { str(\"W\") }\n  # Compose rules by name\n  # Express alternation with |\n  rule(:direction) { north | south | east | west }\n\n  # Express sequence with \u003e\u003e\n  # Express optionality with `.maybe`\n  # Name matched strings with `.named`\n  rule(:road_name) {\n    interstate.named(:interstate).maybe \u003e\u003e\n      number.named(:number) \u003e\u003e\n      direction.named(:direction).maybe \u003e\u003e\n      business.named(:business).maybe\n  }\n  # You MUST name a starting rule:\n  root(:road_name)\nend\n```\n\n#### Applying the Parser\n\nAn instance of a `Lingo::Parser` subclass has a `.parse` method which returns a tree of `Lingo::Node`s.\n\n```ruby\nRoadParser.new.parse(\"250B\") # =\u003e \u003cLingo::Node ... \u003e\n```\n\nIt uses the rule named by `root`.\n\n#### Making Rules\n\nThese methods help you create rules:\n\n- `str(\"string\")` matches string exactly\n- `match(/[abc]/)` matches the regex exactly\n- `a | b` matches `a` _or_ `b`\n- `a \u003e\u003e b` matches `a` _followed by_ `b`\n- `a.maybe` matches `a` or nothing\n- `a.repeat` matches _one-or-more_ `a`s\n- `a.repeat(0)` matches _zero-or-more_ `a`s\n- `a.absent` matches _not-`a`_\n- `a.named(:a)` names the result `:a` for handling by a visitor\n\n### Visitor\n\nAfter parsing, you get a tree of `Lingo::Node`s. To turn that into an application object, write a visitor.\n\nThe visitor may define `enter` and `exit` hooks for nodes named with `.named` in the Parser. It may set up some state during `#initialize`, then access itself from the `visitor` variable during hooks.\n\n\n```ruby\nclass RoadVisitor \u003c Lingo::Visitor\n  # Set up an accumulator\n  getter :road\n  def initialize\n    @road = Road.new\n  end\n\n  # When you find a named node, you can do something with it.\n  # You can access the current visitor as `visitor`\n  enter(:interstate) {\n    # since we found this node, this is a business route\n    visitor.road.interstate = true\n  }\n\n  # You can access the named Lingo::Node as `node`.\n  # Get the matched string with `.full_value`\n  enter(:number) {\n    visitor.road.number = node.full_value.to_i\n  }\n\n  enter(:direction) {\n    visitor.road.direction = node.full_value\n  }\n\n  enter(:business) {\n    visitor.road.business = true\n  }\nend\n```\n\n#### Visitor Hooks\n\nDuring the depth-first visitation of the resulting tree of `Lingo::Node`s, you can handle visits to nodes named with `.named`:\n\n- `enter(:match)` is called when entering a node named `:match`\n- `exit(:match)` is called when exiting a node named `:match`\n\nWithin the hooks, you can access two magic variables:\n\n- `visitor` is the Visitor itself\n- `node` is the matched `Lingo::Node` which exposes:\n  - `#full_value`: the full matched string\n  - `#line`, `#column`: position information for this match\n\n## About this Project\n\n### Goals\n\n- Low barrier to entry: easy-to-learn API, short zero-to-working time\n- Easy-to-read code, therefore easy-to-modify\n- Useful errors (not accomplished)\n\n### Non-goals\n\n- Blazing-fast performance\n- Theoretical correctness\n\n### TODO\n\n- [ ] Add some kind of debug output\n\n### How slow is it?\n\nLet's compare the built-in JSON parser to a Lingo JSON parser:\n\n```\n./lingo/benchmark $ crystal run --release slow_json.cr\nStdlib JSON 126.45k (± 1.55%)        fastest\nLingo::JSON 660.18  (± 1.28%) 191.54× slower\n```\n\nOuch, that's __a lot slower__.\n\nBut, it's on par with Ruby and `parslet`, the inspiration for this project:\n\n```\n$ ruby parslet_json_benchmark.rb\nCalculating -------------------------------------\n       Parslet JSON      4.000  i/100ms\n       Built-in JSON     3.657k i/100ms\n-------------------------------------------------\n       Parslet JSON      45.788  (± 4.4%) i/s -    232.000\n       Built-in JSON     38.285k (± 5.3%) i/s -    193.821k\n\nComparison:\n       Built-in JSON:    38285.2 i/s\n       Parslet JSON :       45.8 i/s - 836.13x slower\n```\n\nBoth Parslet and Lingo are slower than handwritten parsers. But, they're easier to write!\n\n## Development\n\n- Run the __tests__ with `crystal spec`\n- Install Ruby \u0026 `guard`, then start a __watcher__ with `guard`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmosolgo%2Flingo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frmosolgo%2Flingo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frmosolgo%2Flingo/lists"}