{"id":26610507,"url":"https://github.com/iwillspeak/firethorn","last_synced_at":"2025-04-10T00:04:54.264Z","repository":{"id":40576759,"uuid":"430342788","full_name":"iwillspeak/Firethorn","owner":"iwillspeak","description":"Implementation of Red / Green syntax trees. Inspired by Rowan","archived":false,"fork":false,"pushed_at":"2024-06-23T22:04:49.000Z","size":47,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-10T00:04:46.466Z","etag":null,"topics":["ast","parse-trees"],"latest_commit_sha":null,"homepage":"","language":"F#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iwillspeak.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-21T11:08:02.000Z","updated_at":"2025-01-18T09:13:14.000Z","dependencies_parsed_at":"2024-06-23T22:48:35.561Z","dependency_job_id":null,"html_url":"https://github.com/iwillspeak/Firethorn","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iwillspeak%2FFirethorn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iwillspeak%2FFirethorn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iwillspeak%2FFirethorn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iwillspeak%2FFirethorn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iwillspeak","download_url":"https://codeload.github.com/iwillspeak/Firethorn/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248131320,"owners_count":21052819,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast","parse-trees"],"created_at":"2025-03-24T01:48:30.080Z","updated_at":"2025-04-10T00:04:54.240Z","avatar_url":"https://github.com/iwillspeak.png","language":"F#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Firethorn\n\nRed-Green syntax trees for F#. Inspired by Rowan.\n\n## Design Goals\n\n * Compact representation of trees.\n * Complete representation of the input text.\n * Trees should be cheaply updateable.\n * Handle partial data and errors.\n * Query nodes in the tree by their location in the source.\n\n## Architecture\n\nThere are two main layers to the syntax trees supported by\nFirethorn. The `Green` tree represents abstract syntactic data; the\n`Red` or `Syntax` tree attaches location and parent information to\ngreen tree nodes.\n\nTrees which share structure can share portions of their green\ntree. e.g. in `1 + 1` the node for `1` may be the same green node. The\nred nodes however would differ as the locations within the tree are\ndifferent.\n\nA simplified definition of the green tree contains the following:\n\n```f#\ntype GreenToken =\n\t{ Kind: SyntaxKind\n\t  Text: string }\nand GreenNode =\n\t{ Kind: SyntaxKind\n\t  Width: int\n\t  Children: Choice\u003cGreenNode, GreenToken\u003e list }\n```\n\nIn the green tree each node has no specific location, only a\nwidth. The width of a token is the width of the text it contains. The\nwith of nodes is cached on the node. It is the sum of the widths of\nits children.\n\nThe red tree is the a layer over this tree similar to the following:\n\n```f#\ntype SyntaxToken =\n\t{ Offset: int\n\t  Parent: SyntaxNode option\n\t  Green: GreenToken }\nand SyntaxNode =\n\t{ Offset: int\n\t  Parent: SyntaxNode option\n\t  Green: GreenNode }\n```\n\nThis adds absolute offsets of the start of each node. This means a\ngreen node on its own has no specific location. It only has a location\nderived from the offset of its parent red node.\n\n## AST Layer\n\nOn top of this low-level untyped tree we can then layer a typed\nAST. This is done by having `cast` methods for each node type that\ntake the underlying `SyntaxNode` and provide a typed wrapper. An\nexample for a simplified binary expression node:\n\n```F#\ntype OperatorSyntax(syntax: SyntaxToken) =\n\t\n\tstatic member Cast(node: SyntaxNode) =\n\t\tif node.Kind = SyntaxKind PLUS then\n\t\t\tSome(OperatorSyntax(node))\n\t\telse\n\t\t\tNone\n\ntype ConstantSyntax(syntax: SyntaxNode) =\n\n\tstatic member Cast(node: SyntaxNode) =\n\t\tif node.Kind = SyntaxKind CONST then\n\t\t\tSome(ConstantSyntax(node))\n\t\telse\n\t\t\tNone\n\nand BinarySyntax(syntax: SyntaxNode) =\n\t\n\tstatic member Cast(node: SyntaxNode) =\n\t\tif node.Kind = SyntaxKind BINEXPR then\n\t\t\tSome(BinarySyntax(node))\n\t\telse\n\t\t\tNone\n\n\tmember _.Left =\n\t\tsyntax.Children()\n\t\t|\u003e Seq.choose ExpressionSyntax.Cast\n\t\t|\u003e Seq.tryHead\n\n\tmember _.Operator =\n\t\tsyntax.ChildrenWithTokens()\n\t\t|\u003e Seq.tryPick(\n\t\t\tNodeOrToken.asToken\n\t\t\t\u003e\u003e (Option.bind OperatorSyntax.Cast)\n\t\t)\n\t\n\tmember _.Right =\n\t\tsyntax.Children()\n\t\t|\u003e Seq.choose ExpressionSyntax.Cast\n\t\t|\u003e Seq.skip 1\n\t\t|\u003e Seq.tryHead\n\nand ExpressionSyntax =\n\t| Bin of BinarySyntax\n\t| Const of ConstantSyntax\n\t\n\tstatic member Cast(node: SyntaxNode) =\n\t\t(BinarySyntax.Cast node |\u003e Option.map Bin)\n\t\t|\u003e Option.orElseWith (fun () -\u003e ConstantSyntax.Cast |\u003e Option.map Const)\n\t\t\n```\n\nIn this model we have strongly typed tree structure defined by a union\ntype `ExpressionSyntax`. Each node type in the tree can expose the\nimportant parts of that node with strongly typed properties. All\nproperties return either `Option` or `Seq` values to model the fact\nthat any part of the tree could be missing or empty.\n\n## Building Parse Trees\n\nTrees can be built from the bottom up by repeatedly calling `GreenToken.Create` and `GreenNode.Create`. The final node can then be converted into a red tree with the `SyntaxNode.CreateRoot` method. This allows for ergonomic building of syntax during testing, and provides a simple API for programatically generated syntax such as macro expansion. For some simple languages and parsers this may be enough.\n\nA higher level API is also provided to build nodes using `GreenNodeBuilder`. This type allows for incremental building of nodes and is particularly suited to top-down parser construction:\n\n```f#\nlet builder = GreenNodeBuilder()\nbuilder.StartNode(SytaxKind BINOP)\nbuilder.Token(SyntaxKind OP, \"+\")\nbuilder.Token(SyntaxKind NUM, \"12\")\nbuilder.Token(SyntaxKind NUM, \"45\")\nbuilder.FinishNode()\n```\n\nFor cases where it isn't known ahead of time if a node will be needed the `Mark` API allows a point to be 'marked' and applied later to create a node. This would be equivalent to the above:\n\n```f#\nlet mark = builder.Mark()\nbuilder.Token(SyntaxKind OP, \"+\")\nbuilder.Token(SyntaxKind NUM, \"12\")\nbuilder.Token(SyntaxKind NUM, \"45\")\nbuilder.ApplyMark(mark, SytaxKind BINOP)\n```\n\nOnce a tee is built it can be converted into a single root node with the `BuildRoot` API:\n\n```f#\nlet syntaxRoot = builder.BuildRoot(SyntaxKind PROGRAM)\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiwillspeak%2Ffirethorn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiwillspeak%2Ffirethorn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiwillspeak%2Ffirethorn/lists"}