{"id":13440783,"url":"https://github.com/alecthomas/participle","last_synced_at":"2025-05-13T11:09:22.575Z","repository":{"id":37332756,"uuid":"62147469","full_name":"alecthomas/participle","owner":"alecthomas","description":"A parser library for Go","archived":false,"fork":false,"pushed_at":"2025-05-07T13:15:00.000Z","size":999,"stargazers_count":3692,"open_issues_count":28,"forks_count":195,"subscribers_count":48,"default_branch":"master","last_synced_at":"2025-05-13T11:09:13.445Z","etag":null,"topics":["ast","ebnf","go","golang","grammar","lexer","parser","parser-api","parser-framework","parser-library","participle"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alecthomas.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":null,"funding":".github/FUNDING.yml","license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["alecthomas"]}},"created_at":"2016-06-28T14:25:14.000Z","updated_at":"2025-05-12T21:36:41.000Z","dependencies_parsed_at":"2023-01-26T10:31:30.615Z","dependency_job_id":"2e6cbc9f-eb89-468c-a162-bceeb0a5d335","html_url":"https://github.com/alecthomas/participle","commit_stats":{"total_commits":397,"total_committers":35,"mean_commits":"11.342857142857143","dds":"0.21914357682619645","last_synced_commit":"f13baaebbd623f027c9b86ef35b0649b4befd622"},"previous_names":["alecthomas/parser"],"tags_count":37,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alecthomas%2Fparticiple","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alecthomas%2Fparticiple/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alecthomas%2Fparticiple/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alecthomas%2Fparticiple/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alecthomas","download_url":"https://codeload.github.com/alecthomas/participle/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253929367,"owners_count":21985802,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast","ebnf","go","golang","grammar","lexer","parser","parser-api","parser-framework","parser-library","participle"],"created_at":"2024-07-31T03:01:26.162Z","updated_at":"2025-05-13T11:09:22.550Z","avatar_url":"https://github.com/alecthomas.png","language":"Go","readme":"# A dead simple parser package for Go\n\u003ca id=\"markdown-a-dead-simple-parser-package-for-go\" name=\"a-dead-simple-parser-package-for-go\"\u003e\u003c/a\u003e\n\n[![PkgGoDev](https://pkg.go.dev/badge/github.com/alecthomas/participle/v2)](https://pkg.go.dev/github.com/alecthomas/participle/v2) [![GHA Build](https://github.com/alecthomas/participle/actions/workflows/ci.yml/badge.svg)](https://github.com/alecthomas/participle/actions)\n [![Go Report Card](https://goreportcard.com/badge/github.com/alecthomas/participle/v2)](https://goreportcard.com/report/github.com/alecthomas/participle/v2) [![Slack chat](https://img.shields.io/static/v1?logo=slack\u0026style=flat\u0026label=slack\u0026color=green\u0026message=gophers)](https://gophers.slack.com/messages/CN9DS8YF3)\n\n\u003c!-- MarkdownTOC autolink=\"true\" lowercase=\"only_ascii\" --\u003e\n\n- [V2](#v2)\n- [Introduction](#introduction)\n- [Tutorial](#tutorial)\n- [Tag syntax](#tag-syntax)\n- [Overview](#overview)\n- [Grammar syntax](#grammar-syntax)\n- [Capturing](#capturing)\n\t- [Capturing boolean value](#capturing-boolean-value)\n- [\"Union\" types](#union-types)\n- [Custom parsing](#custom-parsing)\n- [Lexing](#lexing)\n\t- [Stateful lexer](#stateful-lexer)\n\t- [Example stateful lexer](#example-stateful-lexer)\n\t- [Example simple/non-stateful lexer](#example-simplenon-stateful-lexer)\n\t- [Experimental - code generation](#experimental---code-generation)\n- [Options](#options)\n- [Examples](#examples)\n- [Performance](#performance)\n- [Concurrency](#concurrency)\n- [Error reporting](#error-reporting)\n- [Comments](#comments)\n- [Limitations](#limitations)\n- [EBNF](#ebnf)\n- [Syntax/Railroad Diagrams](#syntaxrailroad-diagrams)\n\n\u003c!-- /MarkdownTOC --\u003e\n\n## V2\n\nThis is version 2 of Participle.\n\nIt can be installed with:\n\n```shell\n$ go get github.com/alecthomas/participle/v2@latest\n```\n\nThe latest version from v0 can be installed via:\n\n```shell\n$ go get github.com/alecthomas/participle@latest\n```\n\n## \u003ca name='Introduction'\u003e\u003c/a\u003eIntroduction\n\nThe goal of this package is to provide a simple, idiomatic and elegant way of\ndefining parsers in Go.\n\nParticiple's method of defining grammars should be familiar to any Go\nprogrammer who has used the `encoding/json` package: struct field tags define\nwhat and how input is mapped to those same fields. This is not unusual for Go\nencoders, but is unusual for a parser.\n\n## Tutorial\n\nA [tutorial](TUTORIAL.md) is available, walking through the creation of an .ini parser.\n\n## Tag syntax\n\nParticiple supports two forms of struct tag grammar syntax.\n\nThe easiest to read is when the grammar uses the entire struct tag content, eg.\n\n```go\nField string `@Ident @(\",\" Ident)*`\n```\n\nHowever, this does not coexist well with other tags such as JSON, etc. and\nmay cause issues with linters. If this is an issue then you can use the\n`parser:\"\"` tag format. In this case single quotes can be used to quote\nliterals making the tags somewhat easier to write, eg.\n\n```go\nField string `parser:\"@ident (',' Ident)*\" json:\"field\"`\n```\n\n\n\n## Overview\n\nA grammar is an annotated Go structure used to both define the parser grammar,\nand be the AST output by the parser. As an example, following is the final INI\nparser from the tutorial.\n\n ```go\n type INI struct {\n   Properties []*Property `@@*`\n   Sections   []*Section  `@@*`\n }\n\n type Section struct {\n   Identifier string      `\"[\" @Ident \"]\"`\n   Properties []*Property `@@*`\n }\n\n type Property struct {\n   Key   string `@Ident \"=\"`\n   Value *Value `@@`\n }\n\n type Value struct {\n   String *string  `  @String`\n   Float *float64  `| @Float`\n   Int    *int     `| @Int`\n }\n ```\n\n\u003e **Note:** Participle also supports named struct tags (eg. \u003ccode\u003eHello string \u0026#96;parser:\"@Ident\"\u0026#96;\u003c/code\u003e).\n\nA parser is constructed from a grammar and a lexer:\n\n```go\nparser, err := participle.Build[INI]()\n```\n\nOnce constructed, the parser is applied to input to produce an AST:\n\n```go\nast, err := parser.ParseString(\"\", \"size = 10\")\n// ast == \u0026INI{\n//   Properties: []*Property{\n//     {Key: \"size\", Value: \u0026Value{Int: \u002610}},\n//   },\n// }\n```\n\n## Grammar syntax\n\nParticiple grammars are defined as tagged Go structures. Participle will\nfirst look for tags in the form `parser:\"...\"`. It will then fall back to\nusing the entire tag body.\n\nThe grammar format is:\n\n- `@\u003cexpr\u003e` Capture expression into the field.\n- `@@` Recursively capture using the fields own type.\n- `\u003cidentifier\u003e` Match named lexer token.\n- `( ... )` Group.\n- `\"...\"` or `'...'` Match the literal (note that the lexer must emit tokens matching this literal exactly).\n- `\"...\":\u003cidentifier\u003e` Match the literal, specifying the exact lexer token type to match.\n- `\u003cexpr\u003e \u003cexpr\u003e ...` Match expressions.\n- `\u003cexpr\u003e | \u003cexpr\u003e | ...` Match one of the alternatives. Each alternative is tried in order, with backtracking.\n- `~\u003cexpr\u003e` Match any token that is _not_ the start of the expression (eg: `@~\";\"` matches anything but the `;` character into the field).\n- `(?= ... )` Positive lookahead group - requires the contents to match further input, without consuming it.\n- `(?! ... )` Negative lookahead group - requires the contents not to match further input, without consuming it.\n\nThe following modifiers can be used after any expression:\n\n- `*` Expression can match zero or more times.\n- `+` Expression must match one or more times.\n- `?` Expression can match zero or once.\n- `!` Require a non-empty match (this is useful with a sequence of optional matches eg. `(\"a\"? \"b\"? \"c\"?)!`).\n\nNotes:\n\n- Each struct is a single production, with each field applied in sequence.\n- `@\u003cexpr\u003e` is the mechanism for capturing matches into the field.\n- if a struct field is not keyed with \"parser\", the entire struct tag\n  will be used as the grammar fragment. This allows the grammar syntax to remain\n  clear and simple to maintain.\n\n## Capturing\n\nPrefixing any expression in the grammar with `@` will capture matching values\nfor that expression into the corresponding field.\n\nFor example:\n\n```go\n// The grammar definition.\ntype Grammar struct {\n  Hello string `@Ident`\n}\n\n// The source text to parse.\nsource := \"world\"\n\n// After parsing, the resulting AST.\nresult == \u0026Grammar{\n  Hello: \"world\",\n}\n```\n\nFor slice and string fields, each instance of `@` will accumulate into the\nfield (including repeated patterns). Accumulation into other types is not\nsupported.\n\nFor integer and floating point types, a successful capture will be parsed\nwith `strconv.ParseInt()` and `strconv.ParseFloat()` respectively.\n\nA successful capture match into a `bool` field will set the field to true.\n\nTokens can also be captured directly into fields of type `lexer.Token` and\n`[]lexer.Token`.\n\nCustom control of how values are captured into fields can be achieved by a\nfield type implementing the `Capture` interface (`Capture(values []string)\nerror`).\n\nAdditionally, any field implementing the `encoding.TextUnmarshaler` interface\nwill be capturable too. One caveat is that `UnmarshalText()` will be called once\nfor each captured token, so eg. `@(Ident Ident Ident)` will be called three times.\n\n### Capturing boolean value\n\nBy default, a boolean field is used to indicate that a match occurred, which\nturns out to be much more useful and common in Participle than parsing true\nor false literals. For example, parsing a variable declaration with a\ntrailing optional syntax:\n\n```go\ntype Var struct {\n  Name string `\"var\" @Ident`\n  Type string `\":\" @Ident`\n  Optional bool `@\"?\"?`\n}\n```\n\nIn practice this gives more useful ASTs. If bool were to be parsed literally\nthen you'd need to have some alternate type for Optional such as string or a\ncustom type.\n\nTo capture literal boolean values such as `true` or `false`, implement the\nCapture interface like so:\n\n```go\ntype Boolean bool\n\nfunc (b *Boolean) Capture(values []string) error {\n\t*b = values[0] == \"true\"\n\treturn nil\n}\n\ntype Value struct {\n\tFloat  *float64 `  @Float`\n\tInt    *int     `| @Int`\n\tString *string  `| @String`\n\tBool   *Boolean `| @(\"true\" | \"false\")`\n}\n```\n\n## \"Union\" types\n\nA very common pattern in parsers is \"union\" types, an example of which is\nshown above in the `Value` type. A common way of expressing this in Go is via\na sealed interface, with each member of the union implementing this\ninterface.\n\neg. this is how the `Value` type could be expressed in this way:\n\n```go\ntype Value interface { value() }\n\ntype Float struct { Value float64 `@Float` }\nfunc (f Float) value() {}\n\ntype Int struct { Value int `@Int` }\nfunc (f Int) value() {}\n\ntype String struct { Value string `@String` }\nfunc (f String) value() {}\n\ntype Bool struct { Value Boolean `@(\"true\" | \"false\")` }\nfunc (f Bool) value() {}\n```\n\nThanks to the efforts of [Jacob Ryan McCollum](https://github.com/mccolljr), Participle\nnow supports this pattern. Simply construct your parser with the `Union[T](member...T)`\noption, eg.\n\n```go\nparser := participle.MustBuild[AST](participle.Union[Value](Float{}, Int{}, String{}, Bool{}))\n```\n\nCustom parsers may also be defined for union types with the [ParseTypeWith](https://pkg.go.dev/github.com/alecthomas/participle/v2#ParseTypeWith) option.\n\n## Custom parsing\n\nThere are three ways of defining custom parsers for nodes in the grammar:\n\n1. Implement the [Capture](https://pkg.go.dev/github.com/alecthomas/participle/v2#Capture) interface.\n2. Implement the [Parseable](https://pkg.go.dev/github.com/alecthomas/participle/v2#Parseable) interface.\n3. Use the [ParseTypeWith](https://pkg.go.dev/github.com/alecthomas/participle/v2#ParseTypeWith) option to specify a custom parser for union interface types.\n\n\n## Lexing\n\nParticiple relies on distinct lexing and parsing phases. The lexer takes raw\nbytes and produces tokens which the parser consumes. The parser transforms\nthese tokens into Go values.\n\nThe default lexer, if one is not explicitly configured, is based on the Go\n`text/scanner` package and thus produces tokens for C/Go-like source code. This\nis surprisingly useful, but if you do require more control over lexing the\nincluded stateful [`participle/lexer`](#markdown-stateful-lexer) lexer should\ncover most other cases. If that in turn is not flexible enough, you can\nimplement your own lexer.\n\nConfigure your parser with a lexer using the `participle.Lexer()` option.\n\nTo use your own Lexer you will need to implement two interfaces:\n[Definition](https://pkg.go.dev/github.com/alecthomas/participle/v2/lexer#Definition)\n(and optionally [StringsDefinition](https://pkg.go.dev/github.com/alecthomas/participle/v2/lexer#StringDefinition) and [BytesDefinition](https://pkg.go.dev/github.com/alecthomas/participle/v2/lexer#BytesDefinition)) and [Lexer](https://pkg.go.dev/github.com/alecthomas/participle/v2/lexer#Lexer).\n\n### Stateful lexer\n\nIn addition to the default lexer, Participle includes an optional\nstateful/modal lexer which provides powerful yet convenient\nconstruction of most lexers.  (Notably, indentation based lexers cannot\nbe expressed using the `stateful` lexer -- for discussion of how these\nlexers can be implemented, see [#20](https://github.com/alecthomas/participle/issues/20)).\n\nIt is sometimes the case that a simple lexer cannot fully express the tokens\nrequired by a parser. The canonical example of this is interpolated strings\nwithin a larger language. eg.\n\n```go\nlet a = \"hello ${name + \", ${last + \"!\"}\"}\"\n```\n\nThis is impossible to tokenise with a normal lexer due to the arbitrarily\ndeep nesting of expressions. To support this case Participle's lexer is now\nstateful by default.\n\nThe lexer is a state machine defined by a map of rules keyed by the state\nname. Each rule within the state includes the name of the produced token, the\nregex to match, and an optional operation to apply when the rule matches.\n\nAs a convenience, any `Rule` starting with a lowercase letter will be elided\nfrom output, though it is recommended to use `participle.Elide()` instead, as it\nbetter integrates with the parser.\n\nLexing starts in the `Root` group. Each rule is matched in order, with the first\nsuccessful match producing a lexeme. If the matching rule has an associated Action\nit will be executed.\n\nA state change can be introduced with the Action `Push(state)`. `Pop()` will\nreturn to the previous state.\n\nTo reuse rules from another state, use `Include(state)`.\n\nA special named rule `Return()` can also be used as the final rule in a state\nto always return to the previous state.\n\nAs a special case, regexes containing backrefs in the form `\\N` (where `N` is\na digit) will match the corresponding capture group from the immediate parent\ngroup. This can be used to parse, among other things, heredocs. See the\n[tests](https://github.com/alecthomas/participle/blob/master/lexer/stateful_test.go#L59)\nfor an example of this, among others.\n\n### Example stateful lexer\n\nHere's a cut down example of the string interpolation described above. Refer to\nthe [stateful example](https://github.com/alecthomas/participle/tree/master/_examples/stateful)\nfor the corresponding parser.\n\n```go\nvar lexer = lexer.Must(Rules{\n\t\"Root\": {\n\t\t{`String`, `\"`, Push(\"String\")},\n\t},\n\t\"String\": {\n\t\t{\"Escaped\", `\\\\.`, nil},\n\t\t{\"StringEnd\", `\"`, Pop()},\n\t\t{\"Expr\", `\\${`, Push(\"Expr\")},\n\t\t{\"Char\", `[^$\"\\\\]+`, nil},\n\t},\n\t\"Expr\": {\n\t\tInclude(\"Root\"),\n\t\t{`whitespace`, `\\s+`, nil},\n\t\t{`Oper`, `[-+/*%]`, nil},\n\t\t{\"Ident\", `\\w+`, nil},\n\t\t{\"ExprEnd\", `}`, Pop()},\n\t},\n})\n```\n\n### Example simple/non-stateful lexer\n\nOther than the default and stateful lexers, it's easy to define your\nown _stateless_ lexer using the `lexer.MustSimple()` and\n`lexer.NewSimple()` functions.  These functions accept a slice of\n`lexer.SimpleRule{}` objects consisting of a key and a regex-style pattern.\n\n\u003e **Note:** The stateful lexer replaces the old regex lexer.\n\nFor example, the lexer for a form of BASIC:\n\n```go\nvar basicLexer = lexer.MustSimple([]lexer.SimpleRule{\n    {\"Comment\", `(?i)rem[^\\n]*`},\n    {\"String\", `\"(\\\\\"|[^\"])*\"`},\n    {\"Number\", `[-+]?(\\d*\\.)?\\d+`},\n    {\"Ident\", `[a-zA-Z_]\\w*`},\n    {\"Punct\", `[-[!@#$%^\u0026*()+_={}\\|:;\"'\u003c,\u003e.?/]|]`},\n    {\"EOL\", `[\\n\\r]+`},\n    {\"whitespace\", `[ \\t]+`},\n})\n```\n\n### Experimental - code generation\n\nParticiple v2 now has experimental support for generating code to perform\nlexing.\n\nThis will generally provide around a 10x improvement in lexing performance\nwhile producing O(1) garbage.\n\nTo use:\n1. Serialize the `stateful` lexer definition to a JSON file (pass to `json.Marshal`).\n2. Run the `participle` command (see `scripts/participle`) to generate go code from the lexer JSON definition. For example:\n```\nparticiple gen lexer \u003cpackage name\u003e [--name SomeCustomName] \u003c mylexer.json | gofmt \u003e mypackage/mylexer.go\n```\n(see `genLexer` in `conformance_test.go` for a more detailed example)\n\n3. When constructing your parser, use the generated lexer for your lexer definition, such as:\n```\nvar ParserDef = participle.MustBuild[someGrammer](participle.Lexer(mylexer.SomeCustomnameLexer))\n```\n\nConsider contributing to the tests in `conformance_test.go` if they do not\nappear to cover the types of expressions you are using the generated\nlexer.\n\n**Known limitations of the code generated lexer:**\n\n* The lexer is always greedy. e.g., the regex `\"[A-Z][A-Z][A-Z]?T\"` will not match `\"EST\"` in the generated lexer because the quest operator is a greedy match and does not \"give back\" to try other possibilities; you can overcome by using `|` if you have a non-greedy match, e.g., `\"[A-Z][A-Z]|(?:[A-Z]T|T)\"` will produce correct results in both lexers (see [#276](https://github.com/alecthomas/participle/issues/276) for more detail); this limitation allows the generated lexer to be very fast and memory efficient\n* Backreferences in regular expressions are not currently supported\n\n## Options\n\nThe Parser's behaviour can be configured via [Options](https://pkg.go.dev/github.com/alecthomas/participle/v2#Option).\n\n## Examples\n\nThere are several [examples included](https://github.com/alecthomas/participle/tree/master/_examples),\nsome of which are linked directly here. These examples should be run from the\n`_examples` subdirectory within a cloned copy of this repository.\n\nExample | Description\n--------|---------------\n[BASIC](https://github.com/alecthomas/participle/tree/master/_examples/basic) | A lexer, parser and interpreter for a [rudimentary dialect](https://caml.inria.fr/pub/docs/oreilly-book/html/book-ora058.html) of BASIC.\n[EBNF](https://github.com/alecthomas/participle/tree/master/_examples/ebnf) | Parser for the form of EBNF used by Go.\n[Expr](https://github.com/alecthomas/participle/tree/master/_examples/expr) | A basic mathematical expression parser and evaluator.\n[GraphQL](https://github.com/alecthomas/participle/tree/master/_examples/graphql) | Lexer+parser for GraphQL schemas\n[HCL](https://github.com/alecthomas/participle/tree/master/_examples/hcl) | A parser for the [HashiCorp Configuration Language](https://github.com/hashicorp/hcl).\n[INI](https://github.com/alecthomas/participle/tree/master/_examples/ini) | An INI file parser.\n[Protobuf](https://github.com/alecthomas/participle/tree/master/_examples/protobuf) | A full [Protobuf](https://developers.google.com/protocol-buffers/) version 2 and 3 parser.\n[SQL](https://github.com/alecthomas/participle/tree/master/_examples/sql) | A *very* rudimentary SQL SELECT parser.\n[Stateful](https://github.com/alecthomas/participle/tree/master/_examples/stateful) | A basic example of a stateful lexer and corresponding parser.\n[Thrift](https://github.com/alecthomas/participle/tree/master/_examples/thrift) | A full [Thrift](https://thrift.apache.org/docs/idl) parser.\n[TOML](https://github.com/alecthomas/participle/tree/master/_examples/toml) | A [TOML](https://github.com/toml-lang/toml) parser.\n\nIncluded below is a full GraphQL lexer and parser:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\n\t\"github.com/alecthomas/kong\"\n\t\"github.com/alecthomas/repr\"\n\n\t\"github.com/alecthomas/participle/v2\"\n\t\"github.com/alecthomas/participle/v2/lexer\"\n)\n\ntype File struct {\n\tEntries []*Entry `@@*`\n}\n\ntype Entry struct {\n\tType   *Type   `  @@`\n\tSchema *Schema `| @@`\n\tEnum   *Enum   `| @@`\n\tScalar string  `| \"scalar\" @Ident`\n}\n\ntype Enum struct {\n\tName  string   `\"enum\" @Ident`\n\tCases []string `\"{\" @Ident* \"}\"`\n}\n\ntype Schema struct {\n\tFields []*Field `\"schema\" \"{\" @@* \"}\"`\n}\n\ntype Type struct {\n\tName       string   `\"type\" @Ident`\n\tImplements string   `( \"implements\" @Ident )?`\n\tFields     []*Field `\"{\" @@* \"}\"`\n}\n\ntype Field struct {\n\tName       string      `@Ident`\n\tArguments  []*Argument `( \"(\" ( @@ ( \",\" @@ )* )? \")\" )?`\n\tType       *TypeRef    `\":\" @@`\n\tAnnotation string      `( \"@\" @Ident )?`\n}\n\ntype Argument struct {\n\tName    string   `@Ident`\n\tType    *TypeRef `\":\" @@`\n\tDefault *Value   `( \"=\" @@ )`\n}\n\ntype TypeRef struct {\n\tArray       *TypeRef `(   \"[\" @@ \"]\"`\n\tType        string   `  | @Ident )`\n\tNonNullable bool     `( @\"!\" )?`\n}\n\ntype Value struct {\n\tSymbol string `@Ident`\n}\n\nvar (\n\tgraphQLLexer = lexer.MustSimple([]lexer.SimpleRule{\n\t\t{\"Comment\", `(?:#|//)[^\\n]*\\n?`},\n\t\t{\"Ident\", `[a-zA-Z]\\w*`},\n\t\t{\"Number\", `(?:\\d*\\.)?\\d+`},\n\t\t{\"Punct\", `[-[!@#$%^\u0026*()+_={}\\|:;\"'\u003c,\u003e.?/]|]`},\n\t\t{\"Whitespace\", `[ \\t\\n\\r]+`},\n\t})\n\tparser = participle.MustBuild[File](\n\t\tparticiple.Lexer(graphQLLexer),\n\t\tparticiple.Elide(\"Comment\", \"Whitespace\"),\n\t\tparticiple.UseLookahead(2),\n\t)\n)\n\nvar cli struct {\n\tEBNF  bool     `help\"Dump EBNF.\"`\n\tFiles []string `arg:\"\" optional:\"\" type:\"existingfile\" help:\"GraphQL schema files to parse.\"`\n}\n\nfunc main() {\n\tctx := kong.Parse(\u0026cli)\n\tif cli.EBNF {\n\t\tfmt.Println(parser.String())\n\t\tctx.Exit(0)\n\t}\n\tfor _, file := range cli.Files {\n\t\tr, err := os.Open(file)\n\t\tctx.FatalIfErrorf(err)\n\t\tast, err := parser.Parse(file, r)\n\t\tr.Close()\n\t\trepr.Println(ast)\n\t\tctx.FatalIfErrorf(err)\n\t}\n}\n```\n\n## Performance\n\nOne of the included examples is a complete Thrift parser\n(shell-style comments are not supported). This gives\na convenient baseline for comparing to the PEG based\n[pigeon](https://github.com/PuerkitoBio/pigeon), which is the parser used by\n[go-thrift](https://github.com/samuel/go-thrift). Additionally, the pigeon\nparser is utilising a generated parser, while the participle parser is built at\nrun time.\n\nYou can run the benchmarks yourself, but here's the output on my machine:\n\n    BenchmarkParticipleThrift-12    \t   5941\t   201242 ns/op\t 178088 B/op\t   2390 allocs/op\n    BenchmarkGoThriftParser-12      \t   3196\t   379226 ns/op\t 157560 B/op\t   2644 allocs/op\n\nOn a real life codebase of 47K lines of Thrift, Participle takes 200ms and go-\nthrift takes 630ms, which aligns quite closely with the benchmarks.\n\n## Concurrency\n\nA compiled `Parser` instance can be used concurrently. A `LexerDefinition` can be used concurrently. A `Lexer` instance cannot be used concurrently.\n\n## Error reporting\n\nThere are a few areas where Participle can provide useful feedback to users of your parser.\n\n1. Errors returned by [Parser.Parse*()](https://pkg.go.dev/github.com/alecthomas/participle/v2#Parser.Parse) will be:\n\t1. Of type [Error](https://pkg.go.dev/github.com/alecthomas/participle/v2#Error). This will contain positional information where available.\n\t2. May either be [ParseError](https://pkg.go.dev/github.com/alecthomas/participle/v2#ParseError) or [lexer.Error](https://pkg.go.dev/github.com/alecthomas/participle/v2/lexer#Error)\n2. Participle will make a best effort to return as much of the AST up to the error location as possible.\n3. Any node in the AST containing a field `Pos lexer.Position` [^1] will be automatically\n   populated from the nearest matching token.\n4. Any node in the AST containing a field `EndPos lexer.Position` [^1] will be\n   automatically populated from the token at the end of the node.\n5. Any node in the AST containing a field `Tokens []lexer.Token` will be automatically\n   populated with _all_ tokens captured by the node, _including_ elided tokens.\n\n[^1]: Either the concrete type or a type convertible to it, allowing user defined types to be used.\n\nThese related pieces of information can be combined to provide fairly comprehensive error reporting.\n\n## Comments\n\nComments can be difficult to capture as in most languages they may appear almost\nanywhere. There are three ways of capturing comments, with decreasing fidelity.\n\nThe first is to elide tokens in the parser, then add `Tokens []lexer.Token` as a\nfield to each AST node. Comments will be included. This has the downside that\nthere's no straightforward way to know where the comments are relative to\nnon-comment tokens in that node.\n\nThe second way is to _not_ elide comment tokens, and explicitly capture them at\nevery location in the AST where they might occur. This has the downside that\nunless you place these captures in every possible valid location, users might\ninsert valid comments that then fail to parse.\n\nThe third way is to elide comment tokens and capture them where they're\nsemantically meaningful, such as for documentation comments. Participle supports\nexplicitly matching elided tokens for this purpose.\n\n## Limitations\n\nInternally, Participle is a recursive descent parser with backtracking (see\n`UseLookahead(K)`).\n\nAmong other things, this means that Participle grammars do not support left\nrecursion. Left recursion must be eliminated by restructuring your grammar.\n\n## EBNF\n\nThe old `EBNF` lexer was removed in a major refactoring at\n[362b26](https://github.com/alecthomas/participle/commit/362b26640fa3dc406aa60960f7d9a5b9a909414e)\n-- if you have an EBNF grammar you need to implement, you can either translate\nit into regex-style `lexer.Rule{}` syntax or implement your own EBNF lexer\nyou might be able to use [the old EBNF lexer](https://github.com/alecthomas/participle/blob/2403858c8b2068b4b0cf96a6b36dd7069674039b/lexer/ebnf/ebnf.go)\n-- as a starting point.\n\nParticiple supports outputting an EBNF grammar from a Participle parser. Once\nthe parser is constructed simply call `String()`.\n\nParticiple also [includes a parser](https://pkg.go.dev/github.com/alecthomas/participle/v2/ebnf) for this form of EBNF (naturally).\n\neg. The [GraphQL example](https://github.com/alecthomas/participle/blob/master/_examples/graphql/main.go#L15-L62)\ngives in the following EBNF:\n\n```ebnf\nFile = Entry* .\nEntry = Type | Schema | Enum | \"scalar\" ident .\nType = \"type\" ident (\"implements\" ident)? \"{\" Field* \"}\" .\nField = ident (\"(\" (Argument (\",\" Argument)*)? \")\")? \":\" TypeRef (\"@\" ident)? .\nArgument = ident \":\" TypeRef (\"=\" Value)? .\nTypeRef = \"[\" TypeRef \"]\" | ident \"!\"? .\nValue = ident .\nSchema = \"schema\" \"{\" Field* \"}\" .\nEnum = \"enum\" ident \"{\" ident* \"}\" .\n```\n\n## Syntax/Railroad Diagrams\n\nParticiple includes a [command-line utility](https://github.com/alecthomas/participle/tree/master/cmd/railroad) to take an EBNF representation of a Participle grammar\n(as returned by `Parser.String()`) and produce a Railroad Diagram using\n[tabatkins/railroad-diagrams](https://github.com/tabatkins/railroad-diagrams).\n\nHere's what the GraphQL grammar looks like:\n\n![EBNF Railroad Diagram](railroad.png)\n","funding_links":["https://github.com/sponsors/alecthomas"],"categories":["HarmonyOS","Uncategorized","开源类库","Go","Open source library","Repositories"],"sub_categories":["Windows Manager","Uncategorized","解释器","Interpreter"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falecthomas%2Fparticiple","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falecthomas%2Fparticiple","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falecthomas%2Fparticiple/lists"}