{"id":23626281,"url":"https://github.com/datalust/superpower","last_synced_at":"2025-05-14T04:07:33.198Z","repository":{"id":38801746,"uuid":"67104473","full_name":"datalust/superpower","owner":"datalust","description":"A C# parser construction toolkit with high-quality error reporting","archived":false,"fork":false,"pushed_at":"2024-09-30T20:32:19.000Z","size":447,"stargazers_count":1178,"open_issues_count":10,"forks_count":104,"subscribers_count":43,"default_branch":"dev","last_synced_at":"2025-05-13T14:57:43.764Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datalust.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-09-01T06:29:59.000Z","updated_at":"2025-05-13T12:19:14.000Z","dependencies_parsed_at":"2024-12-27T22:56:20.572Z","dependency_job_id":"7afacacd-d059-4090-92e4-a36947a7e1d1","html_url":"https://github.com/datalust/superpower","commit_stats":{"total_commits":195,"total_committers":21,"mean_commits":9.285714285714286,"dds":"0.29230769230769227","last_synced_commit":"598d1a291d134c90119fda452c495660915de6f9"},"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datalust%2Fsuperpower","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datalust%2Fsuperpower/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datalust%2Fsuperpower/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datalust%2Fsuperpower/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datalust","download_url":"https://codeload.github.com/datalust/superpower/tar.gz/refs/heads/dev","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254069513,"owners_count":22009558,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-27T22:52:48.496Z","updated_at":"2025-05-14T04:07:33.166Z","avatar_url":"https://github.com/datalust.png","language":"C#","readme":"# Superpower [![Build status](https://ci.appveyor.com/api/projects/status/7bj6if6tyc68urpy?svg=true)](https://ci.appveyor.com/project/datalust/superpower) [![NuGet Version](https://img.shields.io/nuget/vpre/Superpower.svg?style=flat)](https://www.nuget.org/packages/Superpower/) [![Stack Overflow](https://img.shields.io/badge/stackoverflow-superpower-orange.svg)](http://stackoverflow.com/questions/tagged/superpower)\n\nA [parser combinator](https://en.wikipedia.org/wiki/Parser_combinator) library based on\n[Sprache](https://github.com/sprache/Sprache). Superpower generates friendlier error messages through its support for\ntoken-driven parsers.\n\n![Logo](https://raw.githubusercontent.com/datalust/superpower/dev/asset/Superpower-White-200px.png)\n\n### What is Superpower?\n\nThe job of a parser is to take a sequence of characters as input, and produce a data structure that's easier\nfor a program to analyze, manipulate, or transform. From this point of view, a parser is just a function from `string`\nto `T` - where `T` might be anything from a simple number, a list of fields in a data format, or the abstract syntax\ntree of some kind of programming language.\n\nJust like other kinds of functions, parsers can be built by hand, from scratch. This is-or-isn't a lot of fun, depending\non the complexity of the parser you need to build (and how you plan to spend your next few dozen nights and weekends).\n\nSuperpower is a library for writing parsers in a declarative style that mirrors\nthe structure of the target grammar. Parsers built with Superpower are fast, robust, and report precise and\ninformative errors when invalid input is encountered.\n\n### Usage\n\nSuperpower is embedded directly into your C# program, without the need for any additional tools or build-time code\ngeneration tasks.\n\n```shell\ndotnet add package Superpower\n```\n\nThe simplest _text parsers_ consume characters directly from the source text: \n\n```csharp\n// Parse any number of capital 'A's in a row\nvar parseA = Character.EqualTo('A').AtLeastOnce();\n```\n\nThe `Character.EqualTo()` method is a built-in parser. The `AtLeastOnce()` method is a _combinator_, that builds a more\ncomplex parser for a sequence of `'A'` characters out of the simple parser for a single `'A'`.\n\nSuperpower includes a library of simple parsers and combinators from which more sophisticated parsers can be built:\n\n```csharp\nTextParser\u003cstring\u003e identifier =\n    from first in Character.Letter\n    from rest in Character.LetterOrDigit.Or(Character.EqualTo('_')).Many()\n    select first + new string(rest);\n\nvar id = identifier.Parse(\"abc123\");\n\nAssert.Equal(\"abc123\", id);\n```\n\nParsers are highly modular, so smaller parsers can be built and tested independently of the larger parsers that use\nthem.\n\n### Tokenization\n\nAlong with text parsers that consume input character-by-character, Superpower supports _token parsers_.\n\nA token parser consumes elements from a list of tokens. A token is a fragment of the input text, tagged with the\nkind of item that fragment represents - usually specified using an `enum`:\n\n```csharp\npublic enum ArithmeticExpressionToken\n{\n    None,\n    Number,\n    Plus,\n```\n\nA major benefit of driving parsing from tokens, instead of individual characters, is that errors can be reported in\nterms of tokens - _unexpected identifier \\`frm\\`, expected keyword \\`from\\`_ - instead of the cryptic _unexpected \\`m\\`_.\n\nToken-driven parsing takes place in two distinct steps:\n\n 1. Tokenization, using a class derived from `Tokenizer\u003cTKind\u003e`, then\n 2. Parsing, using a function of type `TokenListParser\u003cTKind\u003e`.\n\n```csharp\nvar expression = \"1 * (2 + 3)\";\n\n// 1.\nvar tokenizer = new ArithmeticExpressionTokenizer();\nvar tokenList = tokenizer.Tokenize(expression);\n\n// 2.\nvar parser = ArithmeticExpressionParser.Lambda; // parser built with combinators\nvar expressionTree = parser.Parse(tokenList);\n\n// Use the result\nvar eval = expressionTree.Compile();\nConsole.WriteLine(eval()); // -\u003e 5\n```\n\n#### Assembling tokenizers with `TokenizerBuilder\u003cTKind\u003e`\n\nThe job of a _tokenizer_ is to split the input into a list of tokens - numbers, keywords, identifiers, operators -\nwhile discarding irrelevant trivia such as whitespace or comments.\n\nSuperpower provides the `TokenizerBuilder\u003cTKind\u003e` class to quickly assemble tokenizers from _recognizers_,\ntext parsers that match the various kinds of tokens required by the grammar.\n\nA simple arithmetic expression tokenizer is shown below:\n\n```csharp\nvar tokenizer = new TokenizerBuilder\u003cArithmeticExpressionToken\u003e()\n    .Ignore(Span.WhiteSpace)\n    .Match(Character.EqualTo('+'), ArithmeticExpressionToken.Plus)\n    .Match(Character.EqualTo('-'), ArithmeticExpressionToken.Minus)\n    .Match(Character.EqualTo('*'), ArithmeticExpressionToken.Times)\n    .Match(Character.EqualTo('/'), ArithmeticExpressionToken.Divide)\n    .Match(Character.EqualTo('('), ArithmeticExpressionToken.LParen)\n    .Match(Character.EqualTo(')'), ArithmeticExpressionToken.RParen)\n    .Match(Numerics.Natural, ArithmeticExpressionToken.Number)\n    .Build();\n```\n\nTokenizers constructed this way produce a list of tokens by repeatedly attempting to match recognizers \nagainst the input in top-to-bottom order.\n\n#### Writing tokenizers by hand\n\nTokenizers can alternatively be written by hand; this can provide the most flexibility, performance, and control,\nat the expense of more complicated code. A handwritten arithmetic expression tokenizer is included in the test suite,\nand a more complete example can be found [here](https://github.com/serilog/serilog-filters-expressions/blob/dev/src/Serilog.Filters.Expressions/Filters/Expressions/Parsing/FilterExpressionTokenizer.cs).\n\n#### Writing token list parsers\n\nToken parsers are defined in the same manner as text parsers, using combinators to build up more sophisticated parsers\nout of simpler ones.\n\n```csharp\nclass ArithmeticExpressionParser\n{\n    static readonly TokenListParser\u003cArithmeticExpressionToken, ExpressionType\u003e Add =\n        Token.EqualTo(ArithmeticExpressionToken.Plus).Value(ExpressionType.AddChecked);\n        \n    static readonly TokenListParser\u003cArithmeticExpressionToken, ExpressionType\u003e Subtract =\n        Token.EqualTo(ArithmeticExpressionToken.Minus).Value(ExpressionType.SubtractChecked);\n        \n    static readonly TokenListParser\u003cArithmeticExpressionToken, ExpressionType\u003e Multiply =\n        Token.EqualTo(ArithmeticExpressionToken.Times).Value(ExpressionType.MultiplyChecked);\n        \n    static readonly TokenListParser\u003cArithmeticExpressionToken, ExpressionType\u003e Divide = \n        Token.EqualTo(ArithmeticExpressionToken.Divide).Value(ExpressionType.Divide);\n\n    static readonly TokenListParser\u003cArithmeticExpressionToken, Expression\u003e Constant =\n            Token.EqualTo(ArithmeticExpressionToken.Number)\n            .Apply(Numerics.IntegerInt32)\n            .Select(n =\u003e (Expression)Expression.Constant(n));\n\n    static readonly TokenListParser\u003cArithmeticExpressionToken, Expression\u003e Factor =\n        (from lparen in Token.EqualTo(ArithmeticExpressionToken.LParen)\n            from expr in Parse.Ref(() =\u003e Expr)\n            from rparen in Token.EqualTo(ArithmeticExpressionToken.RParen)\n            select expr)\n        .Or(Constant);\n\n    static readonly TokenListParser\u003cArithmeticExpressionToken, Expression\u003e Operand =\n        (from sign in Token.EqualTo(ArithmeticExpressionToken.Minus)\n            from factor in Factor\n            select (Expression)Expression.Negate(factor))\n        .Or(Factor).Named(\"expression\");\n\n    static readonly TokenListParser\u003cArithmeticExpressionToken, Expression\u003e Term =\n        Parse.Chain(Multiply.Or(Divide), Operand, Expression.MakeBinary);\n\n    static readonly TokenListParser\u003cArithmeticExpressionToken, Expression\u003e Expr =\n        Parse.Chain(Add.Or(Subtract), Term, Expression.MakeBinary);\n\n    public static readonly TokenListParser\u003cArithmeticExpressionToken, Expression\u003cFunc\u003cint\u003e\u003e\u003e\n        Lambda = Expr.AtEnd().Select(body =\u003e Expression.Lambda\u003cFunc\u003cint\u003e\u003e(body));\n}\n```\n\n### Error messages\n\nThe [error scenario tests](https://github.com/datalust/superpower/blob/dev/test/Superpower.Tests/ErrorMessageScenarioTests.cs)\ndemonstrate some of the error message formatting capabilities of Superpower. Check out the parsers referenced in the \ntests for some examples.\n\n```csharp\nArithmeticExpressionParser.Lambda.Parse(new ArithmeticExpressionTokenizer().Tokenize(\"1 + * 3\"));\n     // -\u003e Syntax error (line 1, column 5): unexpected operator `*`, expected expression.\n```\n\nTo improve the error reporting for a particular token type, apply the `[Token]` attribute:\n\n```csharp\npublic enum ArithmeticExpressionToken\n{\n    None,\n\n    Number,\n\n    [Token(Category = \"operator\", Example = \"+\")]\n    Plus,\n```\n\n### Performance\n\nSuperpower is built with performance as a priority. Less frequent backtracking, combined with the avoidance of\nallocations and indirect dispatch, mean that Superpower can be quite a bit faster than Sprache.\n\nRecent benchmark for parsing a long arithmetic expression:\n\n```ini\nHost Process Environment Information:\nBenchmarkDotNet.Core=v0.9.9.0\nOS=Windows\nProcessor=?, ProcessorCount=8\nFrequency=2533306 ticks, Resolution=394.7411 ns, Timer=TSC\nCLR=CORE, Arch=64-bit ? [RyuJIT]\nGC=Concurrent Workstation\ndotnet cli version: 1.0.0-preview2-003121\n\nType=ArithmeticExpressionBenchmark  Mode=Throughput  \n\n```\n\n|          Method |      Median |     StdDev | Scaled | Scaled-SD |\n|---------------- |----------- |---------- |------ |--------- |\n|         Sprache | 283.8618 \u0026micro;s | 10.0276 \u0026micro;s |   1.00 |      0.00 |\n| Superpower (Token) |  81.1563 \u0026micro;s |  2.8775 \u0026micro;s |   0.29 |      0.01 |\n\nBenchmarks and results are included in the repository.\n\n**Tips:** if you find you need more throughput: 1) consider a hand-written tokenizer, and 2) avoid the use of LINQ comprehensions and instead use chained combinators like `Then()` and especially `IgnoreThen()` - these allocate fewer delegates (closures) during parsing.\n\n### Examples\n\nSuperpower is introduced, with a worked example, in [this blog post](https://nblumhardt.com/2016/09/superpower/).\n\n**Example** parsers to learn from:\n\n* [_JsonParser_](https://github.com/datalust/superpower/tree/dev/sample/JsonParser/Program.cs) is a complete, annotated\n example implementing the [JSON spec](https://json.org) with good error reporting\n* [_DateTimeTextParser_](https://github.com/datalust/superpower/tree/dev/sample/DateTimeTextParser) shows how Superpower's text parsers work, parsing ISO-8601 date-times\n* [_IntCalc_](https://github.com/datalust/superpower/tree/dev/sample/IntCalc) is a simple arithmetic expresion parser (`1 + 2 * 3`) included in the repository, demonstrating how Superpower token parsing works\n* [_Plotty_](https://github.com/SuperJMN/Plotty) implements an instruction set for a RISC virtual machine\n* [_tcalc_](https://github.com/nblumhardt/tcalc) is an example expression language that computes durations (`1d / 12m`)\n\n**Real-world** projects built with Superpower:\n\n* [_Serilog.Expressions_](https://github.com/serilog/serilog-expressions) uses Superpower to implement an expression and templating language for structured log events\n* The query language of [Seq](https://datalust.co/seq) is implemented using Superpower\n* `seqcli` [extraction patterns](https://github.com/datalust/seqcli#extraction-patterns) use Superpower for plain-text log parsing\n* [_PromQL.Parser_](https://github.com/djluck/PromQL.Parser) is a parser for the Prometheus Query Language\n\n_Have an example we can add to this list? [Let us know](https://github.com/datalust/superpower/issues/new)._\n\n### Getting help\n\nPlease post issues [to the issue tracker](https://github.com/datalust/superpower/issues), or tag your [question on StackOverflow](http://stackoverflow.com/questions/tagged/superpower) with `superpower`.\n\n_The repository's title arose out of a talk_ \"Parsing Text: the Programming Superpower You Need at Your Fingertips\" _given at [DDD Brisbane](http://dddbrisbane.com/) 2015._\n","funding_links":[],"categories":["Parser Library","解析器库","C# #","Identifiers"],"sub_categories":["GUI - other"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatalust%2Fsuperpower","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatalust%2Fsuperpower","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatalust%2Fsuperpower/lists"}