{"id":13822884,"url":"https://github.com/PistonDevelopers/meta","last_synced_at":"2025-05-16T17:32:32.459Z","repository":{"id":31358280,"uuid":"34921144","full_name":"PistonDevelopers/meta","owner":"PistonDevelopers","description":"A DSL parsing library for human readable text documents","archived":false,"fork":false,"pushed_at":"2021-04-12T10:57:19.000Z","size":382,"stargazers_count":88,"open_issues_count":11,"forks_count":5,"subscribers_count":45,"default_branch":"master","last_synced_at":"2024-04-23T23:15:03.896Z","etag":null,"topics":["meta-language","parsing","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PistonDevelopers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":"bvssvni"}},"created_at":"2015-05-01T19:53:08.000Z","updated_at":"2024-03-24T14:59:57.000Z","dependencies_parsed_at":"2022-08-25T10:11:42.123Z","dependency_job_id":null,"html_url":"https://github.com/PistonDevelopers/meta","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PistonDevelopers%2Fmeta","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PistonDevelopers%2Fmeta/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PistonDevelopers%2Fmeta/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PistonDevelopers%2Fmeta/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PistonDevelopers","download_url":"https://codeload.github.com/PistonDevelopers/meta/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":213893313,"owners_count":15653524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["meta-language","parsing","rust"],"created_at":"2024-08-04T08:02:22.415Z","updated_at":"2024-08-04T08:09:11.766Z","avatar_url":"https://github.com/PistonDevelopers.png","language":"Rust","funding_links":["https://github.com/sponsors/bvssvni"],"categories":["Rust"],"sub_categories":[],"readme":"# Piston-Meta\nA DSL parsing library for human readable text documents\n\n### Introduction\n\nPiston-Meta makes it easy to write parsers for human readable text documents.\nIt can be used for language design, custom formats and data driven development.\n\nMeta parsing is a development technique that goes back to the first modern computer.\nThe idea is to turn pieces of a computer program into a programmable pipeline,\nthereby accelerating development.\nAn important, but surprisingly reusable part across projects, is the concept of generating\nstructured data from text, since text is easy to modify and reason about.\n\nMost programs that work with text use the following pipeline:\n\n```ignore\nf : text -\u003e data\n```\n\nThe problem with this approach is that `f` changes from project to project,\nand the task of transforming text into a data structure can get very complex.\nFor example, to create a parser for the syntax of a programming language,\none might need several thousands lines of code.\nThis slows down development and increases the chance of making errors.\n\nMeta parsing is a technique where `f` gets split into two steps:\n\n```ignore\nf \u003c=\u003e f2 . f1\nf1 : text -\u003e meta data\nf2 : meta data -\u003e data\n```\n\nThe first step `f1` takes text and converts it into meta data.\nA DSL (Domain Specific Language) is used to describe how this transformation happens.\nThe second step `f2` converts meta data into data, and this is often written as code.\n\n### Rules\n\nThe meta language is used to describe how to read other documents.\nFirst you define some strings to reuse, then some node rules.\nThe last node is used to read the entire document.\n\n`20 document = [.l(string:\"string\") .l(node:\"node\") .w?]`\n\nStrings start with an underscore and can be reused among the rules:\n\n`_opt: \"optional\"`\n\nNodes start with a number that gets multiplied with 1000 and used as debug id.\nIf you get an error `#4003`, then it was caused by a rule in the node starting with 4.\n\n|Rule|Description|\n|----|-----------|\n|.l(rule)|Separates sub rule with lines.|\n|.l+(rule)|Separates sub rule with lines, with indention (whitespace sensitive)|\n|.r?(rule)|Repeats sub rule until it fails, allows zero repetitions.|\n|.r!(rule)|Repeats sub rule until it fails, requires at least one repetition.|\n|...any_characters?:name|Reads a string until any characters, allows zero characters. Name is optional.|\n|...any_characters!:name|Reads a string until any characters, requires at least one character. Name is optional.|\n|..any_characters?:name|Reads a string until any characters or whitespace, allows zero characters. Name is optional.|\n|..any_characters!:name|Reads a string until any characters or whitespace, requires at least one character. Name is optional.|\n|.w?|Reads whitespace. The whitespace is optional.|\n|.w!|Reads whitespace. The whitespace is required.|\n|?rule|Makes the rule optional.|\n|\"token\":name|Expects a token, sets name to `true`. Name is optional.|\n|\"token\":!name|Expects a token, sets name to `false`. Name is required.|\n|!\"token\":name|Fails if token is read, sets name to `true` if it is not read. Name is optional.|\n|!\"token\":!name|Fails if token is read, sets name to `false` if it is not read. Name is required.|\n|!rule|Fails if rule is read.|\n|.s?(by_rule rule)|Separates rule by another rule, allows zero repetitions.|\n|.s!(by_rule rule)|Separates rule by another rule, requires at least one repetition.|\n|.s?.(by_rule rule)|Separates rule by another rule, allows trailing.|\n|{rules}|Selects a rule. Tries the first rule, then the second, etc. Rules are separated by whitespace.|\n|[rules]|A sequence of rules. Rules are separated by whitespace.|\n|node|Uses a node without a name. The read data is put in the current node.|\n|node:name|Uses a node with a name. The read data is put in a new node with the name.|\n|.t?:name|Reads a JSON string with a name. The string can be empty. Name is optional.|\n|.t!:name|Reads a JSON string with a name. The string can not be empty. Name is optional.|\n|.$:name|Reads a number with a name. The name is optional.|\n|.$_:name|Reads a number with underscore as visible separator, for example `10_000`. The name is optional.|\n\n### \"Hello world\" in Piston-Meta\n\n```rust\nextern crate piston_meta;\n\nuse piston_meta::*;\n\nfn main() {\n    let text = r#\"hi James!\"#;\n    let rules = r#\"\n        1 say_hi = [\"hi\" .w? {\"James\":\"james\" \"Peter\":\"peter\"} \"!\"]\n        2 document = say_hi\n    \"#;\n    // Parse rules with meta language and convert to rules for parsing text.\n    let rules = match syntax_errstr(rules) {\n        Err(err) =\u003e {\n            println!(\"{}\", err);\n            return;\n        }\n        Ok(rules) =\u003e rules\n    };\n    let mut data = vec![];\n    match parse_errstr(\u0026rules, text, \u0026mut data) {\n        Err(err) =\u003e {\n            println!(\"{}\", err);\n            return;\n        }\n        Ok(()) =\u003e {}\n    };\n    json::print(\u0026data);\n}\n```\n\n### Bootstrapping\n\nWhen the meta language changes, bootstrapping is used to hoist the old meta syntax into the new meta syntax. Here is how it works:\n\n1. Piston-Meta contains composable rules that can parse many human readable text formats.\n2. Piston-Meta knows how to parse and convert to its own rules, known as \"bootstrapping\".\n3. Therefore, you can tell Piston-Meta how to parse other text formats using a meta language!\n4. Including the text format describing how to parse its own syntax, which generates equivalent rules to the ones hard coded in Rust.\n5. New versions of the meta language can describe older versions to keep backwards compatibility, by changing the self syntax slightly, so it can read an older version of itself.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPistonDevelopers%2Fmeta","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPistonDevelopers%2Fmeta","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPistonDevelopers%2Fmeta/lists"}