{"id":20482449,"url":"https://github.com/agoose77/derpy","last_synced_at":"2026-06-08T10:01:14.983Z","repository":{"id":76066883,"uuid":"71269948","full_name":"agoose77/derpy","owner":"agoose77","description":"Context Free Grammar \"Parsing with Derivatives\", in Python","archived":false,"fork":false,"pushed_at":"2019-02-06T11:23:58.000Z","size":265,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-05-13T11:39:54.958Z","etag":null,"topics":["ast","derivative","grammar","parsing","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/agoose77.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-10-18T16:46:01.000Z","updated_at":"2026-03-13T16:42:23.000Z","dependencies_parsed_at":null,"dependency_job_id":"6e705858-fc4c-45e9-9c86-27d84cf4e52f","html_url":"https://github.com/agoose77/derpy","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/agoose77/derpy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agoose77%2Fderpy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agoose77%2Fderpy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agoose77%2Fderpy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agoose77%2Fderpy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/agoose77","download_url":"https://codeload.github.com/agoose77/derpy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/agoose77%2Fderpy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34057158,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-08T02:00:07.615Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast","derivative","grammar","parsing","python"],"created_at":"2024-11-15T16:12:53.771Z","updated_at":"2026-06-08T10:01:14.957Z","avatar_url":"https://github.com/agoose77.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Parsing with derivatives\nA Python implementation of parsing with derivatives. Provides a concise infix notation to support legible, complex grammars.\n\nSee http://maniagnosis.crsr.net/2012/04/parsing-with-derivatives-introduction.html for a Java implementation, or http://matt.might.net/articles/parsing-with-derivatives/ for the original author's publication.\n\n## What is parsing?\nAccording to Wikipedia, parsing is \n\u003e the formal analysis by a computer of a sentence or other string of words into its constituents, resulting in a parse tree showing their syntactic relation to each other, which may also contain semantic and other information.\n\nTypically, the process of producing a parse tree is separated into two distinct phases; lexing, and parsing.\nLexing simply breaks down a continuous stream of characters into course, discrete 'tokens'. Parsing then consumes these tokens to build the parse tree. \n\nThis library provides both a simple tokeniser, and the parsing constructs required to build a parse tree from any given input and context free grammar.\n\n## What can this library be used for?\nThere are several scenarios in which one might write a lexer/parser. Such scenarios include transpiling between languages, modifying source code (such as decorating all functions starting with the name \"funky\"), writing a DSL ...\n\n## Installation\nFor now, there is no package on PyPI, so you must use the Git link:\n\n`pip install git+https://github.com/agoose77/derpy.git#egg=derpy`\n\nAdd `[python]` to the line to include the codegen support (astor).\n\n## Example parser\n![Parser syntax](https://latex.codecogs.com/png.latex?\\dpi{150}\u0026space;\\large\u0026space;s\u0026space;=\u0026space;\\epsilon\u0026space;|\u0026space;1\u0026space;\\cdot\u0026space;s)\n\nThis parser would be represented as \n```python\nfrom derpy import rec, lit, empty_string\ns = rec()\ns.parser = empty_string | (lit('1') \u0026 s)\n```\n\nOr in short-hand\n```python\ns = rec()\ns.parser = ~(lit('1') \u0026 s)\n```\n\nThis parser could parse any number of ones, before terminating.\n```python\nfrom derpy import Token, parse\n\nparse(s, [Token('1', 1) for i in range(5)])\n\u003e\u003e {(1, (1, (1, (1, (1, '')))))}\n```\n\n\n## Python Grammar Parsing\nA Python parser example can be found in the `derpy.grammars.python` module.\nIt may not entirely be correct; small errors in the grammar may exist due to a hasty translation from the Python 3 official grammar. Alternatively, the EBNF parser can be used to produce a Python grammar parser.\n\nMost of the lines of code are devoted to outputting a useful AST (but for around 1200 loc, it's still quite compact). A custom `derpy.ast` module is defined to allow a similar API to the built-in ast module (In fact, the ast output was tested with an existing ast-to-code utility, simply by replacing the import to ast with our own).\n\n## [Py]EBNF Grammar Meta Parsing\nAn example of parsing the Python EBNF grammar, to produce the source for a Python parser, can be found in the `derpy.grammars.ebnf`\nTo make this usable as an AST generator requires some formatting of each rule (using a reduction), which can be done by using custom reduction rules on the output of the generator. The produced Python grammar can be compared against the hand-rolled one in `python36`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagoose77%2Fderpy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagoose77%2Fderpy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagoose77%2Fderpy/lists"}