{"id":13399008,"url":"https://github.com/erezsh/plyplus","last_synced_at":"2025-07-20T22:32:47.134Z","repository":{"id":57453717,"uuid":"1618887","full_name":"erezsh/plyplus","owner":"erezsh","description":"a friendly yet powerful LR-parser written in Python","archived":false,"fork":false,"pushed_at":"2018-03-19T22:43:14.000Z","size":543,"stargazers_count":267,"open_issues_count":2,"forks_count":56,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-07-08T15:32:36.470Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/erezsh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2011-04-15T12:47:01.000Z","updated_at":"2024-08-14T09:39:58.000Z","dependencies_parsed_at":"2022-08-29T11:12:01.318Z","dependency_job_id":null,"html_url":"https://github.com/erezsh/plyplus","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/erezsh/plyplus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erezsh%2Fplyplus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erezsh%2Fplyplus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erezsh%2Fplyplus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erezsh%2Fplyplus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/erezsh","download_url":"https://codeload.github.com/erezsh/plyplus/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/erezsh%2Fplyplus/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265610643,"owners_count":23797700,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T19:00:33.424Z","updated_at":"2025-07-20T22:32:47.112Z","avatar_url":"https://github.com/erezsh.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"Note: PlyPlus is now under maintenance only.\n\nPlease check out my new parsing library: https://github.com/erezsh/lark\n\nLark is faster, more powerful, more stable, and has a lot more features.\n\n# PlyPlus - a friendly yet powerful parser library, written in Python.\n\nPlyplus is a general-purpose parser built on top of [PLY](http://www.dabeaz.com/ply/) (LALR(1)), and written in Python. Plyplus features a modern design, and focuses on simplicity without losing power.\n\n## Main Concepts\n\n1. *Separation of code from grammar*: Grammar files are more readable and portable, and it makes the code cleaner too.\n\n2. *Always build an AST (tree)*: Every application, not matter how small, can benefit from the power and simplicity of working with a tree, instead of a state-machine.\n\n3. *Follow Python's Idioms*: Beauty, simplicity and readability are more important than speed. But Plyplus is fast enough!\n\n\n## Features\n\n - EBNF grammar (supported: parentheses, '|', '\\*', '?', and '+', inline tokens, token fragements, and more)\n - LR-parser\n - Builds an AST automagically based on the grammar\n - Selectors: run powerful css-like queries on the AST\n - Nested grammars (a grammar within a grammar. Useful for HTML/CSS, for example)\n - Unicode support\n - Python 2.7, Python 3.3 and PyPy 1.9 compatible\n - Fully-working Python 2.x grammar included\n - New: An experimental Earley-parser engine!\n\n## Q \u0026 A\n\nQ. How capable is Plyplus?\n\nA. Plyplus is capable of parsing any LR-compatible grammar. It supports post-tokenizing code, so it's capable of parsing python (it comes with a ready-to-use python parser). Other features, such as sub-grammars, provide more flexibility to handle the trickier grammars.\n\nQ. How fast is it?\n\nA. Plyplus does not put speed as its first priority. However, right now it manages to parse the entire Python26/Libs directory (200 files, 4mb of text, including post-processing) in about 42 seconds on my humble dual-core 2ghz 2gb-ram machine (and 30 seconds with PyPy).\n\nQ. So what is Plyplus' first priority?\n\nA. Power and simplicity. See the examples and judge for yourself.\n\nQ. I want to use Plyplus in a threaded application. Is it thread safe?\n\nA. Yes, but you must pay attention. Plyplus relies on PLY, it can cause problems if you try to define multiple parsers at the same time using threads. Please make sure not to do that.\n\n\n## Tutorials\n\nLearn how to write a grammar for Plyplus at the [tutorial](/docs/tutorial.md)\n\nLearn how to query the AST using [selectors](/docs/selectors.md)\n\n## Examples\n\nPlyPlus offers benefits both for writing grammars, and for working with their resulting AST. The examples address both of these.\n\n### Calc\n\nA calculator is a bit like the \"hello world\" of parsers. Here is how calc.g might look:\n\n    start: add;\n\n    // Rules\n    ?add: (add add_symbol)? mul;\n    ?mul: (mul mul_symbol)? atom;\n    @atom: neg | number | '\\(' add '\\)';\n    neg: '-' atom;\n\n    // Tokens\n    number: '[\\d.]+';\n    mul_symbol: '\\*' | '/';\n    add_symbol: '\\+' | '-';\n\n    WS: '[ \\t]+' (%ignore);\n\nThis is all we need to get an AST. Now we can write:\n\n    \u003e\u003e\u003e import plyplus\n    \u003e\u003e\u003e plyplus.Grammar(open(\"calc.g\")).parse(\"(1 + 2) * -3\")\n    start(mul(add(number(u'1'), add_symbol(u'+'), number(u'2')), mul_symbol(u'*'), neg(number(u'3'))))\n\nNotice that \"atom\" doesn't appear in the AST. That is because we muted it with the \"@\" prefix. Same was done with \"add\" and \"mul\" using the \"?\" prefix, which only mutes if the parser matches just one item.\n\nTo learn how to evaluate this AST into a solution, check out the simple [Calculator Example](/examples/calc.py).\n\nFor a more thorough explanation of grammars, check out [the tutorial](/docs/tutorial.md). If something is still not clear, feel free to email me and ask!\n\n### Working with the Python AST (using the builtin python grammar)\n\nWe'll use Plyplus' grammar for Python, and play with os.py for a bit (though it could be any Python file).\n\nFor starters, let's do something simple: Let's list all of the functions (or methods) in the os module. We'll query the AST using [selectors](/docs/selectors.md), so click the link if you want to be able to follow (or maybe an understanding of CSS/JQuery is enough?).\n\n    \u003e\u003e\u003e import plyplus, plyplus.grammars\n    \u003e\u003e\u003e g = plyplus.Grammar(plyplus.grammars.open('python.g'))   # load python grammar\n    \u003e\u003e\u003e t = g.parse(file(r'c:\\python27\\lib\\os.py').read())                  # read os.py\n    \u003e\u003e\u003e t.select('funcdef \u003e name \u003e *:is-leaf')\n    ['_get_exports_list', 'makedirs', 'removedirs', 'renames', 'walk', 'execl', 'execle', 'execlp', 'execlpe', ...\n\n(Run it yourself for the full input)\n\nNow let's count how many times os.py calls isinstance:\n\n    \u003e\u003e\u003e len(t.select('/isinstance/'))\n    3\n\nInteresting! But where in the file are they called? We can use the \"line\" attribute to find out (there's also a column attribute!):\n\n    \u003e\u003e\u003e [x.line for x in t.select('/isinstance/')]\n    [669, 689, 709]\n\nLet's look at one of those calls. We'll need to select more context for that.\n\n    \u003e\u003e\u003e t.select('=funccall \u003e name \u003e /isinstance/')[0]\n    funccall(name('isinstance'), arglist(arg(name('cmd')), arg(name('basestring'))))\n\nMore context?\n\n    \u003e\u003e\u003e _.parent().parent().parent()\n    funccall(attrget(name('subprocess'), name('Popen')), arglist(arg(name('cmd')), arg(name('shell'), funccall(...\n\nHard to read? Try looking at it visually! (requires pydot)\n\n    \u003e\u003e\u003e _.to_png_with_pydot(r'calling_popen.png')\n\n![calling\\_popen.png](/docs/calling_popen.png)\n\n### Working with INI-files (using the builtin config grammar)\n\nINI files are too open-handed to be a good candidate for LR-parsing, but PlyPlus can handle them using nested grammars. By parsing different elements separately, a \"]\" symbol can be both a special token and just part of the text, all in the same file.\n\nLet's parse an INI file that comes with NumPy.\n\n    \u003e\u003e\u003e g = plyplus.Grammar(plyplus.grammars.open('config.g'), auto_filter_tokens=False)   # load config grammar\n    \u003e\u003e\u003e t = g.parse(file(r\"C:\\Python26\\Lib\\site-packages\\numpy\\core\\lib\\npy-pkg-config\\npymath.ini\").read())\n\nList the sections:\n\n    \u003e\u003e\u003e t.select('section \u003e start \u003e name *')\n    ['meta', 'variables', 'default', 'msvc']\n\nLet's look at the meta section\n\n    \u003e\u003e\u003e t.select('=section /meta/')\n    [section(start(name('meta')), option(start(name('Name'), start(value('npymath')))), ...\n\n(The start heads denote a sub-grammar)\n\nLet's pretty-print it! We can use a transformer to do it. A transformer is a tree-visitor that returns a new value for each head (branch) it visits.\n\n    \u003e\u003e\u003e class PrettyINI(plyplus.STransformer):\n        def option(self, tree):\n            name = tree.select1('name *')   # select1 asserts only one result\n            value = tree.select1('value *')\n            return '%s = %s' % (name, value)\n        def section(self, tree):\n            name = tree.select1('name *')\n            return '[%s]\\n\\t%s' % (name, '\\n\\t'.join(tree.tail[1:]))\n\nNow that each rule has code to handle it, let's run it!\n\n    \u003e\u003e\u003e meta = t.select1('=section /meta/')\n    \u003e\u003e\u003e print PrettyINI().transform( meta )\n    [meta]\n            Name = npymath\n            Description = Portable, core math library implementing C99 standard\n            Version = 0.1\n\nIt works! Now that it's done, we can use it to output the rest of the file as well:\n\n    \u003e\u003e\u003e print '\\n'.join( PrettyINI().transform(t).tail )\n    ... (left as an excercise to the reader ;)\n\n\n## License\n\nPlyplus uses the [MIT license](https://github.com/jquery/jquery/blob/master/MIT-LICENSE.txt).\n\n## Afterword\n\nI hope this readme inspired you to play with Plyplus a bit, and maybe even use it for your project.\n\nFor more examples, check out the [test module](/plyplus/test/test_parser.py)\n\nIf you have any questions or ideas, please email me at erezshin+plyplus at gmail com\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferezsh%2Fplyplus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ferezsh%2Fplyplus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ferezsh%2Fplyplus/lists"}