{"id":22173473,"url":"https://github.com/jlumbroso/combstruct2json","last_synced_at":"2025-07-26T15:31:40.739Z","repository":{"id":62563981,"uuid":"125648815","full_name":"jlumbroso/combstruct2json","owner":"jlumbroso","description":"Lightweight library to parse combstruct grammars, and standalone tool to convert them to JSON.","archived":false,"fork":false,"pushed_at":"2018-06-19T04:14:19.000Z","size":1013,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-12T20:49:47.826Z","etag":null,"topics":["analytic-combinatorics","combinatorics","parser"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jlumbroso.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-03-17T16:33:57.000Z","updated_at":"2022-11-27T18:02:04.000Z","dependencies_parsed_at":"2022-11-03T16:00:32.113Z","dependency_job_id":null,"html_url":"https://github.com/jlumbroso/combstruct2json","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlumbroso%2Fcombstruct2json","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlumbroso%2Fcombstruct2json/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlumbroso%2Fcombstruct2json/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jlumbroso%2Fcombstruct2json/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jlumbroso","download_url":"https://codeload.github.com/jlumbroso/combstruct2json/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227690966,"owners_count":17805013,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytic-combinatorics","combinatorics","parser"],"created_at":"2024-12-02T07:33:33.359Z","updated_at":"2024-12-02T07:33:36.217Z","avatar_url":"https://github.com/jlumbroso.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# combstruct2json\n\nThis project provides:\n1. A highly optimized, extensible, lightweight linkable library to parse\n[`combstruct` grammars](https://www.maplesoft.com/support/help/maple/view.aspx?path=combstruct),\nimplemented in C/C++.\n2. An independently available Python wrapper that, given a grammar file, outputs a `dict` according to the JSON specification below.\n3. A standalone commandline utility that can be piped to other tools (or called as an external system command from a host language such a Ruby, or JavaScript).\n\nThe folder `examples` contains usage examples for the linkable library, the folder `tests` contains sample grammars that can be parsed.\n\nThis library is already used by projects, such as [boltzoc](https://github.com/jlumbroso/boltzoc), the fast Analytic Sampler Oracle in C.\n\n## Example\n\n```bash\n$ cat tests/cographs\nG  = Set(Co),\nCo = Union(Ge, Gc, v, Prod(v,v)),\nGe = Union(Set(Sc, card=2), Prod(Sc,v)),\nGc = Set(Union(v, Sc), card\u003e=3),\nSc = Set(Union(v, C), card\u003e=2),\nC  = Set(Union(v, Sc), card\u003e=2),\nv  = Atom\n```\n\nwould then produce the following JSON output:\n\n```bash\n$ make all\n$ ./combstruct2json tests/cographs\n{ \"G\": { \"type\": \"op\", \"op\": \"Set\", \"param\": [{ \"type\": \"id\", \"id\": \"Co\" }] }, \"Co\": { \"type\": \"op\", \"op\": \"Union\", \"param\": [ { \"type\": \"id\", \"id\": \"Ge\" }, { \"type\": \"id\", \"id\":\"Gc\" }, { \"type\": \"id\", \"id\": \"v\" }, { \"type\": \"op\", \"op\": \"Prod\", \"param\": [ { \"type\": \"id\", \"id\": \"v\" }, { \"type\": \"id\", \"id\": \"v\" } ] } ] }, \"Ge\": { \"type\": \"op\", \"op\": \"Union\", \"param\": [ { \"type\": \"op\", \"op\": \"Set\", \"param\": [{ \"type\": \"id\", \"id\": \"Sc\" }], \"restriction\": \"card = 2\" }, { \"type\": \"op\", \"op\": \"Prod\", \"param\": [ { \"type\": \"id\", \"id\": \"Sc\" }, { \"type\": \"id\", \"id\": \"v\" } ] } ] }, \"Gc\": { \"type\": \"op\", \"op\": \"Set\", \"param\": [{ \"type\": \"op\", \"op\": \"Union\", \"param\": [ { \"type\": \"id\", \"id\": \"v\" }, { \"type\": \"id\", \"id\":\"Sc\" } ] }], \"restriction\": \"card \u003e= 3\" }, \"Sc\": { \"type\": \"op\", \"op\": \"Set\", \"param\": [{ \"type\": \"op\", \"op\": \"Union\", \"param\": [ { \"type\": \"id\", \"id\": \"v\" }, { \"type\": \"id\", \"id\": \"C\" } ] }], \"restriction\": \"card \u003e= 2\" }, \"C\": { \"type\": \"op\", \"op\": \"Set\", \"param\": [{ \"type\": \"op\", \"op\": \"Union\", \"param\": [ { \"type\": \"id\", \"id\": \"v\" }, { \"type\": \"id\", \"id\": \"Sc\" } ] }], \"restriction\": \"card \u003e= 2\" }, \"v\": { \"type\": \"unit\", \"unit\": \"Atom\" }}\n```\n\nwhich can be prettified, for instance, using Python, for better legibility:\n\n```bash\n$ ./combstruct2json tests/cographs | python -m json.tool | head\n{\n    \"C\": {\n        \"op\": \"Set\",\n        \"param\": [\n            {\n                \"op\": \"Union\",\n                \"param\": [\n                    {\n                        \"id\": \"v\",\n                        \"type\": \"id\"\n                    ...\n```\n\nIf you build and install the Python wrapper, you may also read a grammar directly\nfrom a Python program:\n\n```python\nimport combstruct2json\nd = combstruct2json.read_file(\"tests/cographs\")\nprint(\"Top-level symbols:\")\nprint(d.keys())\n```\n\nwhich would print out:\n\n```text\nTop-level symbols:\n[u'C', u'Co', u'G', u'Ge', u'Gc', u'v', u'Sc']\n```\n\n## Installation\n\nYou can build the project from scratch, if you have the necessary dependencies:\n\n1. You may need to install `flex` and `bison`, if you don't already have them.\n\n2. Run `make all` to create the executable `combstruct2json`, the static C/C++\n   library, and the Python wrapper library.\n\n3. Run `./combstruct2json \u003cfilename\u003e` to print the parsed JSON output, from the\n   grammar contained in the given file.\n\nAlternatively, you may also install the Python library directly from PyPI\n(possibly in your user directory). This step will fetch the latest source after\nit has been processed by lexer/parser and so does not require they be installed:\n\n```bash\n$ pip install combstruct2json\n```\n\n## Draft specification of JSON output\n\nBecause one purpose to enable easier interoperability with existing work using\nsymbolic specifications in Maple, this library uses\n[Maple's specification](https://www.maplesoft.com/support/help/maple/view.aspx?path=combstruct)\nfor `combstruct` grammars as a starting point.\n\nThe output is a JSON string which represents a dictionary mapping symbol names\nto an abstract syntax tree. Each node of the grammar is represented by a node\nin the JSON tree:\n\n- For unit elements (with or without a weight), the `type` is `unit`; the\n  available field is `unit` to describe the type of element (an atom, or epsilon). Example:\n\n  ```\n  { \"type\": \"unit\", \"unit\": \"Epsilon\" }\n  ```\n\n- For variable references, the `type` is `id`; the available field is `id` which\n  should specify the name of the variable that is being referenced.\n  Example:\n\n  ```\n  { \"type\": \"id\", \"id\": \"v\" }\n  ```\n\n- For operators, the `type` is `op`; the available fields are `op` which describes\n  which operator is applied (from `Union`, `Prod`, `Sequence`, `Set`, etc.),\n  `param` which would be a list of parameters on which the operator is applied, and `restriction` which optionally encodes a restriction (for the moment, this is\n  limited to cardinality restrictions). Example:\n\n  ```\n  {\n    \"type\": \"op\",\n    \"op\": \"Union\",\n    \"params\": [\n        { \"type\": \"id\", \"id\": \"v\" },\n        { \"type\": \"id\", \"id\": \"Sc\" }\n        ],\n    \"restriction\": \"card \u003e= 2\"\n  }\n  ```\n\nThis draft specification is designed to produce an easy to parse JSON grammar format\nspecification to improve communication between various tools of a planned analytic\ncombinatorics toolchain.\n\n## Benchmark\n\nAlthough we do not currently have benchmarks, this parser library and tool were\ndesigned using the most low-level tools, so as to be able to parse the possibly\nvery large grammars that may be the output of algorithms. Such an example,\n[reluctant random walks](https://github.com/jlumbroso/reluctant-walks), provides\none very large example `reluctantQPW2` that has more than 1000 equations (these\nare lines that are not comments) and weighs 4.6 MB:\n\n```bash\n$ awk 'flag { print } /^$/ { flag = 1 }' tests/reluctantQPW2 | wc -l\n    1066\n\n$ time ./combstruct2json tests/reluctantQPW2  \u003e /dev/null\nreal\t0m3.577s\nuser\t0m3.459s\nsys\t0m0.099s\n```\n\nAnother grammar for the same walk model, generated by the same process but with\nmore accuracy, had 5000+ equations and a specification weighing 115.95 MB (it\ncould not be included in this repository because it exceeds GitHub's limits):\n\n```bash\n$ awk 'flag { print } /^$/ { flag = 1 }' tests/reluctantQPW2_ | wc -l\n    5093\n\n$ time ./combstruct2json tests/reluctantQPW2_  \u003e /dev/null\nreal\t6m7.458s\nuser\t5m53.586s\nsys\t0m9.186s\n```\n\n## Acknowledgements\n\nThanks to Alexandre de Faveri.\n\n## Bibliography\n\nPhilippe Flajolet, Paul Zimmermann and  Bernard van Cutsem. [*A calculus for the random generation of combinatorial structures.*](http://algo.inria.fr/flajolet/Publications/RR-1830.pdf) 29 pages. Theoretical Computer Science, vol. 132 (1-2), pp. 1-35, 1994.\n\nPaul Zimmermann. [*Gaia: a package for the random generation of combinatorial structures.*](http://plouffe.fr/simon/math/fpsac1993b.pdf#page=153) Maple Technical Newsletter, vol. 1 (1), pp. 143-147, 1994.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlumbroso%2Fcombstruct2json","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjlumbroso%2Fcombstruct2json","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjlumbroso%2Fcombstruct2json/lists"}