{"id":37075819,"url":"https://github.com/nuchi/sublime-from-cfg","last_synced_at":"2026-01-14T08:55:19.908Z","repository":{"id":45589772,"uuid":"423669656","full_name":"nuchi/sublime-from-cfg","owner":"nuchi","description":"Generate a sublime-syntax file from a non-left-recursive, follow-determined, context-free grammar","archived":false,"fork":false,"pushed_at":"2021-12-06T23:03:09.000Z","size":211,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-04T13:46:06.709Z","etag":null,"topics":["bnf","context-free-grammar","ebnf","grammar","parser","parser-generator","ply","sly","sublime-syntax","sublime-text"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nuchi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-02T01:29:15.000Z","updated_at":"2024-12-05T16:39:06.000Z","dependencies_parsed_at":"2022-07-26T11:47:06.705Z","dependency_job_id":null,"html_url":"https://github.com/nuchi/sublime-from-cfg","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nuchi/sublime-from-cfg","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuchi%2Fsublime-from-cfg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuchi%2Fsublime-from-cfg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuchi%2Fsublime-from-cfg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuchi%2Fsublime-from-cfg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nuchi","download_url":"https://codeload.github.com/nuchi/sublime-from-cfg/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nuchi%2Fsublime-from-cfg/sbom","scorecard":{"id":698013,"data":{"date":"2025-08-11","repo":{"name":"github.com/nuchi/sublime-from-cfg","commit":"bca74c145353952bb6569bf9bed125f2717179f3"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.4,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/28 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/test.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:31: update your workflow using https://app.stepsecurity.io/secureworkflow/nuchi/sublime-from-cfg/test.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/test.yml:35: update your workflow using https://app.stepsecurity.io/secureworkflow/nuchi/sublime-from-cfg/test.yml/master?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/test.yml:38","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 4 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-22T04:17:07.730Z","repository_id":45589772,"created_at":"2025-08-22T04:17:07.730Z","updated_at":"2025-08-22T04:17:07.730Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28414714,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bnf","context-free-grammar","ebnf","grammar","parser","parser-generator","ply","sly","sublime-syntax","sublime-text"],"created_at":"2026-01-14T08:55:19.286Z","updated_at":"2026-01-14T08:55:19.890Z","avatar_url":"https://github.com/nuchi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"**Install this package at [Pypi](https://pypi.org/project/sublime-from-cfg/)**\n\n# Context-free grammar to Sublime-syntax file\n\nThis project produces sublime-syntax highlighting files from a description of a context-free grammar.\n\n**Note: I used this tool to generate [a syntax definition file for the Faust programming language.](https://github.com/nuchi/faust-sublime-syntax)**\n\nIt implements a \"Generalised Recursive Descent Parser\" as described in [_Generalised recursive descent parsing and follow-determinism_](https://link.springer.com/content/pdf/10.1007%2FBFb0026420.pdf) by Adrian Johnstone and Elizabeth Scott. It's essentially a non-deterministic [LL(1)](https://en.wikipedia.org/wiki/LL_parser) parser. If the grammar happens to be LL(1) then no backtracking will happen and it's just an LL(1) parser. If the grammar is not LL(1), then alternatives will be tried in sequence, backtracking until one succeeds.\n\nIMPORTANT: The grammar must be non-left-recursive, and also must be follow-determined. If the grammar is left-recursive then the program will complain and alert the user, but I don't know an algorithm to detect whether the grammar is follow-determined. **IF THE GRAMMAR IS _NOT_ FOLLOW-DETERMINED, THEN THE LANGUAGE RECOGNIZED BY THE GENERATED SYNTAX WILL SIMPLY NOT MATCH THE INPUT GRAMMAR.**\n\nA grammar is _follow-determined_ if whenever a nonterminal X produces both `\u003cstring\u003e` and `\u003cstring\u003e y \u003c...\u003e`, then y is not in the follow set of X. Intuitively, a grammar is follow-determined whenever a single lookahead token is enough to tell us whether it's okay to pop out of the context for X or if we should keep going within X. (i.e. if we've just consumed the prefix `\u003cstring\u003e` and need to decide whether to finish with X or to keep consuming, then the presence or absence of the next token in the follow set of X had better be enough to tell us which option to take, because once we pop out of X then we can't backtrack to try other options anymore.)\n\n## Implementation\n\nSublime syntax files allow one to define _contexts_; within each context one can match against any number of regular expressions (including lookaheads) and then perform actions like pushing other contexts onto the context stack, pop out of the context, set the scope of the consumed tokens (i.e. instruct Sublime Text that a token is e.g. a function definition and highlight it appropriately), and others. One can also set a branch point and try multiple branches in sequence; if an action taken is to `fail` that branch point, then the syntax engine backtracks and tries the next branch in the sequence.\n\nSee [the Wikipedia page on LL parsers](https://en.wikipedia.org/wiki/LL_parser) for more details on how LL parsers work in general. What I do here is always indicate \"success\" by `pop: 2`; i.e. popping twice out of the current context, and failure by `pop: 1`. Contexts for a given production are pushed onto the stack interleaved by a `pop2!` context which always pops 2 contexts off the stack. Therefore a failure, which pops once, moves into the \"always pop 2\" stream until it hits a failure context (to backtrack and try a different branch) or pops all the way out of the current stack.\n\n## Related\n\nThis project shares the goal of automatically generating a Sublime-syntax file with [Benjamin Schaaf's sbnf project](https://github.com/BenjaminSchaaf/sbnf/). While I started working on this idea before learning about the existence of sbnf, I took a lot of inspiration from that project. In particular the idea of using the extended BNF syntax (allowing `*`, `?`, parenthesized expressions) and passive expressions. More generally, I'm using the exact same `.sbnf` file format as my input. The implementations here are all my own.\n\n### Differences between this project and sbnf\n\n[One of sbnf's goals](https://crates.io/crates/sbnf) is to \"Compile to an efficient syntax, comparable to hand-made ones\". That is not explicitly a goal of this project. There may be some small differences in implementation; I list some examples below.\n\n#### `main` is not implicitly repeated\n\nIn sbnf, the rule `main : 'a' ;` will match any number of repeated `a` characters. In sublime-from-cfg, only one `a` will be matched. The parser will mark an invalid parse, and then reset at the beginning of the next new line (so that the whole rest of the document is not marked invalid).\n\n#### `\u003c\u003e` represents an empty production\n\nWhile sublime-from-cfg automatically rewrites rules involving `?` (optional) and `*` (repetition), you can take extra control of the rule-rewriting by explicitly indicating an empty production via `\u003c\u003e`. For example, the following rewrite is done automatically:\n```diff\n- a  : b c* d ;\n+ a  : b a' ;\n+ a' : d\n+    | c d ;\n```\nBut if you prefer a different rewrite, you can explicitly write something like:\n```\na     : b c-rep d ;\nc-rep : \u003c\u003e\n      | c c-rep ;\n```\n\n#### Setting sort precedence\n\nSometimes a regular expression can match more than one part of a language, where for example a generic expression to match any identifier, like `[a-zA-Z][_a-zA-Z0-9]*`, will also match a reserved word like `import`. To make sure that the reserved word is always tried first, you can add a \"sort\" option:\n```\nIDENTIFIER = '[a-zA-Z][_a-zA-Z0-9]*'\nstatement : IDENTIFIER{entity.name, sort: 1} `=` '\\d+' `;`\n          | 'import'{keyword.operator} IDENTIFIER `;`\n          ;\n```\nAt the moment, the same regular expression can only have one sort value across the whole file (defining it twice will pick one arbitrarily). The default value is 0, and smaller values are tried before larger values. In the example above, `'import'` has lower precedence (the default value 0) than `IDENTIFIER` (value 1), so the syntax engine will try to match `'import'` first.\n\n#### String vs Rule parameters\n\nString parameters and arguments must be specified in ALL_CAPS:\n```diff\n- foobar[scope-1, scope-2] : `foo`{#[scope-1]} `bar`{#[scope-2]} ;\n+ foobar[SCOPE_1, SCOPE_2] : `foo`{#[SCOPE_1]} `bar`{#[SCOPE_2]} ;\n```\n\nRule parameters and arguments must be specified in lower-case:\n```diff\n- foobar[X_Y] : X_Y X_Y ;\n+ foobar[x-y] : x-y x-y ;\n```\n\n## TO-DO:\n\n- [x] **Self-host.** Accept a convenient text description of a grammar rather than require constructing a Python object by hand. [Benjamin Schaaf's sbnf](https://github.com/BenjaminSchaaf/sbnf/) is a project with essentially the same goals as this one, and has a very nice syntax for defining grammars so it'd be nice to allow inputs in that format.\n    - [x] Parse the sbnf file format\n    - [x] Handle variables and rules\n    - [x] Handle parameters\n    - [x] Handle `*`, `?`, `~`, and `( ... )`\n    - [x] Handle options for scope, meta_scope\n    - [x] Handle options for include-prototype, captures\n    - [x] Handle `prototype`\n    - [x] Handle `%embed` and `%include`\n    - [x] Handle global parameters\n    - [x] Handle command-line parameters\n- [ ] Detect whether the input grammar is follow-determined. This may be undecidable for all I know.\n- [ ] Automatically rewrite rules involving left recursion\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnuchi%2Fsublime-from-cfg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnuchi%2Fsublime-from-cfg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnuchi%2Fsublime-from-cfg/lists"}