{"id":16758222,"url":"https://github.com/foo123/codemirror-grammar","last_synced_at":"2025-03-17T02:31:09.814Z","repository":{"id":11825803,"uuid":"14379363","full_name":"foo123/codemirror-grammar","owner":"foo123","description":"Transform a JSON grammar into a syntax-highlight parser for CodeMirror","archived":false,"fork":false,"pushed_at":"2022-08-08T06:48:12.000Z","size":4988,"stargazers_count":80,"open_issues_count":4,"forks_count":8,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-02-27T16:43:45.153Z","etag":null,"topics":["bnf","codemirror-editor","grammar-parser","peg","syntax-highlighter"],"latest_commit_sha":null,"homepage":"https://foo123.github.io/examples/codemirror-grammar/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/foo123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-11-13T23:19:02.000Z","updated_at":"2023-11-05T10:15:18.000Z","dependencies_parsed_at":"2022-08-08T08:30:44.716Z","dependency_job_id":null,"html_url":"https://github.com/foo123/codemirror-grammar","commit_stats":null,"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foo123%2Fcodemirror-grammar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foo123%2Fcodemirror-grammar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foo123%2Fcodemirror-grammar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/foo123%2Fcodemirror-grammar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/foo123","download_url":"https://codeload.github.com/foo123/codemirror-grammar/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243837009,"owners_count":20355813,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bnf","codemirror-editor","grammar-parser","peg","syntax-highlighter"],"created_at":"2024-10-13T04:04:32.618Z","updated_at":"2025-03-17T02:31:09.344Z","avatar_url":"https://github.com/foo123.png","language":"JavaScript","readme":"codemirror-grammar\n==================\n\n__Transform a JSON grammar into a CodeMirror syntax-highlight parser__\n\n\n![CodeMirror Grammar](/codemirror-grammar.png)\n\nA simple and light-weight (~ 55kB minified, ~ 18kB zipped) [CodeMirror](https://github.com/marijnh/codemirror) add-on\n\nto generate syntax-highlight parsers (codemirror modes) from a grammar specification in JSON format.\n\n\n**see also:**\n\n* [Abacus](https://github.com/foo123/Abacus) advanced Combinatorics and Algebraic Number Theory Symbolic Computation library for JavaScript, Python\n* [Plot.js](https://github.com/foo123/Plot.js) simple and small library which can plot graphs of functions and various simple charts and can render to Canvas, SVG and plain HTML\n* [HAAR.js](https://github.com/foo123/HAAR.js) image feature detection based on Haar Cascades in JavaScript (Viola-Jones-Lienhart et al Algorithm)\n* [HAARPHP](https://github.com/foo123/HAARPHP) image feature detection based on Haar Cascades in PHP (Viola-Jones-Lienhart et al Algorithm)\n* [FILTER.js](https://github.com/foo123/FILTER.js) video and image processing and computer vision Library in pure JavaScript (browser and node)\n* [Xpresion](https://github.com/foo123/Xpresion) a simple and flexible eXpression parser engine (with custom functions and variables support), based on [GrammarTemplate](https://github.com/foo123/GrammarTemplate), for PHP, JavaScript, Python\n* [Regex Analyzer/Composer](https://github.com/foo123/RegexAnalyzer) Regular Expression Analyzer and Composer for PHP, JavaScript, Python\n* [GrammarTemplate](https://github.com/foo123/GrammarTemplate) grammar-based templating for PHP, JavaScript, Python\n* [codemirror-grammar](https://github.com/foo123/codemirror-grammar) transform a formal grammar in JSON format into a syntax-highlight parser for CodeMirror editor\n* [ace-grammar](https://github.com/foo123/ace-grammar) transform a formal grammar in JSON format into a syntax-highlight parser for ACE editor\n* [prism-grammar](https://github.com/foo123/prism-grammar) transform a formal grammar in JSON format into a syntax-highlighter for Prism code highlighter\n* [highlightjs-grammar](https://github.com/foo123/highlightjs-grammar) transform a formal grammar in JSON format into a syntax-highlight mode for Highlight.js code highlighter\n* [syntaxhighlighter-grammar](https://github.com/foo123/syntaxhighlighter-grammar) transform a formal grammar in JSON format to a highlight brush for SyntaxHighlighter code highlighter\n* [SortingAlgorithms](https://github.com/foo123/SortingAlgorithms) implementations of Sorting Algorithms in JavaScript\n* [PatternMatchingAlgorithms](https://github.com/foo123/PatternMatchingAlgorithms) implementations of Pattern Matching Algorithms in JavaScript\n\n\n\n**Note:** The invariant codebase for all the `*-grammar` add-ons resides at [editor-grammar](https://github.com/foo123/editor-grammar) repository (used as a `git submodule`)\n\n\n### Contents\n\n* [Live Playground Example](https://foo123.github.io/examples/codemirror-grammar)\n* [Todo](#todo)\n* [Features](#features)\n* [How To use](#how-to-use)\n* [API Reference](/api-reference.md)\n* [Grammar Reference](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md)\n* [Other Examples](#other-examples)\n\n[![Build your own syntax-highlight mode on the fly](/test/screenshot.png)](https://foo123.github.io/examples/codemirror-grammar)\n\n\n### Todo\n\nCode Indentation is Codemirror default, see [Modularity and Future Directions](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#modularity-and-future-directions)\n\n\n* handle arbitrary, user-defined, code `(de-)indentation` in the `grammar` specification (e.g via `indent action` tokens)\n* handle arbitrary, user-defined, code `matching` (e.g `brackets`, `tags`, etc..) in the `grammar` specification (e.g via `match action` tokens) [DONE]\n\u003c!--* handle arbitrary, user-defined, `(operator) precedence` relations in the `grammar` specification (e.g via `precedence action` tokens)--\u003e\n* handle arbitrary, user-defined, `local/global/scoped` relations in the `grammar` specification (e.g via `scope action` tokens) [DONE]\n* and so on..\n* enable grammar add-on to pre-compile a grammar specification directly into mode source code, so it can be used without the add-on as standalone mode [TODO, maybe]\n\n\n\n### Features\n\n* A [`Grammar`](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md) can **extend other `Grammars`** (so arbitrary `variations` and `dialects` can be handled more easily)\n* `Grammar` includes: [`Style Model`](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#style-model) , [`Lex Model`](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#lexical-model) and [`Syntax Model` (optional)](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#syntax-model), plus a couple of [*settings*](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#extra-settings) (see examples)\n* **`Grammar` specification can be minimal**, defaults will be used (see example grammars)\n* [`Grammar.Syntax Model`](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#syntax-model) can enable highlight in a more *context-specific* way, plus detect possible *syntax errors* and display appropriate *error messages* (see below)\n* [`Grammar.Syntax Model`](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#syntax-model) can contain **recursive references** (see `/test/grammar-js-recursion.html`)\n* [`Grammar.Syntax Model`](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#syntax-pegbnf-like-notations) can be (fully) specificed using [`PEG`](https://en.wikipedia.org/wiki/Parsing_expression_grammar)-like notation or [`BNF`](https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_Form)-like notation  (**NEW feature**)\n* [`Grammar.Syntax Model`](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#syntax-pegbnf-like-notations) implements **positive / negative lookahead tokens** (analogous to `PEG` `and-`/`not-` predicates)  (**NEW feature**)\n* [`Grammar.Syntax Model`](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#syntax-model) can include **external (sub-)grammars so that new multiplexed / mixed grammars** are created easily and intuitively (see test examples) (**NEW feature**)\n* `Grammar` can define [*action* tokens](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#action-tokens) to perform *complex context-specific* parsing functionality, including **associated tag matching** and **duplicate identifiers** (see for example `xml.grammar` example) (**NEW feature**)\n* Generated highlight modes can support **toggle comments** and **keyword autocompletion** functionality if defined in the grammar\n* **Context-sensitive autocompletion** extracted directly from the grammar specification  (**NEW feature**)\n* **Dynamic (Context-sensitive) autocompletion** from typed user actions like code/token/symbols  (**NEW feature**)\n* Generated highlight modes can support **lint-like syntax-annotation** functionality generated from the grammar\n* Generated highlight modes can support custom, user-defined, **code folding** functionality from the [grammar `fold` model](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#code-folding)  (**NEW feature**)\n* Generated highlight modes can support custom, user-defined, **code token matching** functionality from the [grammar `match` model](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#code-matching)  (**NEW feature**)\n* Generated parsers are **optimized for speed and size**\n* Can generate a syntax-highlight parser from a grammar **interactively and on-the-fly** ( see example, http://foo123.github.io/examples/codemirror-grammar )\n* see also [Modularity and Future Directions](https://github.com/foo123/editor-grammar/blob/master/grammar-reference.md#modularity-and-future-directions)\n\n\n\n### How to use:\n\nSee working examples under [/test](/test) folder.\n\nAn example for XML:\n\n\n```javascript\n// 1. a partial xml grammar in simple JSON format\nvar xml_grammar = {\n    \n// prefix ID for regular expressions, represented as strings, used in the grammar\n\"RegExpID\"                          : \"RE::\",\n\n\"Extra\"                             : {\n    \n    \"fold\"                          : \"xml\"\n    //\"electricChars\"               : \"\u003c\"\n    \n},\n    \n// Style model\n\"Style\"                             : {\n\n     \"comment\"                      : \"comment\"\n    ,\"declaration\"                  : \"tag\"\n    ,\"doctype\"                      : \"meta\"\n    ,\"meta\"                         : \"meta\"\n    ,\"cdata\"                        : \"atom\"\n    ,\"tag\"                          : \"tag\"\n    ,\"attribute\"                    : \"attribute\"\n    ,\"string\"                       : \"string\"\n    ,\"atom\"                         : \"atom\"\n    ,\"number\"                       : \"number\"\n    ,\"error\"                        : \"error\"\n    \n},\n\n// Lexical model\n\"Lex\"                               : {\n     \n     \"comment:comment\"              : [\"\u003c!--\", \"--\u003e\"]\n    ,\"declaration:block\"            : [\"\u003c?xml\", \"?\u003e\"]\n    ,\"doctype:block\"                : [\"RE::/\u003c!doctype\\\\b/i\", \"\u003e\"]\n    ,\"meta:block\"                   : [\"RE::/\u003c\\\\?[_a-zA-Z][\\\\w\\\\._\\\\-]*/\", \"?\u003e\"]\n    ,\"cdata:block\"                  : [\"\u003c![CDATA[\", \"]]\u003e\"]\n    ,\"open_tag\"                     : \"RE::/\u003c((?:[_a-zA-Z][_a-zA-Z0-9\\\\-]*:)?[_a-zA-Z][_a-zA-Z0-9\\\\-]*)\\\\b/\"\n    ,\"close_tag\"                    : \"RE::/\u003c\\\\/((?:[_a-zA-Z][_a-zA-Z0-9\\\\-]*:)?[_a-zA-Z][_a-zA-Z0-9\\\\-]*)\u003e/\"\n    ,\"attribute\"                    : \"RE::/[_a-zA-Z][_a-zA-Z0-9\\\\-]*/\"\n    ,\"string:line-block\"            : [[\"\\\"\"], [\"'\"]]\n    ,\"number\"                       : [\"RE::/[0-9]\\\\d*/\", \"RE::/#[0-9a-fA-F]+/\"]\n    ,\"atom\"                         : [\"RE::/\u0026#x[a-fA-F\\\\d]+;/\", \"RE::/\u0026#[\\\\d]+;/\", \"RE::/\u0026[a-zA-Z][a-zA-Z0-9]*;/\"]\n    ,\"text\"                         : \"RE::/[^\u003c\u0026]+/\"\n    \n    // actions\n    ,\"@tag:action\"                  : {\"context\":true}\n    ,\"tag@:action\"                  : {\"context\":false}\n    ,\"@unique_id:action\"            : {\"unique\":[\"xml\", \"$1\"],\"msg\":\"Duplicate id value \\\"$0\\\"\",\"mode\":\"hash\"}\n    ,\"@unique_att:action\"           : {\"unique\":[\"att\", \"$0\"],\"msg\":\"Duplicate attribute \\\"$0\\\"\",\"mode\":\"hash\",\"in-context\":true}\n    ,\"@tag_opened:action\"           : {\"push\":\"\u003c$1\u003e\",\"ci\":true}\n    ,\"@tag_closed:action\"           : {\"pop\":\"\u003c$1\u003e\",\"ci\":true,\"msg\":\"Tags \\\"$0\\\" and \\\"$1\\\" do not match\"}\n    ,\"@tag_autoclosed:action\"       : {\"pop\":null}\n    ,\"@autocomplete:action\"         : {\"define\":[\"autocomplete\",\"$1\"],\"msg\":false,\"autocomplete\":true,\"mode\":\"hash\"}\n    ,\"@out_of_place:error\"          : \"\\\"$2$3\\\" can only be at the beginning of XML document\"\n    \n},\n    \n// Syntax model (optional)\n\"Syntax\"                            : {\n     \n     \"tag_att\"                      : \"'id'.attribute @unique_att '=' string @unique_id | attribute @unique_att '=' (string | number)\"\n    ,\"start_tag\"                    : \"open_tag.tag @tag @autocomplete @tag_opened tag_att* ('\u003e'.tag | '/\u003e'.tag @tag_autoclosed) tag@\"\n    ,\"end_tag\"                      : \"close_tag.tag @autocomplete @tag_closed\"\n    ,\"xml\"                          : \"(^^1 declaration? doctype?) (declaration.error @out_of_place | doctype.error @out_of_place | comment | meta | cdata | start_tag | end_tag | atom | text)*\"\n    \n},\n    \n// what to parse and in what order\n\"Parser\"                            : [ [\"xml\"] ]\n\n};\n        \n// 2. parse the grammar into a Codemirror syntax-highlight mode\nvar xml_mode = CodeMirrorGrammar.getMode( xml_grammar );\n\n\n// 3. use it with Codemirror\nCodeMirror.defineMode(\"xml\", xml_mode);\n\n// enable user-defined code folding in the specification (new feature)\nxml_mode.supportCodeFolding = true;\nCodeMirror.registerHelper(\"fold\", xml_mode.foldType, xml_mode.folder);\n\n// enable user-defined code matching in the specification (new feature)\nxml_mode.supportCodeMatching = true;\nxml_mode.matcher.options = {maxHighlightLineLength:1000}; // default\nCodeMirror.defineOption(\"matching\", false, function( cm, val, old ) {\n    if ( old \u0026\u0026 old != CodeMirror.Init )\n    {\n        cm.off( \"cursorActivity\", xml_mode.matcher );\n        xml_mode.matcher.clear( cm );\n    }\n    if ( val )\n    {\n        cm.on( \"cursorActivity\", xml_mode.matcher );\n        xml_mode.matcher( cm );\n    }\n});\n\n// enable syntax lint-like validation in the grammar\nxml_mode.supportGrammarAnnotations = true;\nCodeMirror.registerHelper(\"lint\", \"xml\", xml_mode.validator);\n\n// enable user-defined autocompletion (if defined)\nxml_mode.supportAutoCompletion = true;\nCodeMirror.commands['my_autocompletion'] = function( cm ) {\n    CodeMirror.showHint(cm, xml_mode.autocompleter, {prefixMatch:true, caseInsensitiveMatch:false});\n};\n// this also works (takes priority if set)\nxml_mode.autocompleter.options = {prefixMatch:true, caseInsensitiveMatch:false};\n// or for context-sensitive autocompletion, extracted from the grammar\nxml_mode.autocompleter.options = {prefixMatch:true, caseInsensitiveMatch:false, inContext:true};\n// or for dynamic (context-sensitive) autocompletion, extracted from user actions\nxml_mode.autocompleter.options = {prefixMatch:true, caseInsensitiveMatch:false, inContext:true|false, dynamic:true};\n\nvar editor = CodeMirror.fromTextArea(document.getElementById(\"code\"), {\n    mode: \"xml\",\n    lineNumbers: true,\n    indentUnit: 4,\n    indentWithTabs: false,\n    lint: true,  // enable lint validation\n    matching: true,  // enable token matching, e.g braces, tags etc..\n    extraKeys: {\"Ctrl-Space\": 'my_autocompletion', \"Ctrl-L\": \"toggleComment\"},\n    foldGutter: true,\n    gutters: [\"CodeMirror-lint-markers\", \"CodeMirror-linenumbers\", \"CodeMirror-foldgutter\"]\n});\n\n```\n\n\nResult:\n\n![xml-grammar-1](/test/grammar-xml.png)\n![xml-grammar-2](/test/grammar-xml-2.png)\n![xml-grammar-2](/test/grammar-xml-3.png)\n\n\n\n### Other Examples:\n\n\n![htmlmixed-grammar](/test/grammar-htmlmixed.png)\n\n![js-recursive-grammar](/test/grammar-js-recursion.png)\n![js-recursive-grammar-autocomplete](/test/grammar-js-recursion-2.png)\n\n![js-scoped-grammar](/test/grammar-js-scoped.png)\n![js-scoped-grammar](/test/grammar-js-scoped-2.png)\n\n![css-grammar](/test/grammar-css.png)\n\n![python-grammar](/test/grammar-python.png)\n\n![php-grammar](/test/grammar-php.png)\n\n![scheme-grammar](/test/grammar-scheme.png)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoo123%2Fcodemirror-grammar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffoo123%2Fcodemirror-grammar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffoo123%2Fcodemirror-grammar/lists"}