{"id":13682612,"url":"https://github.com/microsoft/vscode-textmate","last_synced_at":"2025-05-13T17:08:22.890Z","repository":{"id":2440222,"uuid":"42534055","full_name":"microsoft/vscode-textmate","owner":"microsoft","description":"A library that helps tokenize text using Text Mate grammars.","archived":false,"fork":false,"pushed_at":"2025-02-18T14:58:35.000Z","size":6085,"stargazers_count":614,"open_issues_count":60,"forks_count":116,"subscribers_count":56,"default_branch":"main","last_synced_at":"2025-05-01T06:39:27.951Z","etag":null,"topics":["grammar","grammar-files","textmate","vscode"],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/microsoft.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-09-15T17:09:15.000Z","updated_at":"2025-04-22T17:18:02.000Z","dependencies_parsed_at":"2023-12-15T09:45:15.665Z","dependency_job_id":"78472581-4bb7-4e61-9ed0-8eb68f14df29","html_url":"https://github.com/microsoft/vscode-textmate","commit_stats":{"total_commits":417,"total_committers":22,"mean_commits":"18.954545454545453","dds":"0.36450839328537166","last_synced_commit":"f0f2a44b4c055b0e0fa3869e5de338efda5b7cf1"},"previous_names":[],"tags_count":52,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2Fvscode-textmate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2Fvscode-textmate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2Fvscode-textmate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/microsoft%2Fvscode-textmate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/microsoft","download_url":"https://codeload.github.com/microsoft/vscode-textmate/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253396460,"owners_count":21901846,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grammar","grammar-files","textmate","vscode"],"created_at":"2024-08-02T13:01:49.641Z","updated_at":"2025-05-13T17:08:17.879Z","avatar_url":"https://github.com/microsoft.png","language":"TypeScript","funding_links":[],"categories":["JavaScript","TypeScript"],"sub_categories":[],"readme":"# VSCode TextMate [![Build Status](https://dev.azure.com/ms/vscode-textmate/_apis/build/status/microsoft.vscode-textmate?branchName=master)](https://dev.azure.com/ms/vscode-textmate/_build/latest?definitionId=172\u0026branchName=master)\n\nAn interpreter for grammar files as defined by TextMate. TextMate grammars use the oniguruma dialect (https://github.com/kkos/oniguruma). Supports loading grammar files from JSON or PLIST format. This library is used in VS Code. Cross - grammar injections are currently not supported.\n\n## Installing\n\n```sh\nnpm install vscode-textmate\n```\n\n## Using\n\n```javascript\nconst fs = require('fs');\nconst path = require('path');\nconst vsctm = require('vscode-textmate');\nconst oniguruma = require('vscode-oniguruma');\n\n/**\n * Utility to read a file as a promise\n */\nfunction readFile(path) {\n    return new Promise((resolve, reject) =\u003e {\n        fs.readFile(path, (error, data) =\u003e error ? reject(error) : resolve(data));\n    })\n}\n\nconst wasmBin = fs.readFileSync(path.join(__dirname, './node_modules/vscode-oniguruma/release/onig.wasm')).buffer;\nconst vscodeOnigurumaLib = oniguruma.loadWASM(wasmBin).then(() =\u003e {\n    return {\n        createOnigScanner(patterns) { return new oniguruma.OnigScanner(patterns); },\n        createOnigString(s) { return new oniguruma.OnigString(s); }\n    };\n});\n\n// Create a registry that can create a grammar from a scope name.\nconst registry = new vsctm.Registry({\n    onigLib: vscodeOnigurumaLib,\n    loadGrammar: (scopeName) =\u003e {\n        if (scopeName === 'source.js') {\n            // https://github.com/textmate/javascript.tmbundle/blob/master/Syntaxes/JavaScript.plist\n            return readFile('./JavaScript.plist').then(data =\u003e vsctm.parseRawGrammar(data.toString()))\n        }\n        console.log(`Unknown scope name: ${scopeName}`);\n        return null;\n    }\n});\n\n// Load the JavaScript grammar and any other grammars included by it async.\nregistry.loadGrammar('source.js').then(grammar =\u003e {\n    const text = [\n        `function sayHello(name) {`,\n        `\\treturn \"Hello, \" + name;`,\n        `}`\n    ];\n    let ruleStack = vsctm.INITIAL;\n    for (let i = 0; i \u003c text.length; i++) {\n        const line = text[i];\n        const lineTokens = grammar.tokenizeLine(line, ruleStack);\n        console.log(`\\nTokenizing line: ${line}`);\n        for (let j = 0; j \u003c lineTokens.tokens.length; j++) {\n            const token = lineTokens.tokens[j];\n            console.log(` - token from ${token.startIndex} to ${token.endIndex} ` +\n              `(${line.substring(token.startIndex, token.endIndex)}) ` +\n              `with scopes ${token.scopes.join(', ')}`\n            );\n        }\n        ruleStack = lineTokens.ruleStack;\n    }\n});\n\n/* OUTPUT:\n\nUnknown scope name: source.js.regexp\n\nTokenizing line: function sayHello(name) {\n - token from 0 to 8 (function) with scopes source.js, meta.function.js, storage.type.function.js\n - token from 8 to 9 ( ) with scopes source.js, meta.function.js\n - token from 9 to 17 (sayHello) with scopes source.js, meta.function.js, entity.name.function.js\n - token from 17 to 18 (() with scopes source.js, meta.function.js, punctuation.definition.parameters.begin.js\n - token from 18 to 22 (name) with scopes source.js, meta.function.js, variable.parameter.function.js\n - token from 22 to 23 ()) with scopes source.js, meta.function.js, punctuation.definition.parameters.end.js\n - token from 23 to 24 ( ) with scopes source.js\n - token from 24 to 25 ({) with scopes source.js, punctuation.section.scope.begin.js\n\nTokenizing line:        return \"Hello, \" + name;\n - token from 0 to 1 (  ) with scopes source.js\n - token from 1 to 7 (return) with scopes source.js, keyword.control.js\n - token from 7 to 8 ( ) with scopes source.js\n - token from 8 to 9 (\") with scopes source.js, string.quoted.double.js, punctuation.definition.string.begin.js\n - token from 9 to 16 (Hello, ) with scopes source.js, string.quoted.double.js\n - token from 16 to 17 (\") with scopes source.js, string.quoted.double.js, punctuation.definition.string.end.js\n - token from 17 to 18 ( ) with scopes source.js\n - token from 18 to 19 (+) with scopes source.js, keyword.operator.arithmetic.js\n - token from 19 to 20 ( ) with scopes source.js\n - token from 20 to 24 (name) with scopes source.js, support.constant.dom.js\n - token from 24 to 25 (;) with scopes source.js, punctuation.terminator.statement.js\n\nTokenizing line: }\n - token from 0 to 1 (}) with scopes source.js, punctuation.section.scope.end.js\n\n*/\n\n```\n\n## For grammar authors\n\nSee [vscode-tmgrammar-test](https://github.com/PanAeon/vscode-tmgrammar-test) that can help you write unit tests against your grammar.\n\n## API doc\n\nSee [the main.ts file](./src/main.ts)\n\n## Developing\n\n* Clone the repository\n* Run `npm install`\n* Compile in the background with `npm run watch`\n* Run tests with `npm test`\n* Run benchmark with `npm run benchmark`\n* Troubleshoot a grammar with `npm run inspect -- PATH_TO_GRAMMAR PATH_TO_FILE`\n\n## Code of Conduct\n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\n\n## License\n[MIT](https://github.com/Microsoft/vscode-textmate/blob/master/LICENSE.md)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fvscode-textmate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicrosoft%2Fvscode-textmate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicrosoft%2Fvscode-textmate/lists"}