Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/s0/tree-sitter-hast
NPM package to convert tree-sitter parsed syntax trees to syntax-highlighted hast
https://github.com/s0/tree-sitter-hast
syntax-highlighting syntax-tree tree-sitter unist
Last synced: about 2 months ago
JSON representation
NPM package to convert tree-sitter parsed syntax trees to syntax-highlighted hast
- Host: GitHub
- URL: https://github.com/s0/tree-sitter-hast
- Owner: s0
- License: mit
- Created: 2019-06-15T03:20:36.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2022-12-03T13:48:30.000Z (about 2 years ago)
- Last Synced: 2024-10-24T02:29:50.814Z (2 months ago)
- Topics: syntax-highlighting, syntax-tree, tree-sitter, unist
- Language: TypeScript
- Homepage: https://www.npmjs.com/package/tree-sitter-hast
- Size: 682 KB
- Stars: 12
- Watchers: 2
- Forks: 2
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# tree-sitter-hast
[](https://dev.azure.com/samlanning/tree-sitter/_build/latest?definitionId=2&branchName=master) [](https://lgtm.com/projects/g/samlanning/tree-sitter-hast/alerts/) [](https://lgtm.com/projects/g/samlanning/tree-sitter-hast/context:javascript) [](https://www.npmjs.com/package/tree-sitter-hast)
NPM package to convert [`tree-sitter`](https://tree-sitter.github.io/) parsed syntax trees to syntax-highlighted [`hast`](https://github.com/syntax-tree/hast) (Hypertext Abstract Syntax Trees).
The main reason for this is so that tree-sitter can be used to syntax-highlight code in [`unified`](https://unified.js.org/) projects such as [`remark`](https://github.com/remarkjs/remark) or [`rehype`](https://github.com/rehypejs/rehype). Via plugins such as [`remark-tree-sitter`](https://github.com/samlanning/remark-tree-sitter).
## Installation
```bash
npm install tree-sitter-hast
```
or
```bash
yarn add tree-sitter-hast
```
`tree-sitter-hast` is written in TypeScript and includes type definitions, so there is no need to install a separate `@types/tree-sitter-hast` package if you are using TypeScript.
## Usage
### Scope Mappings
For syntax highlighting,
this package uses the same process that Atom uses with `tree-sitter`.
The HTML classes that are used for syntax-highlighting do not
correspond directly to nodes in the tree produced by tree-sitter,
so scope mappings are used to specify which classes should be
applied to which syntax nodes.
(You can read mode in [Atom's documentation on Creating a Grammar](https://flight-manual.atom.io/hacking-atom/sections/creating-a-grammar/#syntax-highlighting)).
Every Atom package that provides language support using the new `tree-sitter` mechanism
also includes a scope mapping,
and this package provides functionality to directly use these packages for highlighting.
To use an atom language package, like any package you first need to install it using `npm install` or `yarn add`.
Unfortunately most APM packages are not made available on NPM,
so I've started to make some of them available under the NPM organization [`@atom-languages`](https://www.npmjs.com/org/atom-languages).
After installing a language package, you can use `loadLanguagesFromPackage` to prepare them to be used with `tree-sitter-hast`.
#### Example
```bash
npm install tree-sitter-hast @atom-languages/language-typescript
```
[`examples/example-1.js`](examples/example-1.js)
```js
const treeSitterHast = require('tree-sitter-hast');
treeSitterHast
.loadLanguagesFromPackage('@atom-languages/language-typescript')
.then(languages => {
console.log(Array.from(languages.keys()));
});
```
**Output:**
```js
[ 'flow', 'tsx', 'typescript' ]
```
### Highlighting
Highlighting is made available through the following functions:
* `highlightText(language, text, [options])` - highlight some plain text, using a language that's been made available by `loadLanguagesFromPackage`
* `highlightText(parser, scopeMappings, text, [options])` - highlight some plain text, and use a `Parser` that's already been prepared
* `highlightTree(scopeMappings, text, tree, [options])` - highlight a tree that's already been parsed by `tree-sitter`
#### Example
The following 3 examples all produce the same output.
```bash
npm install tree-sitter-hast @atom-languages/language-typescript
```
[`examples/example-2-1.js`](examples/example-2-1.js)
```js
const treeSitterHast = require('tree-sitter-hast');
const text = 'let v = 3';
treeSitterHast
.loadLanguagesFromPackage('@atom-languages/language-typescript')
.then(languages => {
const ts = languages.get('typescript');
const highlighted = treeSitterHast.highlightText(ts, text);
console.log(JSON.stringify(highlighted, null, 2));
});
```
[`examples/example-2-2.js`](examples/example-2-2.js)
```js
const Parser = require('tree-sitter');
const treeSitterHast = require('tree-sitter-hast');
const text = 'let v = 3';
treeSitterHast
.loadLanguagesFromPackage('@atom-languages/language-typescript')
.then(languages => {
const ts = languages.get('typescript');
const parser = new Parser();
parser.setLanguage(ts.grammar);
const highlighted = treeSitterHast.highlightText(parser, ts.scopeMappings, text);
console.log(JSON.stringify(highlighted, null, 2));
});
```
[`examples/example-2-3.js`](examples/example-2-3.js)
```js
const Parser = require('tree-sitter');
const treeSitterHast = require('tree-sitter-hast');
const text = 'let v = 3';
treeSitterHast
.loadLanguagesFromPackage('@atom-languages/language-typescript')
.then(languages => {
const ts = languages.get('typescript');
const parser = new Parser();
parser.setLanguage(ts.grammar);
const tree = parser.parse(text);
const highlighted = treeSitterHast.highlightTree(ts.scopeMappings, text, tree);
console.log(JSON.stringify(highlighted, null, 2));
});
```
**Output:**
```js
{
"type": "element",
"tagName": "span",
"properties": {
"className": [
"source",
"ts"
]
},
"children": [
{
"type": "element",
"tagName": "span",
"properties": {
"className": [
"storage",
"type"
]
},
"children": [
{
"type": "text",
"value": "let"
}
]
},
{
"type": "text",
"value": " v "
},
//...
]
}
```
### Exporting HTML
From this point, converting the HAST to an HTML can be done in a single call using `hast-util-to-html` (part of `rehype`):
```bash
npm install hast-util-to-html tree-sitter-hast @atom-languages/language-typescript
```
[`examples/example-3.js`](examples/example-3.js)
```js
const toHtml = require('hast-util-to-html');
const Parser = require('tree-sitter');
const treeSitterHast = require('tree-sitter-hast');
const text = 'let v = 3';
treeSitterHast
.loadLanguagesFromPackage('@atom-languages/language-typescript')
.then(languages => {
const ts = languages.get('typescript');
const highlighted = treeSitterHast.highlightText(ts, text);
// stringify to HTML
console.log(toHtml(highlighted));
});
```
**Output:**
```html
let v = 3
```
### Whitelisting Classes
Sometimes including the full list of classes applied by the scope mappings can be too much,
and you'd like to only include those that you have stylesheets for.
To do this, you can pass in a `classWhitelist` via the options parameters to `highlightText` or `highlightTree`.
```bash
npm install hast-util-to-html tree-sitter-hast @atom-languages/language-typescript
```
[`examples/example-4.js`](examples/example-4.js)
```js
const toHtml = require('hast-util-to-html');
const Parser = require('tree-sitter');
const treeSitterHast = require('tree-sitter-hast');
const text = 'let v = 3';
treeSitterHast
.loadLanguagesFromPackage('@atom-languages/language-typescript')
.then(languages => {
const ts = languages.get('typescript');
const highlighted = treeSitterHast.highlightText(ts, text, {classWhitelist: ['storage', 'numeric']});
// stringify to HTML
console.log(toHtml(highlighted));
});
```
**Output:**
```html
let v = 3
```
## TODO
* Move prepare-language.ts over to highlight-tree-sitter
* Pull out HAST type definitions into own repo (DefinitelyTyped?)
* Update highlight-tree-sitter to not produce HTML when not needed
* Move over matches patch to highlight-tree-sitter