{"id":13716652,"url":"https://github.com/syntax-tree/mdast-util-to-nlcst","last_synced_at":"2025-05-03T10:31:56.587Z","repository":{"id":35422160,"uuid":"39687248","full_name":"syntax-tree/mdast-util-to-nlcst","owner":"syntax-tree","description":"utility to transform mdast to nlcst","archived":false,"fork":false,"pushed_at":"2024-04-30T12:18:33.000Z","size":264,"stargazers_count":9,"open_issues_count":0,"forks_count":1,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-04-07T14:51:23.172Z","etag":null,"topics":["markdown","mdast","mdast-util","natural-language","nlcst","nlcst-util","syntax-tree","unist"],"latest_commit_sha":null,"homepage":"https://unifiedjs.com","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/syntax-tree.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"unifiedjs","open_collective":"unified"}},"created_at":"2015-07-25T13:51:21.000Z","updated_at":"2024-04-30T12:18:35.000Z","dependencies_parsed_at":"2024-06-18T18:20:28.686Z","dependency_job_id":"681559cb-8c32-485b-a71d-3cf9ea8ced88","html_url":"https://github.com/syntax-tree/mdast-util-to-nlcst","commit_stats":{"total_commits":156,"total_committers":2,"mean_commits":78.0,"dds":"0.012820512820512775","last_synced_commit":"e05a1eddef74b372d4b50b6cb8e73ec91bb8ef79"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/syntax-tree%2Fmdast-util-to-nlcst","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/syntax-tree%2Fmdast-util-to-nlcst/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/syntax-tree%2Fmdast-util-to-nlcst/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/syntax-tree%2Fmdast-util-to-nlcst/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/syntax-tree","download_url":"https://codeload.github.com/syntax-tree/mdast-util-to-nlcst/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251089988,"owners_count":21534562,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["markdown","mdast","mdast-util","natural-language","nlcst","nlcst-util","syntax-tree","unist"],"created_at":"2024-08-03T00:01:12.985Z","updated_at":"2025-05-03T10:31:56.205Z","avatar_url":"https://github.com/syntax-tree.png","language":"JavaScript","funding_links":["https://github.com/sponsors/unifiedjs","https://opencollective.com/unified"],"categories":["mdast utilities","JavaScript"],"sub_categories":[],"readme":"# mdast-util-to-nlcst\n\n[![Build][build-badge]][build]\n[![Coverage][coverage-badge]][coverage]\n[![Downloads][downloads-badge]][downloads]\n[![Size][size-badge]][size]\n[![Sponsors][sponsors-badge]][collective]\n[![Backers][backers-badge]][collective]\n[![Chat][chat-badge]][chat]\n\n[mdast][] utility to transform to [nlcst][].\n\n## Contents\n\n* [What is this?](#what-is-this)\n* [When should I use this?](#when-should-i-use-this)\n* [Install](#install)\n* [Use](#use)\n* [API](#api)\n  * [`toNlcst(tree, file, Parser[, options])`](#tonlcsttree-file-parser-options)\n  * [`Options`](#options)\n  * [`ParserConstructor`](#parserconstructor)\n  * [`ParserInstance`](#parserinstance)\n* [Types](#types)\n* [Compatibility](#compatibility)\n* [Security](#security)\n* [Related](#related)\n* [Contribute](#contribute)\n* [License](#license)\n\n## What is this?\n\nThis package is a utility that takes an [mdast][] (markdown) syntax tree as\ninput and turns it into [nlcst][] (natural language).\n\n## When should I use this?\n\nThis project is useful when you want to deal with ASTs and inspect the natural\nlanguage inside markdown.\nUnfortunately, there is no way yet to apply changes to the nlcst back into\nmdast.\n\nThe hast utility [`hast-util-to-nlcst`][hast-util-to-nlcst] does the same but\nuses an HTML tree as input.\n\nThe remark plugin [`remark-retext`][remark-retext] wraps this utility to do the\nsame at a higher-level (easier) abstraction.\n\n## Install\n\nThis package is [ESM only][esm].\nIn Node.js (version 16+), install with [npm][]:\n\n```sh\nnpm install mdast-util-to-nlcst\n```\n\nIn Deno with [`esm.sh`][esmsh]:\n\n```js\nimport {toNlcst} from 'https://esm.sh/mdast-util-to-nlcst@7'\n```\n\nIn browsers with [`esm.sh`][esmsh]:\n\n```html\n\u003cscript type=\"module\"\u003e\n  import {toNlcst} from 'https://esm.sh/mdast-util-to-nlcst@7?bundle'\n\u003c/script\u003e\n```\n\n## Use\n\nSay we have the following `example.md`:\n\n```markdown\nSome *foo*sball.\n```\n\n…and next to it a module `example.js`:\n\n```js\nimport {fromMarkdown} from 'mdast-util-from-markdown'\nimport {toNlcst} from 'mdast-util-to-nlcst'\nimport {ParseEnglish} from 'parse-english'\nimport {read} from 'to-vfile'\nimport {inspect} from 'unist-util-inspect'\n\nconst file = await read('example.md')\nconst mdast = fromMarkdown(file)\nconst nlcst = toNlcst(mdast, file, ParseEnglish)\n\nconsole.log(inspect(nlcst))\n```\n\nYields:\n\n```txt\nRootNode[1] (1:1-1:17, 0-16)\n└─0 ParagraphNode[1] (1:1-1:17, 0-16)\n    └─0 SentenceNode[4] (1:1-1:17, 0-16)\n        ├─0 WordNode[1] (1:1-1:5, 0-4)\n        │   └─0 TextNode \"Some\" (1:1-1:5, 0-4)\n        ├─1 WhiteSpaceNode \" \" (1:5-1:6, 4-5)\n        ├─2 WordNode[2] (1:7-1:16, 6-15)\n        │   ├─0 TextNode \"foo\" (1:7-1:10, 6-9)\n        │   └─1 TextNode \"sball\" (1:11-1:16, 10-15)\n        └─3 PunctuationNode \".\" (1:16-1:17, 15-16)\n```\n\n## API\n\nThis package exports the identifier [`toNlcst`][api-to-nlcst].\nThere is no default export.\n\n### `toNlcst(tree, file, Parser[, options])`\n\nTurn an mdast tree into an nlcst tree.\n\n\u003e 👉 **Note**: `tree` must have positional info and `file` must be a `VFile`\n\u003e corresponding to `tree`.\n\n###### Parameters\n\n* `tree` ([`MdastNode`][mdast-node])\n  — mdast tree to transform\n* `file` ([`VFile`][vfile])\n  — virtual file\n* `Parser` ([`ParserConstructor`][api-parser-constructor] or\n  [`ParserInstance`][api-parser-instance])\n  — parser to use\n* `options` ([`Options`][api-options], optional)\n  — configuration\n\n###### Returns\n\nnlcst tree ([`NlcstNode`][nlcst-node]).\n\n### `Options`\n\nConfiguration (TypeScript type).\n\n##### Fields\n\n###### `ignore`\n\nList of [mdast][] node types to ignore (`Array\u003cstring\u003e`, optional).\n\nThe types `'table'`, `'tableRow'`, and `'tableCell'` are always ignored.\n\n\u003cdetails\u003e\u003csummary\u003eShow example\u003c/summary\u003e\n\nSay we have the following file `example.md`:\n\n```md\nA paragraph.\n\n\u003e A paragraph in a block quote.\n```\n\n…and if we now transform with `ignore: ['blockquote']`, we get:\n\n```txt\nRootNode[2] (1:1-3:1, 0-14)\n├─0 ParagraphNode[1] (1:1-1:13, 0-12)\n│   └─0 SentenceNode[4] (1:1-1:13, 0-12)\n│       ├─0 WordNode[1] (1:1-1:2, 0-1)\n│       │   └─0 TextNode \"A\" (1:1-1:2, 0-1)\n│       ├─1 WhiteSpaceNode \" \" (1:2-1:3, 1-2)\n│       ├─2 WordNode[1] (1:3-1:12, 2-11)\n│       │   └─0 TextNode \"paragraph\" (1:3-1:12, 2-11)\n│       └─3 PunctuationNode \".\" (1:12-1:13, 11-12)\n└─1 WhiteSpaceNode \"\\n\\n\" (1:13-3:1, 12-14)\n```\n\n\u003c/details\u003e\n\n###### `source`\n\nList of [mdast][] node types to mark as [nlcst][] source nodes\n(`Array\u003cstring\u003e`, optional).\n\nThe type `'inlineCode'` is always marked as source.\n\n\u003cdetails\u003e\u003csummary\u003eShow example\u003c/summary\u003e\n\nSay we have the following file `example.md`:\n\n```md\nA paragraph.\n\n\u003e A paragraph in a block quote.\n```\n\n…and if we now transform with `source: ['blockquote']`, we get:\n\n```txt\nRootNode[3] (1:1-3:32, 0-45)\n├─0 ParagraphNode[1] (1:1-1:13, 0-12)\n│   └─0 SentenceNode[4] (1:1-1:13, 0-12)\n│       ├─0 WordNode[1] (1:1-1:2, 0-1)\n│       │   └─0 TextNode \"A\" (1:1-1:2, 0-1)\n│       ├─1 WhiteSpaceNode \" \" (1:2-1:3, 1-2)\n│       ├─2 WordNode[1] (1:3-1:12, 2-11)\n│       │   └─0 TextNode \"paragraph\" (1:3-1:12, 2-11)\n│       └─3 PunctuationNode \".\" (1:12-1:13, 11-12)\n├─1 WhiteSpaceNode \"\\n\\n\" (1:13-3:1, 12-14)\n└─2 ParagraphNode[1] (3:1-3:32, 14-45)\n    └─0 SentenceNode[1] (3:1-3:32, 14-45)\n        └─0 SourceNode \"\u003e A paragraph in a block quote.\" (3:1-3:32, 14-45)\n```\n\n\u003c/details\u003e\n\n### `ParserConstructor`\n\nCreate a new parser (TypeScript type).\n\n###### Type\n\n```ts\ntype ParserConstructor = new () =\u003e ParserInstance\n```\n\n### `ParserInstance`\n\nnlcst parser (TypeScript type).\n\nFor example, [`parse-dutch`][parse-dutch], [`parse-english`][parse-english], or\n[`parse-latin`][parse-latin].\n\n###### Type\n\n```ts\ntype ParserInstance = {\n  tokenizeSentencePlugins: ((node: NlcstSentence) =\u003e undefined)[]\n  tokenizeParagraphPlugins: ((node: NlcstParagraph) =\u003e undefined)[]\n  tokenizeRootPlugins: ((node: NlcstRoot) =\u003e undefined)[]\n  parse(value: string | null | undefined): NlcstRoot\n  tokenize(value: string | null | undefined): Array\u003cNlcstSentenceContent\u003e\n}\n```\n\n## Types\n\nThis package is fully typed with [TypeScript][].\nIt exports the types [`Options`][api-options],\n[`ParserConstructor`][api-parser-constructor], and\n[`ParserInstance`][api-parser-instance].\n\n## Compatibility\n\nProjects maintained by the unified collective are compatible with maintained\nversions of Node.js.\n\nWhen we cut a new major release, we drop support for unmaintained versions of\nNode.\nThis means we try to keep the current release line, `mdast-util-to-nlcst@^7`,\ncompatible with Node.js 16.\n\n## Security\n\nUse of `mdast-util-to-nlcst` does not involve [**hast**][hast] so there are no\nopenings for [cross-site scripting (XSS)][xss] attacks.\n\n## Related\n\n* [`mdast-util-to-hast`](https://github.com/syntax-tree/mdast-util-to-hast)\n  — transform mdast to hast\n* [`hast-util-to-nlcst`](https://github.com/syntax-tree/hast-util-to-nlcst)\n  — transform hast to nlcst\n* [`hast-util-to-mdast`](https://github.com/syntax-tree/hast-util-to-mdast)\n  — transform hast to mdast\n* [`hast-util-to-xast`](https://github.com/syntax-tree/hast-util-to-xast)\n  — transform hast to xast\n* [`hast-util-sanitize`](https://github.com/syntax-tree/hast-util-sanitize)\n  — sanitize hast nodes\n\n## Contribute\n\nSee [`contributing.md`][contributing] in [`syntax-tree/.github`][health] for\nways to get started.\nSee [`support.md`][support] for ways to get help.\n\nThis project has a [code of conduct][coc].\nBy interacting with this repository, organization, or community you agree to\nabide by its terms.\n\n## License\n\n[MIT][license] © [Titus Wormer][author]\n\n\u003c!-- Definitions --\u003e\n\n[build-badge]: https://github.com/syntax-tree/mdast-util-to-nlcst/workflows/main/badge.svg\n\n[build]: https://github.com/syntax-tree/mdast-util-to-nlcst/actions\n\n[coverage-badge]: https://img.shields.io/codecov/c/github/syntax-tree/mdast-util-to-nlcst.svg\n\n[coverage]: https://codecov.io/github/syntax-tree/mdast-util-to-nlcst\n\n[downloads-badge]: https://img.shields.io/npm/dm/mdast-util-to-nlcst.svg\n\n[downloads]: https://www.npmjs.com/package/mdast-util-to-nlcst\n\n[size-badge]: https://img.shields.io/badge/dynamic/json?label=minzipped%20size\u0026query=$.size.compressedSize\u0026url=https://deno.bundlejs.com/?q=mdast-util-to-nlcst\n\n[size]: https://bundlejs.com/?q=mdast-util-to-nlcst\n\n[sponsors-badge]: https://opencollective.com/unified/sponsors/badge.svg\n\n[backers-badge]: https://opencollective.com/unified/backers/badge.svg\n\n[collective]: https://opencollective.com/unified\n\n[chat-badge]: https://img.shields.io/badge/chat-discussions-success.svg\n\n[chat]: https://github.com/syntax-tree/unist/discussions\n\n[npm]: https://docs.npmjs.com/cli/install\n\n[esm]: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c\n\n[esmsh]: https://esm.sh\n\n[typescript]: https://www.typescriptlang.org\n\n[license]: license\n\n[author]: https://wooorm.com\n\n[health]: https://github.com/syntax-tree/.github\n\n[contributing]: https://github.com/syntax-tree/.github/blob/main/contributing.md\n\n[support]: https://github.com/syntax-tree/.github/blob/main/support.md\n\n[coc]: https://github.com/syntax-tree/.github/blob/main/code-of-conduct.md\n\n[xss]: https://en.wikipedia.org/wiki/Cross-site_scripting\n\n[mdast]: https://github.com/syntax-tree/mdast\n\n[mdast-node]: https://github.com/syntax-tree/mdast#nodes\n\n[nlcst]: https://github.com/syntax-tree/nlcst\n\n[nlcst-node]: https://github.com/syntax-tree/nlcst#node\n\n[hast]: https://github.com/syntax-tree/hast\n\n[hast-util-to-nlcst]: https://github.com/syntax-tree/hast-util-to-nlcst\n\n[remark-retext]: https://github.com/remarkjs/remark-retext\n\n[vfile]: https://github.com/vfile/vfile\n\n[parse-english]: https://github.com/wooorm/parse-english\n\n[parse-latin]: https://github.com/wooorm/parse-latin\n\n[parse-dutch]: https://github.com/wooorm/parse-dutch\n\n[api-to-nlcst]: #tonlcsttree-file-parser-options\n\n[api-options]: #options\n\n[api-parser-constructor]: #parserconstructor\n\n[api-parser-instance]: #parserinstance\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsyntax-tree%2Fmdast-util-to-nlcst","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsyntax-tree%2Fmdast-util-to-nlcst","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsyntax-tree%2Fmdast-util-to-nlcst/lists"}