{"id":13771910,"url":"https://github.com/wooorm/dioscuri","last_synced_at":"2025-04-19T02:52:34.352Z","repository":{"id":53131463,"uuid":"327290147","full_name":"wooorm/dioscuri","owner":"wooorm","description":"A gemtext (`text/gemini`) parser with support for streaming, ASTs, and CSTs","archived":false,"fork":false,"pushed_at":"2022-11-22T18:14:04.000Z","size":196,"stargazers_count":41,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-29T06:43:02.469Z","etag":null,"topics":["ast","cst","gemini","gemtext","html","mdast","parse"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wooorm.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":"funding.yml","license":"license","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":"wooorm"}},"created_at":"2021-01-06T11:25:01.000Z","updated_at":"2024-10-10T08:50:12.000Z","dependencies_parsed_at":"2023-01-23T14:00:15.734Z","dependency_job_id":null,"html_url":"https://github.com/wooorm/dioscuri","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wooorm%2Fdioscuri","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wooorm%2Fdioscuri/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wooorm%2Fdioscuri/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wooorm%2Fdioscuri/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wooorm","download_url":"https://codeload.github.com/wooorm/dioscuri/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249106036,"owners_count":21213647,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast","cst","gemini","gemtext","html","mdast","parse"],"created_at":"2024-08-03T17:00:57.489Z","updated_at":"2025-04-19T02:52:34.336Z","avatar_url":"https://github.com/wooorm.png","language":"JavaScript","funding_links":["https://github.com/sponsors/wooorm"],"categories":["Tools"],"sub_categories":["Gemtext converters"],"readme":"# dioscuri\n\n[![Build][build-badge]][build]\n[![Coverage][coverage-badge]][coverage]\n[![Downloads][downloads-badge]][downloads]\n[![Size][size-badge]][size]\n\nA gemtext (`text/gemini`) parser with support for streaming, ASTs, and CSTs.\n\nDo you:\n\n*   🤨 think that HTTP and HTML are bloated?\n*   😔 feel markdown has superfluous features?\n*   🤔 find gopher too light?\n*   🥰 like BRUTALISM?\n\nThen [Gemini][] might be for you (see [this post][devault] or [this\none][christine] on why it’s cool).\n\n## Contents\n\n*   [What is this?](#what-is-this)\n*   [When should I use this?](#when-should-i-use-this)\n*   [Install](#install)\n*   [Use](#use)\n*   [API](#api)\n    *   [`buffer(doc, encoding?, options?)`](#bufferdoc-encoding-options)\n    *   [`stream(options?)`](#streamoptions)\n    *   [`fromGemtext(doc, encoding?)`](#fromgemtextdoc-encoding)\n    *   [`toGemtext(tree)`](#togemtexttree)\n    *   [`fromMdast(tree, options?)`](#frommdasttree-options)\n    *   [`toMdast(tree)`](#tomdasttree)\n*   [gast](#gast)\n    *   [`Root`](#root)\n    *   [`Break`](#break)\n    *   [`Heading`](#heading)\n    *   [`Link`](#link)\n    *   [`List`](#list)\n    *   [`ListItem`](#listitem)\n    *   [`Pre`](#pre)\n    *   [`Quote`](#quote)\n    *   [`Text`](#text)\n*   [Types](#types)\n*   [Compatibility](#compatibility)\n*   [Related](#related)\n*   [Contribute](#contribute)\n*   [Security](#security)\n*   [License](#license)\n\n## What is this?\n\n**Dioscuri** (named for the gemini twins Castor and Pollux) is a\ntokenizer/lexer/parser/etc for gemtext (the `text/gemini` markup format).\nIt gives you several things:\n\n*   buffering and streaming interfaces that compile to HTML\n*   interfaces to create **[unist][]** compliant abstract syntax trees and\n    serialize those back to gemtext\n*   interfaces to transform to and from **[mdast][]** (markdown ast)\n*   parts that could be used to generate CSTs\n\nThese tools can be used if you now have markdown but want to transform it to\ngemtext.\nOr if you want to combine your posts into an RSS feed or on your “homepage”.\nAnd many other things!\n\n## When should I use this?\n\nUse this for all your gemtext needs!\n\n## Install\n\nThis package is [ESM only][esm].\nIn Node.js (version 14.14+, 16.0+), install with [npm][]:\n\n[npm][]:\n\n```sh\nnpm install dioscuri\n```\n\nIn Deno with [`esm.sh`][esmsh]:\n\n```js\nimport * as dioscuri from 'https://esm.sh/dioscuri@1'\n```\n\nIn browsers with [`esm.sh`][esmsh]:\n\n```html\n\u003cscript type=\"module\"\u003e\n  import * as dioscuri from 'https://esm.sh/dioscuri@1?bundle'\n\u003c/script\u003e\n```\n\n## Use\n\nSee each interface below for examples.\n\n## API\n\nThis package exports the identifiers `buffer`, `stream`, `fromGemtext`,\n`toGemtext`, `fromMdast`, `toMdast`.\nThe raw `compiler` and `parser` are also exported.\nThere is no default export.\n\n### `buffer(doc, encoding?, options?)`\n\nCompile gemtext to HTML.\n\n###### `doc`\n\nGemtext to parse (`string` or [`Buffer`][buffer]).\n\n###### `encoding`\n\n[Character encoding][encoding] to understand `doc` as when it’s a\n[`Buffer`][buffer] (`string`, default: `'utf8'`).\n\n###### `options.defaultLineEnding`\n\nValue to use for line endings not in `doc` (`string`, default: first line\nending or `'\\n'`).\n\nGenerally, discuri copies line endings (`'\\n'` or `'\\r\\n'`) in the document over\nto the compiled HTML.\nIn some cases, such as `\u003e a`, extra line endings are added:\n`\u003cblockquote\u003e\\n\u003cp\u003ea\u003c/p\u003e\\n\u003c/blockquote\u003e`.\n\n###### `options.allowDangerousProtocol`\n\nWhether to allow potentially dangerous protocols in URLs (`boolean`, default:\n`false`).\nURLs relative to the current protocol are always allowed (such as, `image.jpg`).\nOtherwise, the allowed protocols are `gemini`, `http`, `https`, `irc`, `ircs`,\n`mailto`, and `xmpp`.\n\n###### Returns\n\nCompiled HTML (`string`).\n\n###### Example\n\nSay we have a gemtext document, `example.gmi`:\n\n```gemini\n# Hello, world!\n\nSome text\n\n=\u003e https://example.com An example\n\n\u003e A quote\n\n* List\n```\n\n…and our module `example.js` looks as follows:\n\n```js\nimport fs from 'node:fs/promises'\nimport {buffer} from 'dioscuri'\n\nconst doc = await fs.readFile('example.gmi')\n\nconsole.log(buffer(doc))\n```\n\n…now running `node example.js` yields:\n\n```html\n\u003ch1\u003eHello, world!\u003c/h1\u003e\n\u003cbr /\u003e\n\u003cp\u003eSome text\u003c/p\u003e\n\u003cbr /\u003e\n\u003cdiv\u003e\u003ca href=\"https://example.com\"\u003eAn example\u003c/a\u003e\u003c/div\u003e\n\u003cbr /\u003e\n\u003cblockquote\u003e\n\u003cp\u003eA quote\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cbr /\u003e\n\u003cul\u003e\n\u003cli\u003eList\u003c/li\u003e\n\u003c/ul\u003e\n```\n\n### `stream(options?)`\n\nStreaming interface to compile gemtext to HTML.\n`options` is the same as the buffering interface above.\n\n###### Example\n\nAssuming the same `example.gmi` as before and an `example.js` like this:\n\n```js\nimport fs from 'node:fs'\nimport {stream} from 'dioscuri'\n\nfs.createReadStream('example.gmi')\n  .on('error', handleError)\n  .pipe(stream())\n  .pipe(process.stdout)\n\nfunction handleError(error) {\n  throw error // Handle your error here!\n}\n```\n\n…then running `node example.js` yields the same as before.\n\n### `fromGemtext(doc, encoding?)`\n\nParse gemtext to an AST (**[gast][]**).\n`doc` and `encoding` are the same as the buffering interface above.\n\n###### Returns\n\n[Root][].\n\n###### Example\n\nAssuming the same `example.gmi` as before and an `example.js` like this:\n\n```js\nimport fs from 'node:fs/promises'\nimport {fromGemtext} from 'dioscuri'\n\nconst doc = await fs.readFile('example.gmi')\n\nconsole.dir(fromGemtext(doc), {depth: null})\n```\n\n…now running `node example.js` yields (positional info removed for brevity):\n\n```js\n{\n  type: 'root',\n  children: [\n    {type: 'heading', rank: 1, value: 'Hello, world!'},\n    {type: 'break'},\n    {type: 'text', value: 'Some text'},\n    {type: 'break'},\n    {type: 'link', url: 'https://example.com', value: 'An example'},\n    {type: 'break'},\n    {type: 'quote', value: 'A quote'},\n    {type: 'break'},\n    {type: 'list', children: [{type: 'listItem', value: 'List'}]}\n  ]\n}\n```\n\n### `toGemtext(tree)`\n\nSerialize **[gast][]**.\n\n###### Example\n\nSay our script `example.js` looks as follows:\n\n```js\nimport {toGemtext} from 'dioscuri'\n\nconst tree = {\n  type: 'root',\n  children: [\n    {type: 'heading', rank: 1, value: 'Hello, world!'},\n    {type: 'break'},\n    {type: 'text', value: 'Some text'}\n  ]\n}\n\nconsole.log(toGemtext(tree))\n```\n\n…then running `node example.js` yields:\n\n```gemini\n# Hello, world!\n\nSome text\n```\n\n### `fromMdast(tree, options?)`\n\nTransform **[mdast][]** to **[gast][]**.\n\n###### `options.endlinks`\n\nPlace links at the end of the document (`boolean`, default: `false`).\nThe default is to place links before the next heading.\n\n###### `options.tight`\n\nDo not put blank lines between blocks (`boolean`, default: `false`).\nThe default is to place breaks between each block (paragraph, heading, etc).\n\n###### Returns\n\n**[gast][]**, probably.\nSome mdast nodes have no gast representation so they are dropped.\nIf you pass one of those in as `tree`, you’ll get `undefined` out.\n\n###### Example\n\nSay we have a markdown document `example.md`:\n\n````markdown\n# Hello, world!\n\nSome text, *emphasis*, **strong**\\\n`code()`, and ~~scratch that~~strikethrough.\n\nHere’s a [link](https://example.com 'Just an example'), [link reference][*],\nand images: [image reference][*], [](example.png 'Another example').\n\n***\n\n\u003e Some\n\u003e quotes\n\n*   a list\n*   with another item\n\n1.  “Ordered”\n2.  List\n\n```\nA\nPoem\n```\n\n```js\nconsole.log(1)\n```\n\n| Name | Value |\n| ---- | ----- |\n| Beep | 1.2   |\n| Boop | 3.14  |\n\n*   [x] Checked\n*   [ ] Unchecked\n\nFootnotes[^†], ^[even inline].\n\n[*]: https://example.org \"URL definition\"\n\n[^†]: Footnote definition\n````\n\n…and our module `example.js` looks as follows:\n\n```js\nimport fs from 'node:fs/promises'\nimport {gfm} from 'micromark-extension-gfm'\nimport {footnote} from 'micromark-extension-footnote'\nimport {fromMarkdown} from 'mdast-util-from-markdown'\nimport {gfmFromMarkdown} from 'mdast-util-gfm'\nimport {footnoteFromMarkdown} from 'mdast-util-footnote'\nimport {fromMdast, toGemtext} from 'dioscuri'\n\nconst mdast = fromMarkdown(await fs.readFile('example.md'), {\n  extensions: [gfm(), footnote({inlineNotes: true})],\n  mdastExtensions: [gfmFromMarkdown, footnoteFromMarkdown]\n})\n\nconsole.log(toGemtext(fromMdast(mdast)))\n```\n\n…now running `node example.js` yields:\n\n````gemini\n# Hello, world!\n\nSome text, emphasis, strong code(), and strikethrough.\n\nHere’s a link[1], link reference[2], and images: image reference[2], [3].\n\n\u003e Some quotes\n\n* a list\n* with another item\n\n* “Ordered”\n* List\n\n```\nA\nPoem\n```\n\n```js\nconsole.log(1)\n```\n\n```csv\nName,Value\nBeep,1.2\nBoop,3.14\n```\n\n* ✓ Checked\n* ✗ Unchecked\n\nFootnotes[a], [b].\n\n=\u003e https://example.com [1] Just an example\n\n=\u003e https://example.org [2] URL definition\n\n=\u003e example.png [3] Another example\n\n[a] Footnote definition\n\n[b] even inline\n````\n\n### `toMdast(tree)`\n\nTransform **[gast][]** to **[mdast][]**.\n\n###### Returns\n\n**[mdast][]**, probably.\nSome gast nodes have no mdast representation so they are dropped.\nIf you pass one of those in as `tree`, you’ll get `undefined` out.\n\n###### Example\n\nSay we have a gemtext document `example.gmi`:\n\n```gemini\n# Hello, world!\n\nSome text\n\n=\u003e https://example.com An example\n\n\u003e A quote\n\n* List\n```\n\n…and our module `example.js` looks as follows:\n\n```js\nimport fs from 'node:fs/promises'\nimport {fromGemtext, toMdast} from 'dioscuri'\n\nconst doc = await fs.readFile('example.gmi')\n\nconsole.dir(toMdast(fromGemtext(doc)), {depth: null})\n```\n\n…now running `node example.js` yields (position info removed for brevity):\n\n```js\n{\n  type: 'root',\n  children: [\n    {\n      type: 'heading',\n      depth: 1,\n      children: [{type: 'text', value: 'Hello, world!'}]\n    },\n    {\n      type: 'paragraph',\n      children: [{type: 'text', value: 'Some text'}]\n    },\n    {\n      type: 'paragraph',\n      children: [\n        {\n          type: 'link',\n          url: 'https://example.com',\n          title: null,\n          children: [{type: 'text', value: 'An example'}]\n        }\n      ]\n    },\n    {\n      type: 'blockquote',\n      children: [\n        {type: 'paragraph', children: [{type: 'text', value: 'A quote'}]}\n      ]\n    },\n    {\n      type: 'list',\n      ordered: false,\n      spread: false,\n      children: [\n        {\n          type: 'listItem',\n          spread: false,\n          children: [\n            {type: 'paragraph', children: [{type: 'text', value: 'List'}]}\n          ]\n        }\n      ]\n    }\n  ]\n}\n```\n\n## gast\n\n**[gast][]** extends **[unist][]**, a format for syntax trees, to benefit from\nits ecosystem of utilities.\n\n### `Root`\n\n```idl\ninterface Root \u003c: Parent {\n  type: 'root'\n  children: [Break | Heading | Link | List | Pre | Quote | Text]\n}\n```\n\n**Root** ([**Parent**][dfn-parent]) represents a document.\n\n### `Break`\n\n```idl\ninterface Break \u003c: Node {\n  type: 'break'\n}\n```\n\n**Break** ([**Node**][dfn-node]) represents a hard break.\n\n### `Heading`\n\n```idl\ninterface Heading \u003c: Literal {\n  type: 'heading'\n  rank: 1 | 2 | 3\n  value: string?\n}\n```\n\n**Heading** ([**Literal**][dfn-literal]) represents a heading of a section.\n\n### `Link`\n\n```idl\ninterface Link \u003c: Literal {\n  type: 'link'\n  url: string\n  value: string?\n}\n```\n\n**Link** ([**Literal**][dfn-literal]) represents a resource.\n\nA `url` field must be present.\nIt represents a URL to the resource.\n\n### `List`\n\n```idl\ninterface List \u003c: Parent {\n  type: 'list'\n  children: [ListItem]\n}\n```\n\n**List** ([**Parent**][dfn-parent]) represents an enumeration.\n\n### `ListItem`\n\n```idl\ninterface ListItem \u003c: Literal {\n  type: 'listItem'\n  value: string?\n}\n```\n\n**ListItem** ([**Literal**][dfn-literal]) represents an item in a list.\n\n### `Pre`\n\n```idl\ninterface Pre \u003c: Literal {\n  type: 'pre'\n  alt: string?\n  value: string?\n}\n```\n\n**Pre** ([**Literal**][dfn-literal]) represents preformatted text.\n\nAn `alt` field may be present.\nWhen present, the node represents computer code, and the field gives the\nlanguage of computer code being marked up.\n\n### `Quote`\n\n```idl\ninterface Quote \u003c: Literal {\n  type: 'quote'\n  value: string?\n}\n```\n\n**Quote** ([**Literal**][dfn-literal]) represents a quote.\n\n### `Text`\n\n```idl\ninterface Text \u003c: Literal {\n  type: 'text'\n  value: string\n}\n```\n\n**Text** ([**Literal**][dfn-literal]) represents a paragraph.\n\n## Types\n\nThis package is fully typed with [TypeScript][].\nIt exports the additional types `Value` (for the input, string or buffer),\n`BufferEncoding` (`'utf8'` etc), `CompileOptions` (options to turn things to a\nstring), and `FromMdastOptions` (options to turn things into gast).\n\n## Compatibility\n\nThis package is at least compatible with all maintained versions of Node.js.\nAs of now, that is Node.js 14.14+ and 16.0+.\nIt also works in Deno and modern browsers.\n\n## Related\n\n*   [`@derhuerst/gemini`](https://github.com/derhuerst/gemini)\n    – gemini protocol server and client\n*   [`gemini-fetch`](https://github.com/RangerMauve/gemini-fetch)\n    – load gemini protocol data the way you would fetch from HTTP in JavaScript\n\n## Contribute\n\nYes please!\nSee [How to Contribute to Open Source][contribute].\n\n## Security\n\nGemtext is safe.\nAs for the generated HTML: that’s safe by default.\nPass `allowDangerousProtocol: true` if you want to live dangerously.\n\n## License\n\n[MIT][license] © [Titus Wormer][author]\n\n\u003c!-- Definitions --\u003e\n\n[build-badge]: https://github.com/wooorm/dioscuri/workflows/main/badge.svg\n\n[build]: https://github.com/wooorm/dioscuri/actions\n\n[coverage-badge]: https://img.shields.io/codecov/c/github/wooorm/dioscuri.svg\n\n[coverage]: https://codecov.io/github/wooorm/dioscuri\n\n[downloads-badge]: https://img.shields.io/npm/dm/dioscuri.svg\n\n[downloads]: https://www.npmjs.com/package/dioscuri\n\n[size-badge]: https://img.shields.io/bundlephobia/minzip/dioscuri.svg\n\n[size]: https://bundlephobia.com/result?p=dioscuri\n\n[npm]: https://docs.npmjs.com/cli/install\n\n[esm]: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c\n\n[esmsh]: https://esm.sh\n\n[typescript]: https://www.typescriptlang.org\n\n[contribute]: https://opensource.guide/how-to-contribute/\n\n[license]: license\n\n[author]: https://wooorm.com\n\n[gemini]: https://gemini.circumlunar.space\n\n[unist]: https://github.com/syntax-tree/unist\n\n[mdast]: https://github.com/syntax-tree/mdast\n\n[devault]: https://drewdevault.com/2020/11/01/What-is-Gemini-anyway.html\n\n[christine]: https://christine.website/blog/gemini-web-fear-missing-out-2020-08-02\n\n[encoding]: https://nodejs.org/api/buffer.html#buffer_buffers_and_character_encodings\n\n[buffer]: https://nodejs.org/api/buffer.html\n\n[gast]: #gast\n\n[root]: #root\n\n[dfn-parent]: https://github.com/syntax-tree/unist#parent\n\n[dfn-node]: https://github.com/syntax-tree/unist#node\n\n[dfn-literal]: https://github.com/syntax-tree/unist#literal\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwooorm%2Fdioscuri","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwooorm%2Fdioscuri","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwooorm%2Fdioscuri/lists"}