{"id":16759160,"url":"https://github.com/alcalzone/ansi-tokenize","last_synced_at":"2025-09-05T08:21:05.388Z","repository":{"id":177833393,"uuid":"615539449","full_name":"AlCalzone/ansi-tokenize","owner":"AlCalzone","description":"Split a string with ANSI escape codes into individual tokens","archived":false,"fork":false,"pushed_at":"2023-10-01T02:07:28.000Z","size":1275,"stargazers_count":2,"open_issues_count":5,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-10T14:52:22.133Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlCalzone.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"AlCalzone","patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":["https://paypal.me/dgriesel"]}},"created_at":"2023-03-17T23:59:04.000Z","updated_at":"2023-10-13T15:24:53.000Z","dependencies_parsed_at":null,"dependency_job_id":"4cf89329-e2c3-4e84-bba4-7b9ed43b0953","html_url":"https://github.com/AlCalzone/ansi-tokenize","commit_stats":null,"previous_names":["alcalzone/ansi-tokenize"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlCalzone%2Fansi-tokenize","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlCalzone%2Fansi-tokenize/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlCalzone%2Fansi-tokenize/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlCalzone%2Fansi-tokenize/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlCalzone","download_url":"https://codeload.github.com/AlCalzone/ansi-tokenize/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247299848,"owners_count":20916193,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T04:07:28.589Z","updated_at":"2025-04-05T07:11:21.852Z","avatar_url":"https://github.com/AlCalzone.png","language":"TypeScript","funding_links":["https://github.com/sponsors/AlCalzone","https://paypal.me/dgriesel"],"categories":[],"sub_categories":[],"readme":"# @alcalzone/ansi-tokenize\n\n\u003e Efficiently modify strings containing [ANSI escape codes](https://en.wikipedia.org/wiki/ANSI_escape_code#Colors_and_Styles)\n\nIf you find yourself modifying styled strings repeatedly, alternatives like [`slice-ansi`](https://github.com/chalk/slice-ansi/) may end up doing a lot of unnecessary work by re-parsing the string each time. This module provides a way to parse the string into an array of tokens (characters or ANSI codes), which can then be modified and re-serialized into a styled string.\n\n## Install\n\n```\n$ npm install @alcalzone/ansi-tokenize\n```\n\n## Usage\n\n### Tokenize a string\n\n```js\nimport { tokenize } from \"@alcalzone/ansi-tokenize\";\n\n// red \"foo\", followed by unstyled \"bar\"\nconst str = \"\\x1B[31mfoo\\x1B[39mbar\";\nconst tokens = tokenize(str);\n\n// tokens will now look like this:\n[\n\t{\n\t\ttype: \"ansi\",\n\t\tcode: \"\\x1B[31m\",\n\t\tendCode: \"\\x1B[39m\",\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"f\",\n\t\tfullWidth: false,\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"o\",\n\t\tfullWidth: false,\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"o\",\n\t\tfullWidth: false,\n\t},\n\t{\n\t\ttype: \"ansi\",\n\t\tcode: \"\\x1B[39m\",\n\t\tendCode: \"\\x1B[39m\",\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"b\",\n\t\tfullWidth: false,\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"a\",\n\t\tfullWidth: false,\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"r\",\n\t\tfullWidth: false,\n\t},\n];\n```\n\nEach token is either a character\n\n```ts\nexport interface Char {\n\ttype: \"char\";\n\tvalue: string;\n\tfullWidth: boolean;\n}\n```\n\nwhere\n\n-   `value` is the string representation of the character\n-   `fullWidth` is `true` if the character is full width (takes up 2 characters in monospace, like CJK characters)\n\nor an ANSI code\n\n```ts\nexport interface AnsiCode {\n\ttype: \"ansi\";\n\tcode: string;\n\tendCode: string;\n}\n```\n\nwhere\n\n-   `code` is the ANSI code that starts the style\n-   and `endCode` is the corresponding ANSI code that ends the style.\n\nAn `AnsiCode` can also be an end code, in which case `code` and `endCode` will be the same.\n\n### Convert an array of tokens into an array of \"styled\" chars\n\nThis representation is a 1:1 mapping of the original string, but not very useful for modifying the string. The `styledCharsFromTokens` function converts a token array to an array of characters, where each character has an all currently active styles associated with it:\n\n```ts\nexport interface StyledChar {\n\ttype: \"char\";\n\tvalue: string;\n\tfullWidth: boolean;\n\tstyles: AnsiCode[];\n}\n```\n\nUsing the above example:\n\n```js\nimport { tokenize, styledCharsFromTokens } from \"@alcalzone/ansi-tokenize\";\n\n// red \"foo\", followed by unstyled \"bar\"\nconst str = \"\\x1B[31mfoo\\x1B[39mbar\";\nconst tokens = tokenize(str);\n\nconst styledChars = styledCharsFromTokens(tokens);\n\n// styledChars will contain the following:\n[\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"f\",\n\t\tfullWidth: false,\n\t\tstyles: [\n\t\t\t{\n\t\t\t\ttype: \"ansi\",\n\t\t\t\tcode: \"\\x1B[31m\",\n\t\t\t\tendCode: \"\\x1B[39m\",\n\t\t\t},\n\t\t],\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"o\",\n\t\tfullWidth: false,\n\t\tstyles: [\n\t\t\t{\n\t\t\t\ttype: \"ansi\",\n\t\t\t\tcode: \"\\x1B[31m\",\n\t\t\t\tendCode: \"\\x1B[39m\",\n\t\t\t},\n\t\t],\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"o\",\n\t\tfullWidth: false,\n\t\tstyles: [\n\t\t\t{\n\t\t\t\ttype: \"ansi\",\n\t\t\t\tcode: \"\\x1B[31m\",\n\t\t\t\tendCode: \"\\x1B[39m\",\n\t\t\t},\n\t\t],\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"b\",\n\t\tfullWidth: false,\n\t\tstyles: [],\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"a\",\n\t\tfullWidth: false,\n\t\tstyles: [],\n\t},\n\t{\n\t\ttype: \"char\",\n\t\tvalue: \"r\",\n\t\tfullWidth: false,\n\t\tstyles: [],\n\t},\n];\n```\n\n### Modify an array of styled characters\n\nFor modification simply edit the items in the array as necessary, as long as the following rules are followed:\n\n1. The `code` and `endCode` properties must match. You can use the `ansi-styles` module to do this.\n2. The `fullWidth` property must be correct. You can use the `is-fullwidth-code-point` module to do this, or if working with multiple strings, turn those into styled char arrays first.\n\nE.g. to make the first `o` blue and bold:\n\n```js\nimport ansiStyles from \"ansi-styles\";\n\n// ... include the above code\n\nstyledChars[1].styles = [\n\t{\n\t\ttype: \"ansi\",\n\t\tcode: ansiStyles.blue.open,\n\t\tendCode: ansiStyles.blue.close,\n\t},\n\t{\n\t\ttype: \"ansi\",\n\t\tcode: ansiStyles.bold.open,\n\t\tendCode: ansiStyles.bold.close,\n\t},\n];\n```\n\n### Serialize a styled character array back to a string\n\nThe `styledCharsToString` function converts a styled character array back to a string:\n\n```js\nimport { styledCharsToString } from \"@alcalzone/ansi-tokenize\";\n\n// ... include the above code\n\nconst strOut = styledCharsToString(styledChars);\n\n// str will now be '\\x1B[31mf\\x1B[34m\\x1B[1mo\\x1B[22m\\x1B[31mo\\x1B[39mbar'\n```\n\nThis automatically figures out the least amount of escape codes necessary to achieve the desired result, as long as the `styles` arrays contain no unnecessary styles, e.g. blue + red foreground.\n\n## Changelog\n\n\u003c!--\n\tPlaceholder for next release:\n\t### __WORK IN PROGRESS__\n--\u003e\n### 0.1.3 (2023-09-07)\n\n-   Fix: Support links\n\n### 0.1.2 (2023-08-07)\n\n-   Fix: Reduce minimum Node.js version to `14.13.1`\n\n### 0.1.1 (2023-04-05)\n\n-   Fix: Active styles are now correctly reset at the end of the string\n\n### 0.1.0 (2023-03-20)\n\nInitial release\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falcalzone%2Fansi-tokenize","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falcalzone%2Fansi-tokenize","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falcalzone%2Fansi-tokenize/lists"}