{"id":16418517,"url":"https://github.com/nitely/nim-unicodedb","last_synced_at":"2025-07-11T01:35:13.546Z","repository":{"id":54296014,"uuid":"111746421","full_name":"nitely/nim-unicodedb","owner":"nitely","description":"Unicode Character Database (UCD, tr44) for Nim","archived":false,"fork":false,"pushed_at":"2024-12-31T01:38:57.000Z","size":9586,"stargazers_count":34,"open_issues_count":2,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-23T18:37:28.111Z","etag":null,"topics":["nim","ucd","ucd-data","unicode","unicode-data"],"latest_commit_sha":null,"homepage":"https://nitely.github.io/nim-unicodedb/","language":"Nim","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nitely.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-23T00:45:37.000Z","updated_at":"2024-12-31T01:38:07.000Z","dependencies_parsed_at":"2024-03-30T21:26:54.589Z","dependency_job_id":"306417c4-8844-4539-a78e-b13b8a61ba23","html_url":"https://github.com/nitely/nim-unicodedb","commit_stats":null,"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nitely%2Fnim-unicodedb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nitely%2Fnim-unicodedb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nitely%2Fnim-unicodedb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nitely%2Fnim-unicodedb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nitely","download_url":"https://codeload.github.com/nitely/nim-unicodedb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248065281,"owners_count":21041872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["nim","ucd","ucd-data","unicode","unicode-data"],"created_at":"2024-10-11T07:14:20.681Z","updated_at":"2025-04-09T16:20:43.495Z","avatar_url":"https://github.com/nitely.png","language":"Nim","funding_links":[],"categories":[],"sub_categories":[],"readme":"# UnicodeDB\n\n[![Build Status](https://img.shields.io/travis/nitely/nim-unicodedb.svg?style=flat-square)](https://travis-ci.org/nitely/nim-unicodedb)\n[![licence](https://img.shields.io/github/license/nitely/nim-unicodedb.svg?style=flat-square)](https://raw.githubusercontent.com/nitely/nim-unicodedb/master/LICENSE)\n\nThis library aims to bring the unicode database to Nim. Main goal is\nhaving O(1) access for every API and be lightweight in size.\n\n\u003e Note: this library doesn't provide Unicode Common Locale Data (UCLD / CLDR data)\n\n## Install\n\n```\nnimble install unicodedb\n```\n\n## Compatibility\n\nNim +1.0.0\n\n## Usage\n\n### Properties\n\n```nim\nimport unicode\nimport unicodedb/properties\n\nassert Rune('A'.ord).unicodeCategory() == ctgLu  # 'L'etter, 'u'ppercase\nassert Rune('A'.ord).unicodeCategory() in ctgLm+ctgLo+ctgLu+ctgLl+ctgLt\nassert Rune('A'.ord).unicodeCategory() in ctgL\n\necho Rune(0x0660).bidirectional() # 'A'rabic, 'N'umber\n# \"AN\"\n\necho Rune(0x860).combining()\n# 0\n\necho nfcQcNo in Rune(0x0374).quickCheck()\n# true\n```\n[docs](https://nitely.github.io/nim-unicodedb/unicodedb/properties.html)\n\n### Names\n\n```nim\nimport unicode\nimport unicodedb/names\n\necho lookupStrict(\"LEFT CURLY BRACKET\")  # '{'\n# Rune(0x007B)\n\necho \"/\".runeAt(0).name()\n# \"SOLIDUS\"\n```\n[docs](https://nitely.github.io/nim-unicodedb/unicodedb/names.html)\n\n### Compositions\n\n```nim\nimport unicode\nimport unicodedb/compositions\n\necho composition(Rune(108), Rune(803))\n# Rune(7735)\n```\n[docs](https://nitely.github.io/nim-unicodedb/unicodedb/compositions.html)\n\n### Decompositions\n\n```nim\nimport unicode\nimport unicodedb/decompositions\n\necho Rune(0x0F9D).decomposition()\n# @[Rune(0x0F9C), Rune(0x0FB7)]\n```\n[docs](https://nitely.github.io/nim-unicodedb/unicodedb/decompositions.html)\n\n### Types\n\n```nim\nimport unicode\nimport unicodedb/types\n\nassert utmDecimal in Rune(0x0030).unicodeTypes()\nassert utmDigit in Rune(0x00B2).unicodeTypes()\nassert utmNumeric in Rune(0x2CFD).unicodeTypes()\nassert utmLowercase in Rune(0x1E69).unicodeTypes()\nassert utmUppercase in Rune(0x0041).unicodeTypes()\nassert utmCased in Rune(0x0041).unicodeTypes()\nassert utmWhiteSpace in Rune(0x0009).unicodeTypes()\nassert utmWord in Rune(0x1E69).unicodeTypes()\n\nconst alphaNumeric = utmLowercase + utmUppercase + utmNumeric\nassert alphaNumeric in Rune(0x2CFD).unicodeTypes()\nassert alphaNumeric in Rune(0x1E69).unicodeTypes()\nassert alphaNumeric in Rune(0x0041).unicodeTypes()\n```\n[docs](https://nitely.github.io/nim-unicodedb/unicodedb/types.html)\n\n### Widths\n\n```nim\nimport unicode\nimport unicodedb/widths\n\nassert \"🕺\".runeAt(0).unicodeWidth() == uwdtWide\n```\n[docs](https://nitely.github.io/nim-unicodedb/unicodedb/widths.html)\n\n### Scripts\n\n```nim\nimport unicode\nimport unicodedb/scripts\n\nassert \"諸\".runeAt(0).unicodeScript() == sptHan\n```\n[docs](https://nitely.github.io/nim-unicodedb/unicodedb/scripts.html)\n\n### Casing\n\n```nim\nimport sequtils\nimport unicode\nimport unicodedb/casing\n\nassert toSeq(\"Ⓗ\".runeAt(0).lowerCase) == @[\"ⓗ\".runeAt(0)]\nassert toSeq(\"İ\".runeAt(0).lowerCase) == @[0x0069.Rune, 0x0307.Rune]\n\nassert toSeq(\"ⓗ\".runeAt(0).upperCase) == @[\"Ⓗ\".runeAt(0)]\nassert toSeq(\"ffi\".runeAt(0).upperCase) == @['F'.ord.Rune, 'F'.ord.Rune, 'I'.ord.Rune]\n\nassert toSeq(\"ß\".runeAt(0).titleCase) == @['S'.ord.Rune, 's'.ord.Rune]\n\nassert toSeq(\"ᾈ\".runeAt(0).caseFold) == @[\"ἀ\".runeAt(0), \"ι\".runeAt(0)]\n```\n[docs](https://nitely.github.io/nim-unicodedb/unicodedb/casing.html)\n\n### Segmentation\n\n```nim\nimport unicode\nimport unicodedb/segmentation\n\nassert 0x000B.Rune.wordBreakProp == sgwNewline\n```\n[docs](https://nitely.github.io/nim-unicodedb/unicodedb/segmentation.html)\n\n## Related libraries\n\n* [nim-unicodeplus](https://github.com/nitely/nim-unicodeplus)\n* [nim-graphemes](https://github.com/nitely/nim-graphemes)\n* [nim-segmentation](https://github.com/nitely/nim-segmentation)\n* [nim-normalize](https://github.com/nitely/nim-normalize)\n\n## Storage\n\nStorage is based on *multi-stage tables* and\n*minimal perfect hashing* data-structures.\n\n## Sizes\n\nThese are the current collections sizes:\n\n* properties is 40KB. Used by `properties(1)`, `category(1)`,\n  `bidirectional(1)`, `combining(1)` and `quickCheck(1)`\n* compositions is 12KB. Used by: `composition(1)`\n* decompositions is 89KB. Used by `decomposition(1)`\n  and `canonicalDecomposition(1)`\n* names is 578KB. Used by `name(1)` and `lookupStrict(1)`\n* names (lookup) is 241KB. Used by `lookupStrict(1)`\n\n## Missing APIs\n\nNew APIs will be added from time to time. If you need\nsomething that's missing, please open an issue or PR\n(please, do mention the use-case).\n\n## Upgrading Unicode version\n\n\u003e Note: PR's upgrading the unicode version\n\u003e won't get merged, open an issue instead!\n\n* Run `nimble gen` to check there are no changes\n  to `./src/*_data.nim`. If there are try an older\n  Nim version and fix the generators accordingly\n* Run `nimble gen_tests` to update all test data to current\n  unicode version. The tests for a new unicode version run\n  against the previous unicode version.\n* Run `nimble test` and fix all failing tests. This should\n  require just temporarily commenting out\n  all checks for missing unicode points.\n* Overwrite `./gen/UCD` data with\n  [latest unicode UCD](https://unicode.org/Public/UCD/latest/ucd/UCD.zip).\n* Run `nimble gen` to generate the new data.\n* Run `nimble test`. Add checks for missing unicode points back.\n  A handful of unicode points may have change its data, check\n  the unicode changelog page, make sure they are correct and skip them.\n* Note: starting Unicode 15 they added multiple @missing lines\n  which breaks the assumption of a default prop for missing CPs\n  and these lines need to be parsed (see DerivedBidiClass for example).\n  So if they add this to more files, the data gen needs fixing.\n  Look for lines containing `# @missing` with a range other than `0000..10FFFF`. See [Missing_Conventions](https://www.unicode.org/reports/tr44/tr44-30.html#Missing_Conventions)\n\n## Tests\n\nInitial tests were ran against [a dump of] Python's\n`unicodedata` module to ensure correctness.\nAlso, the related libraries have their own custom tests\n(some of the test data is provided by the unicode consortium).\n\n```\nnimble test\n```\n\n## Contributing\n\nI plan to work on most missing *related\nlibraries* (case folding, etc). If you would\nlike to work in one of those, please let me\nknow and I'll add it to the list. If you find\nthe required database data is missing, either open an\nissue or a PR.\n\n## LICENSE\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnitely%2Fnim-unicodedb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnitely%2Fnim-unicodedb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnitely%2Fnim-unicodedb/lists"}