{"id":51552817,"url":"https://github.com/akesson/wordtree","last_synced_at":"2026-07-10T01:01:36.510Z","repository":{"id":362674892,"uuid":"1259241964","full_name":"akesson/wordtree","owner":"akesson","description":"A compact trie for browsable word indexes, fast word lookup, and frequency-aware spelling suggestions.","archived":false,"fork":false,"pushed_at":"2026-06-05T11:17:10.000Z","size":4914,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-05T13:07:11.620Z","etag":null,"topics":["autocomplete","data-structures","fuzzy-search","rust","spell-check","trie"],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/akesson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-04T10:13:48.000Z","updated_at":"2026-06-05T11:17:14.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/akesson/wordtree","commit_stats":null,"previous_names":["akesson/wordtree"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/akesson/wordtree","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akesson%2Fwordtree","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akesson%2Fwordtree/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akesson%2Fwordtree/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akesson%2Fwordtree/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/akesson","download_url":"https://codeload.github.com/akesson/wordtree/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akesson%2Fwordtree/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35316840,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-09T02:00:07.329Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autocomplete","data-structures","fuzzy-search","rust","spell-check","trie"],"created_at":"2026-07-10T01:01:35.615Z","updated_at":"2026-07-10T01:01:36.503Z","avatar_url":"https://github.com/akesson.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# wordtree\n\n\u003e The `wordtree` crate was extracted from a larger private project to accompany an\n\u003e article.\n\nEverything a word-list UI needs — typo-tolerant as-you-type suggestions, exact\nlookup, and a browsable A–Z index — served from one small file that loads by\nmemory-mapping, with no parsing or index-building at startup.\n\nType `appl` into a search box backed by wordtree and you get `apple`, `apply`\nand `application` back; mistype it as `aple` and `apple` still tops the list.\nCompletions and one-edit spelling fixes come merged in a single frequency-ranked\nlist, in tens of microseconds per keystroke against a 638,000-word dictionary.\nThat is the crate's job — shipping a large word list inside an app (a dictionary\nor translation app, a keyboard, a command palette) and answering three questions\nabout it:\n\n- **What is the user trying to type?** `suggestions()` completes the prefix *and*\n  forgives one mistake — a wrong, missing, extra or swapped character — ranking\n  results by word frequency so common words surface first. `completions()` and\n  `corrections()` give either half alone, and the `Caps` budget turns the short\n  as-you-type list into an exhaustive spell-check when you need every candidate.\n- **What does this word point to?** Every word carries a numeric index you assign\n  (`index_of(\"apple\") → 1`), so a match lands directly in your own table of\n  definitions, translations or products — no second lookup structure.\n- **What's in the list?** Words are grouped into folders of ~100 (`path_of`), so a\n  UI can offer a drill-down index for browsing, not just a search box.\n\nIt also ships well: the whole structure — 638k English words in 21 MiB, 113k\nSwedish in 4.4 MiB — serialises with [`rkyv`](https://rkyv.org), and its on-disk\nform *is* its in-memory form. Loading is an mmap; there is nothing to parse or\nrebuild, which matters on phones and other memory-tight, cold-start-sensitive\ntargets.\n\nThe [deep dive](#word-tree) below explains the data structure and edit-distance\ndesign; [`comparisons/REPORT.md`](comparisons/REPORT.md) races it honestly\nagainst the specialist crates (fst, symspell, pruning_radix_trie) — each beats\nwordtree on its own axis; wordtree's case is all three jobs from one file.\n\n## Usage\n\n```rust\nuse wordtree::{Builder, Caps, TreeFn};\n\n// Build a tree from (word, percentile, expr_index) entries.\nlet mut builder = Builder::new();\nbuilder.add_word(\"apple\", 99, 1);\nbuilder.add_word(\"apply\", 80, 2);\nbuilder.add_word(\"apricot\", 50, 3);\nbuilder.organize_into_folders(100); // ~100 words per browsable folder\nlet tree = builder.to_tree();\n\n// Exact word -\u003e expression-index lookup.\nassert_eq!(tree.index_of(\"apple\"), Some(1));\n\n// Browsable folder path for a word.\nlet _path = tree.path_of(\"apricot\");\n\n// Typo-tolerant autocomplete for a (mis)typed query; the closure filters\n// which expression indices are acceptable candidates. \"aple\" is a deletion typo\n// of \"apple\"; length-changing edits (indels) within edit distance 1 are\n// corrected, so \"apple\" (expr_index 1) is among the suggestions.\nlet suggestions = tree.suggestions(\"aple\", |_expr_index| true);\nassert!(suggestions.iter().any(|s| s.expr_index == 1));\n\n// Need only one half? `completions()` is autocomplete only (no edit-distance walk,\n// much cheaper); `corrections()` is spelling correction only (no prefix completion).\nlet _completions = tree.completions(\"ap\", |_| true);   // words extending \"ap\"\nlet _corrections = tree.corrections(\"aple\", |_| true); // spelling corrections only\n\n// The default list is deliberately short (good as-you-type behaviour). For\n// exhaustive spell-check — the *complete* set of words within edit distance 1,\n// including when the query is itself a valid word — widen the per-query budget\n// with `Caps`. `Caps::uniform(64)` recalls 100% of the brute-force DL≤1 set.\nlet _all = tree.corrections_with(\"aple\", |_| true, Caps::uniform(64));\n```\n\n`to_tree()` returns a `Tree` that implements `TreeFn`. After `rkyv`-serialising it, the\nsame query methods are available zero-copy on `ArchivedTree`. The per-query suggestion\nbudget is configurable via `Caps` — see [`comparisons/REPORT.md`](comparisons/REPORT.md)\n§1 for the recall trade-off behind the default.\n\n## License\n\nLicensed under either of\n\n- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE))\n- MIT license ([LICENSE-MIT](LICENSE-MIT))\n\nat your option. The vendored [`ego-tree`](ego-tree/) crate is licensed separately under\nISC (see [ego-tree/LICENSE](ego-tree/LICENSE)).\n\n## Data\n\nThe bundled benchmark and test word lists are derived from PanLex and Wiktionary — see\n[DATA-LICENSE](DATA-LICENSE) for attribution and terms.\n\n---\n\n## Contents\n\n- [Word Tree](#word-tree)\n  - [Uses](#uses)\n  - [Node layout](#node-layout)\n  - [Edit distance](#edit-distance)\n  - [Benchmarks](#benchmarks)\n\n# Word Tree\n\nA custom trie that stores words as a tree of chars, size-optimised into a flat\narray of fixed-width nodes. Each node is followed by all its siblings, so the\ntree is laid out width-first. Some nodes are marked as folders such that, where\npossible, at most ~100 words land in each folder.\n\n## Uses\n\nThe three jobs above map onto the structure: folder-marked nodes drive the browsable\nindex, the char-path to a node yields its expression index, and a frequency-ranked walk\nproduces typo-tolerant autocomplete.\n\nThe design targets alphabetic scripts (tens of distinct characters per language);\nlogographic scripts (thousands of distinct characters) are out of scope.\n\n## Node layout\n\nEach node is a fixed 8-byte (64-bit) record, so a tree read from disk needs no\ntype-casting or transformation (zero-copy mmap):\n\n| field                | bits   | comment                                                              |\n| -------------------- | ------ | -------------------------------------------------------------------- |\n| first_child_pos      | 24     | relative position of the first child                                 |\n| node_char            | 24     | UTF-32 codepoint (low 3 bytes used)                                  |\n| is_folder            | 1      |                                                                      |\n| is_last_sibling      | 1      |                                                                      |\n| max_child_percentile | 10     | max percentile in the subtree — drives the pruning walk (below)      |\n| (spare)              | 4      |                                                                      |\n| **TOT**              | **64** | = 8 bytes                                                            |\n\n`max_child_percentile` powers the [pruning-radix-trie](https://towardsdatascience.com/the-pruning-radix-trie-a-radix-trie-on-steroids-412807f77abc)\nwalk: a subtree whose best percentile can't beat the current top-k is skipped. It stays\ninline because it is read on *every* node during the walk.\n\n### Per-word side tables\n\nA word's own `percentile` (frequency, 0–1000) and 24-bit `expr_index` only exist on\nnodes that terminate a word — about a quarter of all nodes — so storing them inline\nwould waste 5 bytes on every internal node. Instead they live in three side arrays,\nalso part of the zero-copy mmap:\n\n| table        | size                          | role                                                      |\n| ------------ | ----------------------------- | --------------------------------------------------------- |\n| `word_bits`  | 1 bit / node                  | marks which nodes are words                                |\n| `rank_index` | 1× u32 / 64-bit word          | cumulative word count — answers `rank(node) → slot` in O(1)|\n| `values`     | 5 bytes / **word** (u16 + u24)| the `(percentile, expr_index)` pair for each word         |\n\nA word node at position `i` finds its value at `values[rank(i)]` — the number of word\nnodes before it. The bit probe is on the hot browse/descent path; the rank query only\nfires when a per-word value is actually consumed (an exact `index_of`, or a kept\nsuggestion). Storing the pair inline would need 12-byte nodes (~26.5 MiB for English);\nthe 8-byte nodes plus side tables come to ~21.1 MiB, ~20% smaller with no loss of\nfunction — see [`comparisons/REPORT.md`](comparisons/REPORT.md) §2.\n\n## Edit distance\n\nAn edit distance of 1 — counting transposition (swapping two adjacent chars) alongside substitution, insertion and deletion — is sufficient for as-you-type correction.\n\nIt is evaluated incrementally as the trie is walked: one small dynamic-programming\nDamerau-Levenshtein row per node — stored as a fixed `2K+1`-cell diagonal *band*\n(3 cells at edit distance `K=1`) rather than the full row — with a subtree pruned as\nsoon as its band minimum is out of range. This handles all four edit kinds uniformly\nand visits well under 1% of the tree, without a full edit-distance matrix. The\nband keeps each visited node at `O(K)` work instead of `O(query length)`; results are\nidentical to a full-row walk. See [src/editdist/README.md](src/editdist/README.md)\n\n## Benchmarks\n\nBuilding the full tree from the bundled TSV takes roughly 435 ms (English, ~638k words)\nand 71 ms (Swedish, ~113k words) on an Apple M4 Pro. For comparative latency, size and\nsuggestion-quality benchmarks against specialist crates (fst, symspell,\npruning_radix_trie, boomphf), see [`comparisons/REPORT.md`](comparisons/REPORT.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakesson%2Fwordtree","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fakesson%2Fwordtree","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakesson%2Fwordtree/lists"}