{"id":37089329,"url":"https://github.com/bastiangx/wordserve","last_synced_at":"2026-01-14T10:57:28.566Z","repository":{"id":304187347,"uuid":"938254112","full_name":"bastiangx/wordserve","owner":"bastiangx","description":"Prefix completion engine, designed for any msgpack clients in need of a fast and minimal Go server!","archived":false,"fork":false,"pushed_at":"2025-08-24T12:17:08.000Z","size":8041,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-24T17:30:24.543Z","etag":null,"topics":["autocomplete","client-server","golang","ipc","messagepack","msgpack","prefix","words"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/bastiangx/wordserve","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bastiangx.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":".github/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"ko_fi":"bastiangx"}},"created_at":"2025-02-24T16:59:16.000Z","updated_at":"2025-08-24T12:14:56.000Z","dependencies_parsed_at":"2025-08-17T15:07:55.391Z","dependency_job_id":"931fd071-6968-49ea-9f3c-4a938e96b29d","html_url":"https://github.com/bastiangx/wordserve","commit_stats":null,"previous_names":["bastiangx/wordserve"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/bastiangx/wordserve","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bastiangx%2Fwordserve","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bastiangx%2Fwordserve/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bastiangx%2Fwordserve/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bastiangx%2Fwordserve/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bastiangx","download_url":"https://codeload.github.com/bastiangx/wordserve/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bastiangx%2Fwordserve/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28417777,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T10:47:48.104Z","status":"ssl_error","status_checked_at":"2026-01-14T10:46:19.031Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autocomplete","client-server","golang","ipc","messagepack","msgpack","prefix","words"],"created_at":"2026-01-14T10:57:27.621Z","updated_at":"2026-01-14T10:57:28.558Z","avatar_url":"https://github.com/bastiangx.png","language":"Go","funding_links":["https://ko-fi.com/bastiangx"],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003ca href=\"https://github.com/bastiangx/wordserve\"\u003e\n \u003cpicture\u003e\n      \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://files.catbox.moe/sa5maz.png\"\u003e\n      \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://files.catbox.moe/hy2y82.png\"\u003e\n      \u003cimg src=\"https://files.catbox.moe/hy2y82.png\"/\u003e\n    \u003c/picture\u003e\n  \u003c/a\u003e\n\u003c/h1\u003e\n\n\u003cdiv align=\"center\"\u003e\nLightweight prefix completion library | server, designed for any MessagePack clients\n\u003cbr /\u003e\n\u003cbr /\u003e\n\u003cdiv align=\"center\"\u003e\n    \u003cpicture\u003e\n      \u003csource srcset=\"https://github.com/user-attachments/assets/0da6f300-0711-4f85-85c4-6a19c22a7f75\" /\u003e\n      \u003cimg src=\"https://github.com/user-attachments/assets/0da6f300-0711-4f85-85c4-6a19c22a7f75\" alt=\"Example usage of wordserve suggestions engine in a client app\" /\u003e\n    \u003c/picture\u003e\n\u003c/div\u003e\n\n\u003cbr /\u003e\n\u003ca href=\"https://pkg.go.dev/github.com/bastiangx/wordserve\"\u003e\u003cimg src=\"https://img.shields.io/badge/reference-black?style=for-the-badge\u0026logo=go\u0026logoSize=auto\u0026labelColor=%23363A4F\u0026color=%237dc4e4\" alt=\"Go Reference\"\u003e\u003c/a\u003e \u003ca href=\"https://goreportcard.com/report/github.com/bastiangx/wordserve\"\u003e\u003cimg src=\"https://img.shields.io/badge/A%2B-black?style=for-the-badge\u0026logoSize=auto\u0026label=go%20report\u0026labelColor=%23363A4F\u0026color=%23a6da95\" alt=\"Go Report Card\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/bastiangx/wordserve/releases/latest\"\u003e\u003cimg src=\"https://img.shields.io/github/v/release/bastiangx/wordserve?sort=semver\u0026display_name=tag\u0026style=for-the-badge\u0026labelColor=%23363A4F\u0026color=%23b7bdf8\" alt=\"Latest Release\"\u003e\u003c/a\u003e \u003ca href=\"https://github.com/bastiangx/wordserve/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/MIT-black?style=for-the-badge\u0026label=license\u0026labelColor=%23363A4F\u0026color=%23b7bdf8\" alt=\"MIT License\"\u003e\u003c/a\u003e\n\u003cbr /\u003e \u003ca href=\"https://addons.mozilla.org/en-US/firefox/addon/wordserve/\"\u003e\u003cimg src=\"https://img.shields.io/badge/firefox_extension-black?style=for-the-badge\u0026logo=firefox\u0026logoSize=auto\u0026labelColor=%23363A4F\u0026color=%23ef9f76\" alt=\"Firefox addons extension badge\"\u003e\u003c/a\u003e\n\n\u003cbr /\u003e\n  \u003ca href=\"https://github.com/bastiangx/wordserve/issues/new?assignees=\u0026labels=bug\u0026template=BUG-REPORT.yml\u0026title=%5BBug%5D%3A+\"\u003eReport a Bug\u003c/a\u003e\n  ·\n  \u003ca href=\"https://github.com/bastiangx/wordserve/issues/new?assignees=\u0026labels=enhancement\u0026template=FEATURE-REQUEST.yml\u0026title=%5BFeature%5D%3A+\"\u003eRequest a Feature\u003c/a\u003e\n\u003c/div\u003e\n\n#### What's it about?\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd\u003e\n\nWordServe is a minimalistic and high performance **prefix completion library** with a server executable written in Go.\n\nIts designed to provide auto-completion for various clients, especially those using [MessagePack](https://msgpack.org/index.html) as a serialization format.\n\n#### Why?\n\nSo many tools and apps I use on daily basis do not offer any form of word completion, AI/NLP driven or otherwise, there are times when I need to quickly find a word or phrase that I know exists in my vocabulary, but I have no idea how to spell it or don't feel like typing for _that_ long.\n\nWhy not make my own tool that can power any TS/JS/etc clients with a completion server?\n\n#### Similar to?\n\nThink of this as a elementary nvim-cmp or vscode Intellisense daemon, but for any plugin/app that can use a MessagePack client. (which is super [easy to implement](https://www.npmjs.com/package/@msgpack/msgpack) and use compared to JSON parsing btw, in fact, about **411%** [improvement in speed](https://halilibrahimkocaoz.medium.com/message-queues-messagepack-vs-json-for-serialization-749914e3d0bb) and **40%** reduction in payload sizes)\n\n\u003e This is my first attempt on creating a small scaled but usable Go server/library. Expect unstable or incomplete features, as well as some bugs.\n\u003e I primarily made this for myself so I can make a completion plugin for [Obsidian](https://obsidian.md) but hey, you might find it useful too!\n\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n### Prerequisites\n\n- [Go 1.22](https://go.dev/doc/install) or later\n- [Luajit 2.1](https://luajit.org/install.html) _(only for dictionary build scripts)_\n  - A simple `words.txt` file for building the dictionary with most used words and their corresponding frequencies \u003cspan style=\"color: #908caa;\"\u003e -- see [dictionary](#dictionary) for more info\u003c/span\u003e\n\n## Installation\n\n### Go\n\nusing `go install` _(Recommended)_:\n\n```sh\ngo install github.com/bastiangx/wordserve/cmd/wordserve@latest\n```\n\n#### Library Dependency\n\nuse `go get` to add `wordserve` as a dependency in your project:\n\n```sh\ngo get github.com/bastiangx/wordserve\n```\n\nand then import it in your code:\n\n```go\nimport \"github.com/bastiangx/wordserve/pkg/suggest\"\n```\n\n### Releases\n\nDownload the latest precompiled binaries from the [releases page](https://github.com/bastiangx/wordserve/releases/latest).\n\n- `wordserve` automatically downloads and initializes the needed dictionary files from GitHub releases\n- The dictionary files (`dict_*.bin`) are packaged in `data.zip` and the word list (`words.txt`) is available as a separate download\n- If automatic download fails, you can manually download `data.zip` and `words.txt` from the [releases page](https://github.com/bastiangx/wordserve/releases/latest) and extract them to the `data/` directory\n\n\u003e If you're not sure, use 'go install'.\n\n### Building from source\n\nYou can also clone via git and build the old fashioned way:\n\n```sh\ngit clone https://github.com/bastiangx/wordserve.git\ncd wordserve\n# -w -s strips debug info \u0026 symbols | alias wserve\ngo build -ldflags=\"-w -s\" -o wserve ./cmd/wordserve/main.go\n```\n\nThe build process for the dict files is handled by the `wordserve` binary, If you encounter any issues, you can manually run the build script located in `scripts/build-data.lua` using [LuaJIT](https://luajit.org/).\n\n\u003e Make sure the `data/` directory exists and has the `words.txt` file in it before running this.\n\n```sh\nluajit scripts/build-data.lua\n```\n\n## What can it do?\n\n### Batched Word Suggestions\n\n\u003cdiv align=\"center\"\u003e\n \u003cpicture\u003e\n      \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://files.catbox.moe/sd3ikj.png\"\u003e\n      \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://files.catbox.moe/h26n6q.png\"\u003e\n      \u003cimg src=\"https://files.catbox.moe/h26n6q.png\" width=\"550\" /\u003e\n    \u003c/picture\u003e\n  \u003c/div\u003e\n\n\u003cbr /\u003e\n\nWordServe returns suggestions in batches using a radix trie. Memory pools handle rapid queries without triggering garbage collection.\n\n\u003cbr /\u003e\n\n### Responsive\n\n\u003cdiv align=\"center\"\u003e\n \u003cpicture\u003e\n      \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://files.catbox.moe/ca82mt.png\"\u003e\n      \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://files.catbox.moe/8emcdr.png\"\u003e\n      \u003cimg src=\"https://files.catbox.moe/8emcdr.png\" width=\"550\" /\u003e\n    \u003c/picture\u003e\n  \u003c/div\u003e\n\n\u003cbr /\u003e\n\nThe IPC server communicates through stdin/stdout channels with minimal protocol overhead.\n\n- Goroutines handle multiple client connections simultaneously.\n\n\u003cbr /\u003e\n\n### Capital letters\n\n\u003cpicture\u003e\n      \u003csource srcset=\"https://files.catbox.moe/69eg4f.gif\" /\u003e\n      \u003cimg src=\"https://files.catbox.moe/69eg4f.gif\" alt=\"a gif video showing wordserve suggestions engine handling capital letters properly\" /\u003e\n    \u003c/picture\u003e\n\u003cbr /\u003e\n\nIt just works!\n\n### Compact MessagePack Protocol\n\n\u003cdiv align=\"center\"\u003e\n \u003cpicture\u003e\n      \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://files.catbox.moe/vlkcqa.png\"\u003e\n      \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://files.catbox.moe/7kwkwk.png\"\u003e\n      \u003cimg src=\"https://files.catbox.moe/7kwkwk.png\" width=\"550\" /\u003e\n    \u003c/picture\u003e\n  \u003c/div\u003e\n\n\u003cbr /\u003e\n\nBinary MessagePack encoding keeps request and response payloads as small as possible.\n\n\u003cbr /\u003e\n\n### Many Many Words\n\n\u003cdiv align=\"center\"\u003e\n \u003cpicture\u003e\n      \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://files.catbox.moe/z463kh.png\"\u003e\n      \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://files.catbox.moe/w4cn0v.png\"\u003e\n      \u003cimg src=\"https://files.catbox.moe/w4cn0v.png\" width=\"550\" /\u003e\n    \u003c/picture\u003e\n  \u003c/div\u003e\n\u003cbr /\u003e\n\nStart with a simple `words.txt` file containing 65,000+ entries.\n\n- WordServe chunks the dictionary into binary trie files and loads only what's needed, dynamically managing memory based on usage patterns.\n\n\u003cbr /\u003e\n\n### Small memory usage\n\n\u003cpicture\u003e\n      \u003csource srcset=\"https://files.catbox.moe/nv7r2x.gif\" /\u003e\n      \u003cimg src=\"https://files.catbox.moe/nv7r2x.gif\" alt=\"Memory usage of WordServe shown to be around 20MB with 50K words loaded in\" /\u003e\n    \u003c/picture\u003e\n\u003cbr /\u003e\n\nWordServe's memory usage remains low even with large dictionaries, typically around 20MB for 50,000 words default.\nEven after expanding many nodes and normal usage for few hours, it stays under 60MB and has checks to shrink periodically.\n\n## What can it _not_ do?\n\nAs this is the early version and Beta, there are _many_ features that are yet not implemented\n\n- simple fuzzy matching\n- string searching algo (haystack-needle)\n- integrated spelling correction (aspell)\n- support conventional dict formats like `.dict`\n\nWill monitor the issues and usage to see if enough people are interested.\n\n\n## Usage\n\n### Standalone server\n\nyou can run `wordserve` as a dependency in your Go project, a standalone IPC server.\nA simple CLI is also provided for testing and debugging.\n\n### Library\n\nThe library provides simple to use API for prefix completion requests and dictionary management.\n\nRead all about using them in the [API doc](docs/api.md)\n\nMore comprehensive and verbose [Go Package docs](https://pkg.go.dev/github.com/bastiangx/wordserve/pkg/suggest)\n\n```go\ncompleter := suggest.NewLazyCompleter(\"./data\", 10000, 50000)\n\nif err := completer.Initialize(); err != nil {\n    log.Fatalf(\"Failed to initialize: %v\", err)\n}\n\nsuggestions := completer.Complete(\"amer\", 10)\n```\n\nor for static check:\n\n```go\ncompleter := suggest.NewCompleter()\n\ncompleter.AddWord(\"example\", 500)\ncompleter.AddWord(\"excellent\", 400)\n\nsuggestions := completer.Complete(\"ex\", 5)\n```\n\n\u003e You can inspect the _informal_ flow diagram on the core internals:\n\n\u003ca href=\"https://files.catbox.moe/6wy79k.png\"\u003e\n\u003cimg src=\"https://files.catbox.moe/6wy79k.png\" alt=\"A flow diagram of WordServe's core internals\"\u003e\n\u003c/a\u003e\n\n### Client Integration\n\nThe [Client doc](docs/client.md) gives some guide on how to use WordServe in your TS/JS app.\n\n```ts\nimport { spawn, ChildProcess } from \"child_process\";\nimport { encode, decode } from \"@msgpack/msgpack\";\n\nclass WordServeClient {\n  private process: ChildProcess;\n  private requestId = 0;\n\n  constructor(binaryPath: string = \"wordserve\") {\n    this.process = spawn(binaryPath, [], {\n      stdio: [\"pipe\", \"pipe\", \"pipe\"],\n    });\n  }\n\n  async getCompletions(\n    prefix: string,\n    limit: number = 20,\n  ): Promise\u003cSuggestion[]\u003e {\n    const request = {\n      id: `req_${++this.requestId}`,\n      p: prefix,\n      l: limit, // (optional)\n    };\n\n    const binaryRequest = encode(request);\n    this.process.stdin!.write(binaryRequest);\n\n    return new Promise((resolve, reject) =\u003e {\n      this.process.stdout!.once(\"data\", (data: Buffer) =\u003e {\n        try {\n          const response = decode(data) as CompletionResponse;\n          const suggestions = response.s.map((s, index) =\u003e ({\n            word: s.w,\n            rank: s.r,\n            frequency: 65536 - s.r, // Convert rank back to freq score\n          }));\n          resolve(suggestions);\n        } catch (error) {\n          reject(error);\n        }\n      });\n    });\n  }\n}\n```\n\n### CLI\n\nLearn how to use it in the [CLI doc](docs/cli.md)\n\n\u003cpicture\u003e\n      \u003csource srcset=\"https://files.catbox.moe/aoa2s1.gif\" /\u003e\n      \u003cimg src=\"https://files.catbox.moe/aoa2s1.gif\" alt=\"WordServe CLI in action\" /\u003e\n    \u003c/picture\u003e\n\n##### Flags\n\n```sh\nwordserve [flags]\n```\n\n| Flag       | Description                                                                                   | Default Value |\n| :--------- | :-------------------------------------------------------------------------------------------- | :-----------: |\n| -version   | Show current version                                                                          |     false     |\n| -config    | Path to custom config.toml file                                                               |      \"\"       |\n| -data      | Directory containing the binary files                                                         |    \"data/\"    |\n| -v         | Toggle verbose mode                                                                           |     false     |\n| -c         | Run CLI -- useful for testing and debugging                                                   |     false     |\n| -limit     | Number of suggestions to return                                                               |      10       |\n| -prmin     | Minimum Prefix length for suggestions (1 \u003c n \u003c= prmax)                                        |       3       |\n| -prmax     | Maximum Prefix length for suggestions                                                         |      24       |\n| -no-filter | Disable input filtering (DBG only) - shows all raw dictionary entries (numbers, symbols, etc) |     false     |\n| -words     | Maximum number of words to load (use 0 for all words)                                         |    100,000    |\n| -chunk     | Number of words per chunk for lazy loading                                                    |    10,000     |\n\n## Dictionary\n\nRead more about the [dictionary design](docs/dictionary.md) and how it works.\n\n## Configuration\n\nRefer to the [config doc](docs/config.md) on how to manage server, send commands to it and change dictionary on runtime.\n\n## Development\n\nSee the [open issues](https://github.com/bastiangx/wordserve/issues) for a list of proposed features (and known issues).\n\nContributions are welcome! Refer to the [contributing guidelines](.github/CONTRIBUTING.md)\n\n## License\n\nWordServe is licensed under the **MIT license**.\nFeel free to edit and distribute this library as you like.\n\nSee [LICENSE](LICENSE)\n\n---\n\n#### Acknowledgements\n\n- The _Beautiful_ [Rosepine theme](https://rosepinetheme.com/) used for graphics and screenshots throughout the readme.\n- The Incredible mono font, Berkeley Mono by [U.S. Graphics](https://usgraphics.com/products/berkeley-mono) used in screenshots, graphics, gifs and more.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbastiangx%2Fwordserve","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbastiangx%2Fwordserve","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbastiangx%2Fwordserve/lists"}