{"id":20447822,"url":"https://github.com/nlpodyssey/verbaflow","last_synced_at":"2025-07-06T01:04:08.224Z","repository":{"id":65536827,"uuid":"593672071","full_name":"nlpodyssey/verbaflow","owner":"nlpodyssey","description":"Neural Language Model for Go","archived":false,"fork":false,"pushed_at":"2023-06-30T12:34:25.000Z","size":153,"stargazers_count":59,"open_issues_count":4,"forks_count":5,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-13T01:11:28.596Z","etag":null,"topics":["chatbot","deep-learning","language-model","recurrent-neural-networks","rwkv"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nlpodyssey.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-26T15:27:22.000Z","updated_at":"2025-02-02T23:16:22.000Z","dependencies_parsed_at":"2024-06-21T14:15:21.120Z","dependency_job_id":"d47cb317-1eb9-435f-a204-14840cfc6888","html_url":"https://github.com/nlpodyssey/verbaflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nlpodyssey/verbaflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nlpodyssey%2Fverbaflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nlpodyssey%2Fverbaflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nlpodyssey%2Fverbaflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nlpodyssey%2Fverbaflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nlpodyssey","download_url":"https://codeload.github.com/nlpodyssey/verbaflow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nlpodyssey%2Fverbaflow/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263833416,"owners_count":23517373,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","deep-learning","language-model","recurrent-neural-networks","rwkv"],"created_at":"2024-11-15T10:29:43.051Z","updated_at":"2025-07-06T01:04:08.026Z","avatar_url":"https://github.com/nlpodyssey.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# VerbaFlow\n\nWelcome to VerbaFlow, a neural architecture written in Go designed specifically for language modeling tasks. \nBuilt on the robust RWKV RNN, this model is optimized for efficient performance on standard CPUs, enabling smooth running of relatively large language models even on consumer hardware.\n\nWith the ability to utilize pretrained models on the [Pile](https://arxiv.org/abs/2101.00027) dataset, VerbaFlow performs comparably to GPT-like Transformer models in predicting the next token, as well as in other tasks such as text summarization, text classification, question answering, and general conversation. \n\n# Installation\n\nRequirements:\n\n* [Go 1.20](https://golang.org/dl/)\n\nClone this repo or get the library:\n\n```console\ngo get -u github.com/nlpodyssey/verbaflow\n```\n\n# Usage\n\nTo start using VerbaFlow, we recommend using the pre-trained model `RWKV-4-Pile-1B5-Instruct`, available on the [Hugging Face Hub](https://huggingface.co/nlpodyssey/RWKV-4-Pile-1B5-Instruct).\nThis model has been fine-tuned using the [Pile](https://huggingface.co/datasets/the_pile) dataset and has been specially designed to understand and execute human instructions, as fine-tuned on the [xP3](https://huggingface.co/datasets/bigscience/xP3all) dataset. \nThe original `RWKV-4-Pile-1B5-Instruct-test2-20230209` model, from which this model is derived, can be accessed [here](https://huggingface.co/BlinkDL/rwkv-4-pile-1b5).\n\n\u003e The library is optimized to run in x86-64 CPUs. If you want to run it on a different architecture, you can use the `GOARCH=amd64` environment variable.\n\nThe following commands can be used to build and use VerbaFlow:\n\n```console\ngo build ./cmd/verbaflow\n```\n\nThis command builds the go program and creates an executable named `verbaflow`.\n\n```console\n./verbaflow -model-dir models/nlpodyssey/RWKV-4-Pile-1B5-Instruct download\n```\n\nThis command downloads the model specified (in this case, \"nlpodyssey/RWKV-4-Pile-1B5-Instruct\" under the \"models\" directory)\n\n```console\n./verbaflow -model-dir models/nlpodyssey/RWKV-4-Pile-1B5-Instruct convert\n```\n\nThis command converts the downloaded model to the format used by the program.\n\n```console\n./verbaflow -log-level trace -model-dir models/nlpodyssey/RWKV-4-Pile-1B5-Instruct inference --address :50051\n```\n\nThis command runs the gRPC inference endpoint on the specified model.\n\nPlease make sure to have the necessary dependencies installed before running the above commands.\n\n## Examples\n\nOne of the most interesting features of the LLM is the ability to react based on the prompt.\n\nRun the `verbaflow` gRPC endpoint with the command in inference, then run the `prompttester` example entering the following prompts:\n\n### Example 1\n\nPrompt:\n\n```console\necho '\\nQ: Briefly: The Universe is expanding, its constituent galaxies flying apart like pieces of cosmic shrapnel in the aftermath of the Big Bang. Which section of a newspaper would this article likely appear in?\\n\\nA:' | go run ./examples/prompttester --dconfig ./examples/prompttester/config.yaml\n```\n\nExpected output:\n\n```console\nScience and Technology\n```\n\n### Example 2\n\nPrompt:\n\n```console\necho '\\nQ:Translate the following text from French to English Je suis le père le plus heureux du monde\\n\\nA:' | go run ./examples/prompttester --dconfig ./examples/prompttester/config.yaml\n```\n\nExpected output:\n\n```console\nI am the happiest father in the world.\n```\n\n## Dependencies\n\nA list of the main dependencies follows:\n\n- [Spago](http://github.com/nlpodyssey/spago) - Machine Learning framework\n- [RWKV](http://github.com/nlpodyssey/rwkv) - RWKV RNN implementation\n- [GoTokenizers](http://github.com/nlpodyssey/gotokenizers) - Tokenizers library\n- [GoPickle](http://github.com/nlpodyssey/gopickle) - Pickle library for Go\n\n# Roadmap\n\n- [x] Download pretrained models from the Hugging Face models hub\n- [ ] Effective \"prompts\" catalog\n- [x] Better sampling\n- [ ] Beam search\n- [ ] Better Tokenizer\n- [ ] Unit tests\n- [ ] Code refactoring\n- [ ] Documentation\n- [x] gRPC ~~/HTTP~~ API\n\n# Credits\n\nThanks [PENG Bo](https://github.com/BlinkDL) for creating the RWKV RNN and all related resources, including pre-trained models!\n\n# Trivia about the project's name\n\n\"VerbaFlow\" combines \"verba\", which is the Latin word for *words*, and \"flow\", which alludes to the characteristics of recurrent neural networks by evoking the idea of a fluent and continuous flow of words, which is made possible by the network's ability to maintain an internal state and \"remember\" previous words and context when generating new words.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnlpodyssey%2Fverbaflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnlpodyssey%2Fverbaflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnlpodyssey%2Fverbaflow/lists"}