Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/nlpodyssey/verbaflow
Neural Language Model for Go
https://github.com/nlpodyssey/verbaflow
chatbot deep-learning language-model recurrent-neural-networks rwkv
Last synced: 3 months ago
JSON representation
Neural Language Model for Go
- Host: GitHub
- URL: https://github.com/nlpodyssey/verbaflow
- Owner: nlpodyssey
- License: bsd-2-clause
- Created: 2023-01-26T15:27:22.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2023-06-30T12:34:25.000Z (over 1 year ago)
- Last Synced: 2024-06-21T15:34:04.792Z (7 months ago)
- Topics: chatbot, deep-learning, language-model, recurrent-neural-networks, rwkv
- Language: Go
- Homepage:
- Size: 149 KB
- Stars: 57
- Watchers: 4
- Forks: 5
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# VerbaFlow
Welcome to VerbaFlow, a neural architecture written in Go designed specifically for language modeling tasks.
Built on the robust RWKV RNN, this model is optimized for efficient performance on standard CPUs, enabling smooth running of relatively large language models even on consumer hardware.
With the ability to utilize pretrained models on the [Pile](https://arxiv.org/abs/2101.00027) dataset, VerbaFlow performs comparably to GPT-like Transformer models in predicting the next token, as well as in other tasks such as text summarization, text classification, question answering, and general conversation.
# Installation
Requirements:
* [Go 1.20](https://golang.org/dl/)
Clone this repo or get the library:
```console
go get -u github.com/nlpodyssey/verbaflow
```
# Usage
To start using VerbaFlow, we recommend using the pre-trained model `RWKV-4-Pile-1B5-Instruct`, available on the [Hugging Face Hub](https://huggingface.co/nlpodyssey/RWKV-4-Pile-1B5-Instruct).
This model has been fine-tuned using the [Pile](https://huggingface.co/datasets/the_pile) dataset and has been specially designed to understand and execute human instructions, as fine-tuned on the [xP3](https://huggingface.co/datasets/bigscience/xP3all) dataset.
The original `RWKV-4-Pile-1B5-Instruct-test2-20230209` model, from which this model is derived, can be accessed [here](https://huggingface.co/BlinkDL/rwkv-4-pile-1b5).
> The library is optimized to run in x86-64 CPUs. If you want to run it on a different architecture, you can use the `GOARCH=amd64` environment variable.
The following commands can be used to build and use VerbaFlow:
```console
go build ./cmd/verbaflow
```
This command builds the go program and creates an executable named `verbaflow`.
```console
./verbaflow -model-dir models/nlpodyssey/RWKV-4-Pile-1B5-Instruct download
```
This command downloads the model specified (in this case, "nlpodyssey/RWKV-4-Pile-1B5-Instruct" under the "models" directory)
```console
./verbaflow -model-dir models/nlpodyssey/RWKV-4-Pile-1B5-Instruct convert
```
This command converts the downloaded model to the format used by the program.
```console
./verbaflow -log-level trace -model-dir models/nlpodyssey/RWKV-4-Pile-1B5-Instruct inference --address :50051
```
This command runs the gRPC inference endpoint on the specified model.
Please make sure to have the necessary dependencies installed before running the above commands.
## Examples
One of the most interesting features of the LLM is the ability to react based on the prompt.
Run the `verbaflow` gRPC endpoint with the command in inference, then run the `prompttester` example entering the following prompts:
### Example 1
Prompt:
```console
echo '\nQ: Briefly: The Universe is expanding, its constituent galaxies flying apart like pieces of cosmic shrapnel in the aftermath of the Big Bang. Which section of a newspaper would this article likely appear in?\n\nA:' | go run ./examples/prompttester --dconfig ./examples/prompttester/config.yaml
```
Expected output:
```console
Science and Technology
```
### Example 2
Prompt:
```console
echo '\nQ:Translate the following text from French to English Je suis le père le plus heureux du monde\n\nA:' | go run ./examples/prompttester --dconfig ./examples/prompttester/config.yaml
```
Expected output:
```console
I am the happiest father in the world.
```
## Dependencies
A list of the main dependencies follows:
- [Spago](http://github.com/nlpodyssey/spago) - Machine Learning framework
- [RWKV](http://github.com/nlpodyssey/rwkv) - RWKV RNN implementation
- [GoTokenizers](http://github.com/nlpodyssey/gotokenizers) - Tokenizers library
- [GoPickle](http://github.com/nlpodyssey/gopickle) - Pickle library for Go
# Roadmap
- [x] Download pretrained models from the Hugging Face models hub
- [ ] Effective "prompts" catalog
- [x] Better sampling
- [ ] Beam search
- [ ] Better Tokenizer
- [ ] Unit tests
- [ ] Code refactoring
- [ ] Documentation
- [x] gRPC ~~/HTTP~~ API
# Credits
Thanks [PENG Bo](https://github.com/BlinkDL) for creating the RWKV RNN and all related resources, including pre-trained models!
# Trivia about the project's name
"VerbaFlow" combines "verba", which is the Latin word for *words*, and "flow", which alludes to the characteristics of recurrent neural networks by evoking the idea of a fluent and continuous flow of words, which is made possible by the network's ability to maintain an internal state and "remember" previous words and context when generating new words.