https://github.com/ovoshatchery/ovos-solver-plugin-llmcpp
https://github.com/ovoshatchery/ovos-solver-plugin-llmcpp
proof-of-concept work-in-progress
Last synced: about 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/ovoshatchery/ovos-solver-plugin-llmcpp
- Owner: OVOSHatchery
- License: apache-2.0
- Created: 2023-03-21T08:32:17.000Z (over 3 years ago)
- Default Branch: dev
- Last Pushed: 2023-03-29T06:47:35.000Z (over 3 years ago)
- Last Synced: 2025-05-17T02:12:28.957Z (about 1 year ago)
- Topics: proof-of-concept, work-in-progress
- Language: Python
- Homepage:
- Size: 25.4 KB
- Stars: 7
- Watchers: 5
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# 
LLM.cpp Persona
Give OpenVoiceOS some sass with [Alpaca.cpp](https://github.com/antimatter15/alpaca.cpp), [GPT4All.cpp](https://github.com/zanussbaum/gpt4all.cpp) or [Bloomz.cpp](https://github.com/NouamaneTazi/bloomz.cpp)
This plugin requires providing the path to the executable and model, it uses subprocess which allows it to work with these programs without requiring python bindings
Dedicated plugins may exist for each LLM
## Examples
* "What is best in life?"
* "Do you like dogs"
* "Does God exist?"
## Usage
Spoken answers api
```python
from ovos_solver_llmcpp import LLMcppSolver
ALPACA_MODEL_FILE = "./models/ggml-alpaca-7b-q4.bin"
GPT4ALL_MODEL_FILE = "./models/gpt4all-lora-quantized.bin"
# binpath = "~/alpaca.cpp/chat"
binpath = "~/gpt4all.cpp/chat"
bot = LLMcppSolver({"model": GPT4ALL_MODEL_FILE,
"executable_path": binpath})
sentence = bot.spoken_answer("Qual é o teu animal favorito?", {"lang": "pt-pt"})
# Meus animais favoritos são cães, gatos e tartarugas!
for q in ["Does god exist?",
"what is the speed of light?",
"what is the meaning of life?",
"What is your favorite color?",
"What is best in life?"]:
a = bot.get_spoken_answer(q)
print(q, a)
```