Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/redthing1/red-chat-ui
a chatbot-ui fork for local models
https://github.com/redthing1/red-chat-ui
Last synced: 5 days ago
JSON representation
a chatbot-ui fork for local models
- Host: GitHub
- URL: https://github.com/redthing1/red-chat-ui
- Owner: redthing1
- License: mit
- Created: 2024-02-29T22:55:25.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2024-07-14T03:02:06.000Z (6 months ago)
- Last Synced: 2024-11-08T08:26:51.007Z (about 2 months ago)
- Language: TypeScript
- Size: 2.43 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: license
- Security: SECURITY.md
Awesome Lists containing this project
README
# red-chatbot-ui
a [chatbot-ui](https://github.com/mckaywrigley/chatbot-ui) fork for local models. a continuation of [this fork](https://github.com/yportne13/chatbot-ui-llama.cpp), which made it local-first.
# usage
## build the webui
in this respository:
```sh
npm install
npm run build
```
this will build a static webpage to `./out`.
## build [llama.cpp](https://github.com/ggerganov/llama.cpp)
in your llama.cpp directory:
```sh
make -j
```
## run the llama.cpp server with this frontend:
run llama.cpp server:
```sh
/path/to/llama.cpp/server --model /path/to/mistral-7b-openorca.Q4_0.gguf --path /path/to/red-chatbot-ui/out --port 8000
```
tips for llama.cpp server:
+ pass `-ngl ` to put layers on the gpu
+ pass `-c -n ` to set the context window
+ pass `--host ` or `--port ` to set the server endpoint