https://github.com/robjsliwa/llm-webui
Fun project to run your own LLM chat bot using llama.cpp
https://github.com/robjsliwa/llm-webui
chatbot llama llamacpp reactjs
Last synced: 3 months ago
JSON representation
Fun project to run your own LLM chat bot using llama.cpp
- Host: GitHub
- URL: https://github.com/robjsliwa/llm-webui
- Owner: robjsliwa
- Created: 2023-05-19T19:47:30.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2023-05-26T13:41:00.000Z (about 3 years ago)
- Last Synced: 2025-03-07T02:34:43.879Z (over 1 year ago)
- Topics: chatbot, llama, llamacpp, reactjs
- Language: TypeScript
- Homepage:
- Size: 355 KB
- Stars: 1
- Watchers: 2
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# llm-webui
Is a simple, fun project to run your own LLM chat using llama.cpp.
## How to run
1. Clone the repo: https://github.com/robjsliwa/llama-cpp-python for Python bindings for llama.cpp
2. If you want to get latest version of llama.cpp, go to `vendor` folder and run git clone https://github.com/ggerganov/llama.cpp in there or update the hash whatever is easier for you.
3. Build docker image: `docker build -t llama-server .`
4. Run it with: `docker run --rm -it -p 8000:8000 -v /home/data/datasets/wizard-vicuna:/models -e MODEL=/models/Wizard-Vicuna-13B-Uncensored.ggml.q8_0.bin llama-server`
5. Install web ui with: `npm install`
6. Start web ui with: `npm start`
**Note**: You can find great models on Hugging Face here: https://huggingface.co/TheBloke/Wizard-Vicuna-13B-Uncensored-GGML/tree/main