https://github.com/dusanov/cautious-broccoli
Local Llm wrapper with the RAG embeddings
https://github.com/dusanov/cautious-broccoli
htmx htmx-extension llama mistral-7b nodejs python qdrant tailwindcss websockets
Last synced: 4 months ago
JSON representation
Local Llm wrapper with the RAG embeddings
- Host: GitHub
- URL: https://github.com/dusanov/cautious-broccoli
- Owner: dusanov
- Created: 2024-03-22T15:37:21.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-01-12T17:07:17.000Z (5 months ago)
- Last Synced: 2025-01-12T17:36:50.317Z (5 months ago)
- Topics: htmx, htmx-extension, llama, mistral-7b, nodejs, python, qdrant, tailwindcss, websockets
- Language: CSS
- Homepage:
- Size: 301 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Claudia 0.1 - Dusanov embedded resume
Exercise in RAG embedding, htmx, node and python
## Before first run:
After cloning the project:
### Profetisa
Before first run, under profetisa folder create model folder and download llm model into it ( 4.1G ):
```
mkdir model
cd model
wget -c https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF/resolve/main/openhermes-2.5-mistral-7b.Q4_K_M.gguf
```
## Running
to bring app up:
`
docker compose up -d --build
`
Please note that it will take a while to build everything ( 713.1s on my box ). Also, on the first run embedder service will start the download of embedding model / library so it will be unresponsive on first couple of requests.
Embeddings can be monitored by looking at the logs:
`
docker logs claudia-front --follow
`
App url is: [http://localhost:3000](http://localhost:3000)
To bring everything down:
`
docker compose down
`
### Example question
`
What period and what did Dusan work on at the Smartcat doo company from Novi Sad ?
`