https://github.com/okwilkins/rag-cli
A project to show good CLI practices with a fully fledged RAG system.
https://github.com/okwilkins/rag-cli
cli llm ollama python qdrant rag
Last synced: 2 days ago
JSON representation
A project to show good CLI practices with a fully fledged RAG system.
- Host: GitHub
- URL: https://github.com/okwilkins/rag-cli
- Owner: okwilkins
- License: gpl-3.0
- Created: 2024-05-11T18:17:26.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-06-13T22:43:23.000Z (over 1 year ago)
- Last Synced: 2025-01-16T08:11:41.259Z (9 months ago)
- Topics: cli, llm, ollama, python, qdrant, rag
- Language: Python
- Homepage:
- Size: 115 KB
- Stars: 4
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: docs/CONTRIBUTING.md
- License: LICENSE
- Code of conduct: docs/CODE_OF_CONDUCT.md
- Security: docs/SECURITY.md
Awesome Lists containing this project
README
A project to show good CLI practices with a fully fledged RAG system.
# RAG CLI
## Installation
```bash
pip install rag-cli
```
## Features
- CLI tooling for RAG
- Embedder (Ollama)
- Vector store (Qdrant)
## Usage
### Docker
If you don't have a running instance of [Qdrant](https://qdrant.tech/) or [Ollama](https://ollama.com/), you can use the provided docker-compose file to start one.
```bash
docker-compose up --build -d
```
This will start Ollama on `http://localhost:11434` and Qdrant on `http://localhost:6333`.
#### Development
This project uses a dev container, which is the easiest way to set up a consistent development environment. Dev containers provide all the necessary tools, dependencies, and configuration, so you can focus on coding right away.
##### Using Dev Containers
This project uses a dev container for a consistent development environment. To get started:
1. Open the project in Visual Studio Code.
2. On Windows/Linux, press `Ctrl+Shift+P` and run the command `Remote-Containers: Reopen in Container`. On Mac, press `Cmd+Shift+P` and run the same command.
3. VS Code will build and start the dev container, providing access to the project's codebase and dependencies.
Other editors may have similar functionality but this project is optimised for Visual Studio Code.
### Embedder
Before running this command, make sure you have a running instance of [Ollama](https://ollama.com/) and the nomic-embed-text:v1.5 model is available:
```bash
ollama pull nomic-embed-text:v1.5
```
```bash
rag-cli embed --ollama-url http://localhost:11434 --file
```
You can alternatively use stdin to pass the text:
```bash
cat | rag-cli embed --ollama-url http://localhost:11434
```
### Vector store
```bash
rag-cli vector-store \
--qdrant-url http://localhost:6333 \
--collection-name \
--data '{}'
--embedding
```
You can alternatively use stdin to pass embeddings:
```bash
cat | \
rag-cli vector-store \
--qdrant-url http://localhost:6333 \
--collection-name \
--data '{}'
```
### RAG Chat
```bash
rag-cli rag \
--ollama-embedding-url http://localhost:11434 \
--ollama-chat-url http://localhost:11435 \
--qdrant-url http://localhost:6333 \
--collection-name nomic-embed-text-v1.5 \
--top-k 5 \
--min-similarity 0.5
--file
```
You can alternatively use stdin to pass the text:
```bash
cat | \
--ollama-embedding-url http://localhost:11434 \
--ollama-chat-url http://localhost:11435 \
--qdrant-url http://localhost:6333 \
--collection-name nomic-embed-text-v1.5 \
--top-k 5 \
--min-similarity 0.5
```
### End-to-end Pipeline For Storing Embeddings
Here is an example of an end-to-end pipeline for storing embeddings. It takes the following steps:
- Get a random Wikipedia article
- Embed the article
- Store the embedding in Qdrant
Before running the pipeline make sure you have the following installed:
```bash
sudo apt-get update && sudo apt-get install parallel jq curl
```
Also make sure that the `data/articles` and `data/embeddings` directories exist:
```bash
mkdir -p data/articles data/embeddings
```
Then run the pipeline:
```bash
bash scripts/run_pipeline.sh
```
#### Parallel Pipeline
The script `scripts/run_pipeline.sh` can be run in parallel with [GNU Parallel](https://www.gnu.org/software/parallel/) to speed up the process.
```bash
parallel -j 5 -n0 bash scripts/run_pipeline.sh ::: {0..10}
```
## Examples
### Get 10 Random Wikipedia Articles
```bash
parallel -n0 -j 10 '
curl -L -s "https://en.wikipedia.org/api/rest_v1/page/random/summary" | \
jq -r ".title, .description, .extract" | \
tee data/articles/$(cat /proc/sys/kernel/random/uuid).txt
' ::: {0..10}
```
### Run Embedder On All Articles
```bash
parallel '
rag-cli embed --ollama-url http://localhost:11434 --file {1} 2>> output.log | \
jq ".embedding" | \
tee data/embeddings/$(basename {1} .txt) 1> /dev/null
' ::: $(find data/articles/*.txt)
```
### Store All Embeddings In Qdrant
```bash
parallel rag-cli vector-store --qdrant-url http://localhost:6333 --collection-name nomic-embed-text-v1.5 2>> output.log ::: $(find data/embeddings/*)
```
### Run RAG Chat On A Query
```bash
echo "Who invented the blue LED?" | \
rag-cli rag \
--ollama-embedding-url http://localhost:11434 \
--ollama-chat-url http://localhost:11435 \
--qdrant-url http://localhost:6333 \
--collection-name nomic-embed-text-v1.5 \
--top-k 5 \
--min-similarity 0.5 \
2>> output.log
```
This example obviously requires that the articles similar to the query have been embedded and stored in Qdrant. You can do this with the example found in the next section.
### End-to-end Pipeline For A Single Article
```bash
wikipedia_data=$(curl -L -s "https://en.wikipedia.org/api/rest_v1/page/summary/Shuji_Nakamura") && \
payload_data=$(jq "{title: .title, description: .description, extract: .extract}" <(echo $wikipedia_data)) && \
text_to_embed=$(jq -r ".title, .description, .extract" <(echo $wikipedia_data)) && \
echo $text_to_embed | \
rag-cli embed --ollama-url http://localhost:11434 | \
jq -r ".embedding" | \
rag-cli vector-store \
--qdrant-url http://localhost:6333 \
--collection-name nomic-embed-text-v1.5 \
--data "$payload_data"
```