https://github.com/araobp/compact-rag
Compact RAG that runs on Raspberry Pi
https://github.com/araobp/compact-rag
embeddings openai-api raspberry-pi
Last synced: 2 months ago
JSON representation
Compact RAG that runs on Raspberry Pi
- Host: GitHub
- URL: https://github.com/araobp/compact-rag
- Owner: araobp
- License: mit
- Created: 2024-09-30T13:40:53.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-01-26T13:25:24.000Z (over 1 year ago)
- Last Synced: 2025-03-09T00:32:03.648Z (over 1 year ago)
- Topics: embeddings, openai-api, raspberry-pi
- Language: Jupyter Notebook
- Homepage:
- Size: 4.77 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Compact RAG
## Background
I develop a compact RAG (Retrieval-Augmented Generation) that runs on Raspberry Pi. As the database for RAG, I adopt SQLite and implement a vector DB using [sqlite-vec](https://github.com/asg017/sqlite-vec).
## Goal of this project
- Develop a compact RAG that runs on Raspberry Pi, supporting Hybrid RAG: SQL DB and Vector DB.
- The RAG also works as an API server for my other projects: [virtual-showroom](https://github.com/araobp/virtual-showroom).
## Requrements
- OpenAI API key
- LLM model: gpt-4o-mini
- Embeddings model: text-embedding-3-small
- Raspberry Pi
## Architecture
```
Brain
[OpenAI API service]
Unity app |
[VirtualShowroom]-----+ |
| |
Web apps | Compact RAG (app.py)
[Web Browser]---------+------- [Raspberry Pi]---+---USB---[Camera with mic]
| | |
GenAI | SQLite DB +---USB---[Speaker]
[Node-RED]------------+
```
## Compiling sqlite-vec on Rapsberry Pi
```
$ git clone https://github.com/asg017/sqlite-vec
$ cd sqlite-vec
$ sudo apt-get install libsqlite3-dev
$ make loadable
```
Find "vec0.so" in ./dist directory.
## Reference documents, chunking and embeddings for RAG
### Document sources
- [Travel guidebooks](./ref/virtual_showroom) for [virtual-showroom](https://github.com/araobp/virtual-showroom).
### Chunking and Embeddings
- [Step 1. Generating Chunks](./ref/Chunks.ipynb): I run this notebook on my Mac.
- [Step 2. Calculating embeddings](./ref/calc_embeddings.py): I run this script on my Raspberry Pi 3.
## Implementations
- [cx package](./cx) ... Python package
- [API server](./app) ... API Server
## Partition keys and auxiliary columns supported by sqlite-vec
I use partition keys and auxiliary columns for filtering records on the database in this project:
```
CREATE VIRTUAL TABLE virtual_showroom
USING vec0(
context text partition key,
embedding float[1536],
+chunk text
)
```
Reference: https://alexgarcia.xyz/sqlite-vec/features/vec0.html
### Unit tests
- [Unit tests for "cx" package](./unittest/cx)
- [Unit test for the API server](./unittest/api)
### Running the API server
```
$ cd app
$ python app.py
```
The API server provides simple web apps. Access "http://\:5050" with a web browser.
[virtual-showroom](https://github.com/araobp/virtual-showroom) uses this API server to access the OpenAI API service.
#### Starting the API server automatically
Refer to [this article](https://ponnala.medium.com/never-let-your-python-http-server-die-step-by-step-guide-to-auto-start-on-boot-and-crash-recovery-1f7b0f94401e) to start the server automatically.
A sample service file is like this:
```
[Unit]
Description=Python Generative AI API server
After=network.target
[Service]
ExecStart=/usr/bin/python3 -m app --directory
WorkingDirectory=
Restart=always
RestartSec=10
User=
Group=users
Environment=PYTHONPATH=:$PYTHONPATH OPENAI_API_KEY=
[Install]
WantedBy=multi-user.target
```
After having created the service file, do this:
```
$ sudo systemctl daemon-reload
$ sudo systemctl start gen_ai.service
```
Confirm the daemon process running:
```
$ sudo systemctl start gen_ai.service
```
If something wrong happened, check the syslog:
```
$ tail /var/log/syslog
```
## Extra: Some experiments with gpt-4o-mini
- [Character Profiling](./CHARACTER_PROFILING.md)
- [Hand Gesture Recognition](./HAND_GESTURE_RECOGNITION.md)