{"id":18217268,"url":"https://github.com/araobp/compact-rag","last_synced_at":"2026-04-28T11:02:06.761Z","repository":{"id":258601336,"uuid":"865410096","full_name":"araobp/compact-rag","owner":"araobp","description":"Compact RAG that runs on Raspberry Pi","archived":false,"fork":false,"pushed_at":"2025-01-26T13:25:24.000Z","size":5004,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-09T00:32:03.648Z","etag":null,"topics":["embeddings","openai-api","raspberry-pi"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/araobp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-30T13:40:53.000Z","updated_at":"2025-01-26T13:25:28.000Z","dependencies_parsed_at":"2024-11-02T13:23:51.051Z","dependency_job_id":"c9806bf4-c43a-418c-a3e4-d353a9dc1799","html_url":"https://github.com/araobp/compact-rag","commit_stats":{"total_commits":208,"total_committers":1,"mean_commits":208.0,"dds":0.0,"last_synced_commit":"58a1c0fae57bf5c85972ec2a997274cf24a37571"},"previous_names":["araobp/showroom-controller"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araobp%2Fcompact-rag","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araobp%2Fcompact-rag/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araobp%2Fcompact-rag/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/araobp%2Fcompact-rag/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/araobp","download_url":"https://codeload.github.com/araobp/compact-rag/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247736956,"owners_count":20987708,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embeddings","openai-api","raspberry-pi"],"created_at":"2024-11-03T17:04:13.130Z","updated_at":"2026-04-28T11:02:01.738Z","avatar_url":"https://github.com/araobp.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Compact RAG\n\n\u003cimg src=\"./docs/my_raspberry_pi.jpg\" width=400\u003e\n\n## Background\n\nI develop a compact RAG (Retrieval-Augmented Generation) that runs on Raspberry Pi. As the database for RAG, I adopt SQLite and implement a vector DB using [sqlite-vec](https://github.com/asg017/sqlite-vec).\n\n## Goal of this project\n\n- Develop a compact RAG that runs on Raspberry Pi, supporting Hybrid RAG: SQL DB and Vector DB.\n- The RAG also works as an API server for my other projects: [virtual-showroom](https://github.com/araobp/virtual-showroom).\n\n## Requrements\n\n- OpenAI API key\n- LLM model: gpt-4o-mini\n- Embeddings model: text-embedding-3-small\n- Raspberry Pi\n\n## Architecture\n\n```\n                                   Brain\n                           [OpenAI API service]\nUnity app                            |\n[VirtualShowroom]-----+              |\n                      |              |\nWeb apps              |        Compact RAG (app.py)\n[Web Browser]---------+------- [Raspberry Pi]---+---USB---[Camera with mic]\n                      |              |          |\nGenAI                 |          SQLite DB      +---USB---[Speaker]\n[Node-RED]------------+\n```\n\n## Compiling sqlite-vec on Rapsberry Pi\n\n```\n$ git clone https://github.com/asg017/sqlite-vec\n$ cd sqlite-vec\n$ sudo apt-get install libsqlite3-dev\n$ make loadable \n```\n\nFind \"vec0.so\" in ./dist directory.\n\n## Reference documents, chunking and embeddings for RAG\n\n### Document sources\n\n- [Travel guidebooks](./ref/virtual_showroom) for [virtual-showroom](https://github.com/araobp/virtual-showroom).\n\n### Chunking and Embeddings\n\n- [Step 1. Generating Chunks](./ref/Chunks.ipynb): I run this notebook on my Mac.\n- [Step 2. Calculating embeddings](./ref/calc_embeddings.py): I run this script on my Raspberry Pi 3.\n\n## Implementations\n\n- [cx package](./cx) ... Python package\n- [API server](./app) ... API Server\n\n\u003cimg src=\"docs/api_server.jpg\" width=700\u003e\n\n## Partition keys and auxiliary columns supported by sqlite-vec\n\nI use partition keys and auxiliary columns for filtering records on the database in this project:\n\n```\nCREATE VIRTUAL TABLE virtual_showroom\nUSING vec0(\ncontext text partition key,\nembedding float[1536],\n+chunk text\n)\n```\n\nReference: https://alexgarcia.xyz/sqlite-vec/features/vec0.html\n\n### Unit tests\n\n- [Unit tests for \"cx\" package](./unittest/cx)\n- [Unit test for the API server](./unittest/api)\n\n### Running the API server\n\n```\n$ cd app\n$ python app.py\n```\n\nThe API server provides simple web apps. Access \"http://\\\u003cIP address of the API server\\\u003e:5050\" with a web browser.\n\n[virtual-showroom](https://github.com/araobp/virtual-showroom) uses this API server to access the OpenAI API service.\n\n#### Starting the API server automatically\n\nRefer to [this article](https://ponnala.medium.com/never-let-your-python-http-server-die-step-by-step-guide-to-auto-start-on-boot-and-crash-recovery-1f7b0f94401e) to start the server automatically.\n\nA sample service file is like this:\n\n```\n[Unit]\nDescription=Python Generative AI API server\nAfter=network.target\n\n[Service]\nExecStart=/usr/bin/python3 -m app --directory \u003cPath to \"app\" folder\u003e\nWorkingDirectory=\u003cPath to \"app\" folder\u003e\nRestart=always\nRestartSec=10\nUser=\u003cYour user name\u003e\nGroup=users\nEnvironment=PYTHONPATH=\u003cPath to this repo on Raspberry Pi\u003e:$PYTHONPATH OPENAI_API_KEY=\u003cOpenAI API key\u003e\n\n[Install]\nWantedBy=multi-user.target\n```\n\nAfter having created the service file, do this:\n\n```\n$ sudo systemctl daemon-reload\n$ sudo systemctl start gen_ai.service\n```\n\nConfirm the daemon process running:\n\n```\n$ sudo systemctl start gen_ai.service\n```\n\nIf something wrong happened, check the syslog:\n```\n$ tail /var/log/syslog\n```\n\n## Extra: Some experiments with gpt-4o-mini\n\n- [Character Profiling](./CHARACTER_PROFILING.md)\n- [Hand Gesture Recognition](./HAND_GESTURE_RECOGNITION.md)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faraobp%2Fcompact-rag","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faraobp%2Fcompact-rag","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faraobp%2Fcompact-rag/lists"}