https://github.com/timescale/private-rag-example
Private RAG app sample using Llama3, Ollama and PostgreSQL
https://github.com/timescale/private-rag-example
llama3 ollama pgvector rag
Last synced: 2 months ago
JSON representation
Private RAG app sample using Llama3, Ollama and PostgreSQL
- Host: GitHub
- URL: https://github.com/timescale/private-rag-example
- Owner: timescale
- Created: 2024-10-08T17:43:09.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2024-11-06T17:38:49.000Z (7 months ago)
- Last Synced: 2025-03-24T18:03:59.979Z (3 months ago)
- Topics: llama3, ollama, pgvector, rag
- Language: Jupyter Notebook
- Homepage: https://youtu.be/-ikCYKcPoqU
- Size: 336 KB
- Stars: 45
- Watchers: 11
- Forks: 11
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Local RAG Example
[Watch the video tutorial here](https://youtu.be/-ikCYKcPoqU)
[Read the blog post using Mistral here](https://www.timescale.com/blog/build-a-fully-local-rag-app-with-postgresql-mistral-and-ollama/)
This repository contains an example project for building a private Retrieval-Augmented Generation (RAG) application using Llama3.2, Ollama, and PostgreSQL. It demonstrates how to set up a RAG pipeline that does not rely on external API calls, ensuring that sensitive data remains within your infrastructure.
## Prerequisites
* Docker
* Python, [psycopg](https://www.psycopg.org/)
* [Ollama](https://github.com/ollama/ollama)
* [PostgreSQL](https://docs.timescale.com/self-hosted/latest/install/installation-docker/), [pgai](https://github.com/timescale/pgai)
## Docker Setup
* Create a network through which the Ollama and PostgreSQL containers will interact:
`docker network create local-rag`
* [Ollama](https://hub.docker.com/r/ollama/ollama) docker container: (Note: [`--network`](https://docs.docker.com/engine/network/) tag to make sure that the container runs on the network defined)
`docker run -d --network local-rag -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama`
* [Llama3.2](https://www.llama.com/docs/model-cards-and-prompt-formats/llama3_2): `docker exec -it ollama ollama pull llama3.2`
* [Mistral](https://mistral.ai/news/announcing-mistral-7b/?ref=timescale.com): `docker exec -it ollama ollama pull mistral`
* [Nomic Embed v1.5](https://www.nomic.ai/blog/posts/nomic-embed-matryoshka?ref=timescale.com): `docker exec -it ollama ollama pull nomic-embed-text`
* [TimescaleDB](https://docs.timescale.com/self-hosted/latest/install/installation-docker/): `docker run -d --network local-rag --name timescaledb -p 5432:5432 -e POSTGRES_PASSWORD=password timescale/timescaledb-ha:pg16`
* [pgai](https://github.com/timescale/pgai?tab=readme-ov-file#use-a-timescale-cloud-service): `CREATE EXTENSION IF NOT EXISTS "ai" VERSION '0.4.0' CASCADE;` (also installs [`pgvector`](https://github.com/pgvector/pgvector) and `plpython3`)