{"id":49635405,"url":"https://github.com/renswickd/rag-llmops","last_synced_at":"2026-05-05T14:34:43.310Z","repository":{"id":346117620,"uuid":"1188519878","full_name":"renswickd/rag-llmops","owner":"renswickd","description":"A production-style RAG-based AI system with end-to-end LLMOps practices, enabling document ingestion, multi-turn conversational retrieval, and complete CI/CD guidelines.","archived":false,"fork":false,"pushed_at":"2026-05-02T07:57:32.000Z","size":921,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-02T09:18:57.453Z","etag":null,"topics":["azure-container-instances","azure-container-registry","docker-compose","langchain","openai-api","typescript","vite"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/renswickd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-22T07:32:56.000Z","updated_at":"2026-05-02T07:57:36.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/renswickd/rag-llmops","commit_stats":null,"previous_names":["renswickd/rag-llmops"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/renswickd/rag-llmops","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-llmops","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-llmops/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-llmops/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-llmops/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/renswickd","download_url":"https://codeload.github.com/renswickd/rag-llmops/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-llmops/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32653657,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-05T11:29:49.557Z","status":"ssl_error","status_checked_at":"2026-05-05T11:29:48.587Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-container-instances","azure-container-registry","docker-compose","langchain","openai-api","typescript","vite"],"created_at":"2026-05-05T14:34:42.566Z","updated_at":"2026-05-05T14:34:43.301Z","avatar_url":"https://github.com/renswickd.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RAG LLMOps\n\nA self-contained Retrieval-Augmented Generation (RAG) system built as a learning project to explore LLMOps practices. Upload documents, ask questions, and get grounded answers with source citations — all backed by a production-style FastAPI backend.\n\n---\n\n## Objective\n\nThe goal of this project is to build and deploy a full-stack RAG application that demonstrates:\n\n- Clean backend architecture with FastAPI and dependency injection\n- A real document ingestion pipeline (PDF/TXT/MD → chunks → embeddings → FAISS)\n- Multi-turn conversational retrieval with question condensing\n- Per-session document isolation — each session only retrieves from its own documents\n- Persistent conversation history that survives backend restarts\n- A modern React frontend with dark/light mode and session management\n- Containerised deployment on Azure Container Instances\n- LLMOps practices: structured logging, configuration management, and automated testing\n\nThis is a learning project intended to be shared with others for testing.\n\n---\n\n## Architecture\n\n```\nfrontend/          React + Vite + TypeScript + TailwindCSS + shadcn/ui\napi/               FastAPI application\n  routers/         chat, documents, health endpoints\n  schemas/         Pydantic request/response models\n  dependencies.py  Service singleton initialisation\n  main.py          App entry point with lifespan management\nconversation/      Multi-turn chat manager with JSONL history persistence\ningestion/         Document loading, chunking, FAISS vector store, retriever\ncore/              Config loader, structured logging, custom exceptions, session registry\nutils/             LLM and embeddings model loader, session ID generation\nconfig/            config.yaml — all tunable parameters\ndata/\n  uploads/         Archived uploaded files, namespaced by session_id\n  history/         Per-session conversation history as JSONL files\n  session_registry.json  Authoritative session list with metadata\ntests/             pytest unit test suite\ndocs/              Implementation plans and roadmaps\n```\n\n---\n\n## Tech Stack\n\n### Backend\n| Component | Technology |\n|-----------|-----------|\n| API framework | FastAPI + Uvicorn |\n| LLM | Groq (`openai/gpt-oss-20b`) via LangChain |\n| Embeddings | HuggingFace `google/embeddinggemma-300m` |\n| Vector store | FAISS (local filesystem) |\n| Document parsing | PyMuPDF, PyPDF |\n| Chunking | LangChain `RecursiveCharacterTextSplitter` |\n| Logging | structlog (structured JSON) |\n| Config | YAML + python-dotenv |\n\n### Frontend\n| Component | Technology |\n|-----------|-----------|\n| UI library | React 19 + Vite 8 + TypeScript 6 |\n| Styling | TailwindCSS 4 (CSS-first) + shadcn/ui |\n| Components | Radix UI primitives via shadcn/ui |\n| State | Zustand 5 with persist middleware |\n| Markdown rendering | react-markdown + remark-gfm |\n| File upload | react-dropzone |\n| Icons | lucide-react |\n\n### Deployment (planned — Phase 5–6)\n| Component | Technology |\n|-----------|-----------|\n| Containerisation | Docker (multi-stage build) |\n| Local orchestration | docker-compose |\n| Cloud | Azure Container Instances + Azure Container Registry |\n| CI | GitHub Actions |\n\n---\n\n## API Endpoints\n\nAll endpoints are prefixed `/api/v1/`.\n\n| Method | Endpoint | Description |\n|--------|----------|-------------|\n| `GET` | `/health` | Health check — returns app name, version, environment |\n| `POST` | `/documents/upload` | Upload a `.pdf`, `.txt`, or `.md` file; returns `session_id`, `file_name`, and `chunks_created` |\n| `POST` | `/chat` | Send a question with a `session_id`; returns answer, sources, and conversation metadata |\n| `GET` | `/chat/sessions` | List all sessions with full metadata (`session_id`, `created_at`, `documents[]`) |\n| `DELETE` | `/chat/sessions/{session_id}` | Fully delete a session: registry, uploaded files, FAISS entries, and chat history |\n| `GET` | `/chat/sessions/{session_id}/history` | Return stored conversation history for re-hydration after a page refresh |\n\nInteractive docs available at `http://localhost:8000/docs` when running locally.\n\n---\n\n## Session Model\n\nEach document upload creates a **session** — the shared key linking files, conversation history, and FAISS vectors.\n\n```\nsession_id format:  upload_YYYYMMDD_HHMMSS_\u003c8hex\u003e\ne.g.                upload_20260418_143022_a3f9c1b7\n```\n\nSessions are tracked in a **session registry** (`data/session_registry.json`) written at upload time — not at first chat — so `GET /chat/sessions` always reflects all uploaded documents immediately.\n\n### Document isolation\n\nEach uploaded document's chunks are tagged with their `session_id` in FAISS metadata. The retriever filters by `session_id` at query time, so Session A can never return chunks from Session B's documents.\n\n### History persistence\n\nConversation turns are appended to `data/history/\u003csession_id\u003e.jsonl` after each chat response. On backend startup, all JSONL files are scanned and loaded back into memory. Deleting a session removes the JSONL file.\n\n### Full delete\n\n`DELETE /chat/sessions/{session_id}` performs a complete teardown:\n1. Remove from the session registry\n2. Delete `data/uploads/\u003csession_id\u003e/` from disk\n3. Rebuild the FAISS index from the remaining sessions\n4. Clear the in-memory chat history\n\n---\n\n## RAG Pipeline\n\n```\nUpload file\n    └── Generate session_id (upload_YYYYMMDD_HHMMSS_\u003c8hex\u003e)\n    └── Archive to data/uploads/\u003csession_id\u003e/\u003cfilename\u003e\n    └── Write session metadata to session_registry.json\n    └── Parse (PDF / TXT / MD)\n    └── Chunk (RecursiveCharacterTextSplitter, 1000 chars / 150 overlap)\n    └── Tag each chunk with session_id in metadata\n    └── Embed (HuggingFace google/embeddinggemma-300m)\n    └── Add to FAISS index (faiss_index/)\n\nAsk question\n    └── Condense follow-up into standalone query (Groq LLM)\n    └── Retrieve top-k documents filtered by session_id (similarity / MMR / score-threshold)\n    └── Format context with source citations\n    └── Generate grounded answer (Groq LLM)\n    └── Append human + AI turn to data/history/\u003csession_id\u003e.jsonl\n    └── Return answer + sources + history length\n```\n\n---\n\n## Getting Started\n\n### Prerequisites\n\n- Python 3.13 + [uv](https://docs.astral.sh/uv/getting-started/installation/)\n- Node.js 20+ and npm\n- A [Groq API key](https://console.groq.com/)\n- A [HuggingFace token](https://huggingface.co/settings/tokens)\n- Docker (for containerised run)\n\n### Install\n\n```bash\ngit clone \u003crepo-url\u003e\ncd rag-llmops-2\n\n# Backend — uv creates .venv and installs all dependencies from uv.lock\nuv sync\n\n# Frontend\ncd frontend \u0026\u0026 npm install \u0026\u0026 cd ..\n```\n\n### Configure\n\n```bash\ncp .env.example .env\n# Edit .env and fill in GROQ_API_KEY and HF_TOKEN\n```\n\n`.env.example`:\n```\nGROQ_API_KEY=\u003cyour-groq-api-key\u003e\nHF_TOKEN=\u003cyour-hf-token\u003e\nCONFIG_PATH=config/config.yaml\nCORS_ORIGINS=http://localhost:5173\nSERVE_FRONTEND=true\n```\n\nAll other parameters (model names, chunk size, retrieval settings) are in `config/config.yaml`.\n\n### Run — development (two terminals)\n\n```bash\n# Terminal 1 — FastAPI backend\npython run.py\n# API available at http://localhost:8000\n# Interactive docs at http://localhost:8000/docs\n```\n\n```bash\n# Terminal 2 — React frontend (Vite dev server)\ncd frontend \u0026\u0026 npm run dev\n# UI available at http://localhost:5173\n```\n\nThe Vite dev server proxies `/api` requests to `localhost:8000`, so CORS is not an issue in development.\n\n### Run — Docker (single container)\n\n```bash\n# Build image (first time ~5–8 min; rebuilds after code changes are fast)\ndocker build -t rag-llmops:local .\n\n# Run — secrets come from .env, never from the command line\ndocker run --rm -p 8000:8000 --env-file .env rag-llmops:local\n```\n\nThe frontend and API are both served from port 8000 (`SERVE_FRONTEND=true` in `.env`). Open `http://localhost:8000`.\n\n### Run — docker-compose (with persistent volumes)\n\n```bash\ndocker compose up --build   # first run\ndocker compose up           # subsequent runs (no rebuild needed)\ndocker compose down         # stop, keep volumes\ndocker compose down -v      # stop and delete all data volumes\n```\n\n### Test\n\n```bash\n# Backend\npytest tests/ -v\n\n# Frontend build check\ncd frontend \u0026\u0026 npm run build\n```\n\n---\n\n## Configuration\n\nKey settings in `config/config.yaml`:\n\n| Section | Key | Default | Description |\n|---------|-----|---------|-------------|\n| `llm.groq` | `model_name` | `openai/gpt-oss-20b` | Groq model |\n| `llm.groq` | `max_history_turns` | `10` | Sliding window for conversation history |\n| `embedding_model` | `model_name` | `google/embeddinggemma-300m` | HuggingFace embedding model |\n| `data` | `data_dir` | `data` | Root directory for uploads, history, and registry |\n| `data` | `history_dir` | `data/history` | Directory for per-session JSONL history files |\n| `data` | `uploads_subdir` | `uploads` | Sub-directory under `data_dir` for archived files |\n| `data_ingestion` | `chunk_size` | `1000` | Characters per chunk |\n| `data_ingestion` | `chunk_overlap` | `150` | Overlap between chunks |\n| `retriever` | `default_search_type` | `similarity` | `similarity`, `mmr`, or `similarity_score_threshold` |\n| `retriever` | `default_top_k` | `4` | Documents retrieved per query |\n\n---\n\n## Implementation Progress\n\nTracked in [`docs/plan.md`](docs/plan.md).\n\n### Backend\n\n| Component | Details | Status |\n|-----------|---------|--------|\n| Core infrastructure | YAML config loader, structlog JSON logging, `RagAssistantException` with traceback capture | Done |\n| Model loader | Groq `ChatGroq` LLM + HuggingFace `HuggingFaceEmbeddings` initialisation with config-driven parameters | Done |\n| Document ingestion | Load PDF / TXT / MD via LangChain loaders, archive to `data/uploads/\u003csession_id\u003e/`, chunk with `RecursiveCharacterTextSplitter` | Done |\n| FAISS vector store | Create, load, update, and clear FAISS index; persist to disk; per-session chunk metadata | Done |\n| Session registry | `core/session_store.py` — JSON-backed registry written at upload time; returned by `list_sessions`; cleaned up on delete | Done |\n| Retrieval pipeline | Three search modes: `similarity`, `mmr`, `similarity_score_threshold`; per-session filtering via chunk metadata | Done |\n| Conversation chain | `ChatManager` with per-session `InMemoryChatMessageHistory`, sliding window, standalone question condensing via LangChain LCEL | Done |\n| History persistence | Turns appended to `data/history/\u003csession_id\u003e.jsonl`; loaded on startup; deleted with session | Done |\n| History endpoint | `GET /chat/sessions/{session_id}/history` — returns stored turns for frontend re-hydration | Done |\n| Session metadata API | `GET /chat/sessions` returns `SessionMetadata[]` with `session_id`, `created_at`, `documents[]` | Done |\n| Full session delete | `DELETE /chat/sessions/{session_id}` — registry + files + FAISS rebuild + history | Done |\n| API layer | FastAPI routers for `chat`, `documents`, `health`; Pydantic schemas; singleton dependency injection; lifespan startup/shutdown | Done |\n| CORS + static serving | `CORSMiddleware` with env-configurable origins; `SERVE_FRONTEND=true` guard for production static file mount | Done |\n| Unit tests | pytest tests covering all layers — config, exceptions, logging, model loader, ingestion, retrieval, chat manager, and all API endpoints | Done |\n\n### Frontend\n\n| Phase | Description | Status |\n|-------|-------------|--------|\n| 2 | Frontend scaffold (React 19 + Vite 8 + TypeScript 6 + Tailwind v4 + shadcn/ui + Zustand) | Done |\n| 3 | Core UI components (two-panel chat, drag-and-drop upload, dark/light mode, markdown rendering, source citations) | Done |\n| 3+ | Session management with full `SessionMetadata` — sidebar shows document names and dates; header dropdown uses document names; Zustand store updated to `SessionMetadata[]` | Done |\n| 3+ | History re-hydration on page refresh — `switchSession` fetches backend history if no messages are cached; loading spinner shown during fetch | Done |\n| 4 | Frontend testing (Vitest unit + Playwright E2E) | Pending |\n\n### Deployment\n\n| Phase | Description | Status |\n|-------|-------------|--------|\n| 5 | Docker multi-stage build (Node → Python, uv venv, CPU-only torch) | Done |\n| 5 | docker-compose with named volumes, health check, env-file secrets | Done |\n| 6 | Azure Container Registry + Azure Container Instances deployment | Pending |\n| 7 | GitHub Actions CI pipeline | Pending |\n\n---\n\n## Project Structure\n\n```\n.\n├── api/\n│   ├── main.py              # FastAPI app + CORS + conditional static file serving\n│   ├── dependencies.py      # Service singleton initialisation (ChatManager, SessionRegistry, FaissManager)\n│   ├── routers/\n│   │   ├── chat.py          # POST /chat, GET/DELETE /sessions, GET /sessions/{id}/history\n│   │   ├── documents.py     # POST /documents/upload\n│   │   └── health.py\n│   └── schemas/\n│       ├── chat.py          # ChatRequest, ChatResponse, SessionMetadata, SessionListResponse, HistoryResponse\n│       └── document.py      # UploadResponse\n├── conversation/\n│   ├── chat_manager.py      # ChatManager: sessions, JSONL persistence, history load/clear\n│   └── prompt_builder.py    # RAG and condense prompts\n├── ingestion/\n│   ├── data_ingestion.py    # Load, chunk, archive files; injects session_id into chunk metadata\n│   ├── faiss_manager.py     # FAISS index lifecycle (create, load, add, clear)\n│   └── retriever.py         # Similarity / MMR / threshold retrieval with session_id filter\n├── core/\n│   ├── session_store.py     # SessionRegistry — JSON-backed session metadata store\n│   ├── config.py            # YAML config loader\n│   ├── logging_config.py    # structlog setup\n│   └── exceptions.py        # RagAssistantException\n├── utils/\n│   ├── model_loader.py      # Groq LLM + HuggingFace embeddings\n│   └── file_handling.py     # Session ID generation (flat URL-safe format)\n├── config/\n│   └── config.yaml\n├── data/\n│   ├── uploads/             # Archived files per session (data/uploads/\u003csession_id\u003e/)\n│   ├── history/             # JSONL conversation history per session\n│   └── session_registry.json\n├── frontend/\n│   ├── src/\n│   │   ├── api/client.ts    # Typed fetch wrapper for all backend endpoints\n│   │   ├── store/appStore.ts # Zustand store (SessionMetadata[], messages, hydration state, theme)\n│   │   ├── types/index.ts   # TypeScript interfaces mirroring Pydantic schemas\n│   │   └── components/      # Layout, Header, Sidebar, ChatArea, MessageList,\n│   │                        #   ChatInput, DocumentUpload, SessionList, ThemeToggle\n│   ├── vite.config.ts       # Tailwind plugin + @/ alias + /api proxy\n│   └── package.json\n├── tests/                   # pytest unit tests\n├── docs/\n│   ├── plan.md              # Full implementation roadmap\n│   ├── fix_session_plan.md  # Session gap analysis and Phase A–D roadmap\n│   ├── fix_session_plan_a.md # Phase A implementation guide (session foundation)\n│   └── fix_session_plan_bc.md # Phase B+C implementation guide (persistence + metadata UX)\n├── Dockerfile               # Multi-stage build: Node (frontend) → Python (backend)\n├── docker-compose.yaml      # Local orchestration with named volumes and health check\n├── run.py                   # Entry point (calls uvicorn.run() via Python API)\n├── pyproject.toml           # Dependencies + uv config (CPU torch index, dev group)\n├── uv.lock                  # Pinned lock file — regenerate with `uv lock` after pyproject changes\n└── .env.example\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frenswickd%2Frag-llmops","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frenswickd%2Frag-llmops","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frenswickd%2Frag-llmops/lists"}