{"id":49635424,"url":"https://github.com/renswickd/rag-master-repo","last_synced_at":"2026-05-05T14:34:44.456Z","repository":{"id":311354720,"uuid":"1041980789","full_name":"renswickd/rag-master-repo","owner":"renswickd","description":"Master repository for various RAG types.","archived":false,"fork":false,"pushed_at":"2025-10-24T09:05:31.000Z","size":117,"stargazers_count":10,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-24T11:12:40.233Z","etag":null,"topics":["agentic-rag","cache-rag","groq","langgraph","multi-modal-rag","openai","rag","rag-ubac"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/renswickd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-21T09:53:08.000Z","updated_at":"2025-10-24T09:05:35.000Z","dependencies_parsed_at":"2025-08-24T10:15:59.841Z","dependency_job_id":"a81e810e-39b5-48cd-be5f-b25d64bc6b0a","html_url":"https://github.com/renswickd/rag-master-repo","commit_stats":null,"previous_names":["renswickd/rag-master-repo"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/renswickd/rag-master-repo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-master-repo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-master-repo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-master-repo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-master-repo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/renswickd","download_url":"https://codeload.github.com/renswickd/rag-master-repo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renswickd%2Frag-master-repo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32653660,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-05T11:29:49.557Z","status":"ssl_error","status_checked_at":"2026-05-05T11:29:48.587Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentic-rag","cache-rag","groq","langgraph","multi-modal-rag","openai","rag","rag-ubac"],"created_at":"2026-05-05T14:34:43.531Z","updated_at":"2026-05-05T14:34:44.441Z","avatar_url":"https://github.com/renswickd.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# rag-master-repo\n\nA modular Retrieval-Augmented Generation (RAG) repository with swappable pipelines and persistent vector stores (ChromaDB). Supports:\n- Basic text RAG (GROQ)\n- Multi-modal RAG (text + images via CLIP + GPT‑4.1)\n- LangGraph RAG (two-node graph: retrieve → generate)\n- rag-ubac: Role-based access control (UBAC) RAG\n- Cache-RAG: Intelligent caching of Q\u0026A pairs with LangGraph orchestration\n- Agentic-RAG: Tool-using LangGraph ReAct pipeline with resume retriever, web search, and currency conversion\n - HyDE RAG: Hypothetical Document Embedding (HyDE) pipeline — generate a concise hypothetical document that answers the query, retrieve relevant documents using the HyDE embedding, then produce a grounded final answer using the retrieved context.\n\n## Features\n\n- PDF ingestion and chunking\n- Persistent vector storage (Chroma PersistentClient) with per-type collections:\n  - `basic_rag_collection`, `multi_modal_collection`, `langgraph_collection`, `agentic_rag_collection`, `cache_rag_collection`, `rag_ubac_collection`, `hyde_collection`.\n- Modular retrievers, prompts, and pipelines\n- GROQ LLM for basic RAG; OpenAI GPT‑4.1 for multi‑modal; GROQ for LangGraph\n- CLI for vectorizing, querying, inspecting, listing, and deleting collections\n- Grounded prompts to reduce hallucinations\n\nRepository\n- https://github.com/renswickd/rag-master-repo.git\n\n## Setup\n\n1) Quick start\n- From the repo root, run the setup script to create data folders, a virtual environment (`.venv`), and install the repo in editable mode:\n  ```\n  bash ./setup.sh\n  source .venv/bin/activate\n  ```\n\n2) Data layout\n- Place your PDFs in:\n  - `data/source_data/basic-rag/`\n  - `data/source_data/multi-modal/`\n  - `data/source_data/langgraph/`\n  - `data/source_data/rag-ubac/`\n  - `data/source_data/agentic-rag/`\n  - `data/source_data/hyde/`\n\n3) Environment (.env)\n- GROQ (basic, langgraph):\n  ```\n  GROQ_API_KEY=your_groq_key_here\n  ```\n- SerpAPI (agentic web search tool) \u0026 Exchangerate API (you can use other preferred APIs as well):\n  ```\n  SERPAPI_API_KEY=your_serpapi_key_here\n  EXCHANGE_RATE_API=your_api_key\n  ```\n- OpenAI (multi‑modal):\n  ```\n  OPENAI_API_KEY=your_openai_key_here\n  ```\n- Optional (silence tokenizers warning):\n  ```\n  TOKENIZERS_PARALLELISM=false\n  ```\n\n4) Manual install (alternative to setup.sh)\n```\npip install -r requirements.txt\n```\n\n## CLI\n\n- Basic RAG\n  ```\n  python main.py --rag_type basic-rag\n  python main.py --rag_type basic-rag -v         # (re-)vectorize\n  python main.py --rag_type basic-rag --info     # collection info\n  ```\n\n- Multi‑Modal RAG\n  ```\n  python main.py --rag_type multi-modal\n  python main.py --rag_type multi-modal -v\n  python main.py --rag_type multi-modal --info\n  ```\n\n- LangGraph RAG\n  ```\n  python main.py --rag_type langgraph\n  python main.py --rag_type langgraph -v\n  python main.py --rag_type langgraph --info\n  ```\n\n- Agentic-RAG (tool-using agent: 1.resume retriever + 2.web search + 3.currency)\n  ```\n  python main.py --rag_type agentic-rag\n  python main.py --rag_type agentic-rag -v      # (re-)vectorize resume data\n  python main.py --rag_type agentic-rag --info  # collection info\n  ```\n\n- HyDE RAG\n  ```\n  python main.py --rag_type hyde\n  python main.py --rag_type hyde -v            # (re-)vectorize HyDE data (creates/updates `hyde_collection`)\n  python main.py --rag_type hyde --info       # collection info\n  ```\n\n- rag-ubac\n  You will be prompted to enter your role (executive/hr/junior). Answers are restricted by role-based access. UBAC uses metadata filters; re-run vectorization after updating FILE_ACCESS_METADATA.\n  ```\n  python main.py --rag_type rag-ubac --vectorize\n  python main.py --rag_type rag-ubac\n  python main.py --rag_type rag-ubac --info\n  ```\n\n- Cache-RAG\n  ```\n  python main.py --rag_type cache-rag\n  python main.py --rag_type cache-rag --vectorize\n  python main.py --rag_type cache-rag --clear-cache\n  python main.py --rag_type cache-rag --info\n  ```\n\n- Manage collections (ChromaDB)\n  ```\n  python main.py --rag_type basic-rag --list-collections\n  python main.py --rag_type basic-rag --delete-collection\n  ```\n\n- Interactive session: type `/exit` or `/quit` to finish.\n\nData directory is inferred from the RAG type:\n```\ndata/source_data/{basic-rag | multi-modal | langgraph | rag-ubac | agentic-rag}\n```\n\n## Grounded Prompts\n\n- Prompts enforce context-only answers. If no relevant context is retrieved, the system replies:\n  \"I am a helpful assitant for you to assist with the internal knowledge base; No related contents retrived for the provided query - Try modifying your query for assistance.\"\n\n## Project Structure (key paths)\n\n- `projects/retriever/`\n  - `basic_rag_retriever.py`\n  - `multi_modal_retriever.py`\n  - `langgraph_retriever.py`\n  - `rag_ubac_retriever.py`\n  - `cache_rag_retriever.py`\n  - `agentic_rag_retriever.py`\n- `projects/pipeline/`\n  - `basic_rag_pipeline.py`\n  - `multi_modal_rag_pipeline.py`\n  - `langgraph_rag_pipeline.py`\n  - `rag_ubac_pipeline.py`\n  - `cache_rag_pipeline.py`\n  - `agentic_rag_pipeline.py`\n  - `hyde_rag_pipeline.py`\n- `projects/prompts/`\n  - `prompts.py` (basic)\n  - `multi_modal_prompts.py`\n  - `langgraph_prompts.py`\n  - `hyde_prompts.py`\n- `shared/utils/`\n  - `pdf_utils.py` (PyMuPDF)\n  - `chroma_utils.py` (PersistentClient, collection helpers)\n  - `rag_ubac_scripts.py`\n- `shared/configs/`\n  - `static.py` (FILE_ACCESS_METADATA, VALID_ROLES, RAG_UBAC_TYPE)\n - `shared/tools/`\n   - `web_search_tool.py`, `currency_converter_tool.py`, `agentic_retriever_tool.py`\n\n## Tutorials\n\n- Basic RAG: `docs/tutorials/basic-rag-tutorial.md`\n- Multi‑Modal RAG: `docs/tutorials/multi-modal-rag.md`\n- RAG using Langgraph: `docs/tutorials/langgraph-rag.md`\n- RAG-UBAC tutorial: see `docs/tutorials/rag-ubac-tutorial.md`\n- Cache-RAG: `docs/tutorials/cache-rag-tutorial.md`\n- Agentic-RAG: `docs/tutorials/agentic-rag-tutorial.md` \n\n## Read the Stories on Medium\n- [RAG Series: Part 1 - Basic RAG](https://medium.com/@renswick.d/rag-series-part-1-basic-rag-2368c3243666)\n- [RAG Series: Part 2 - RAG using LangGraph](https://medium.com/@renswick.d/rag-series-part-2-rag-with-langgraph-1f5f2e669518)\n- [RAG Series: Part 3 - Agentic RAG with LangGraph](https://medium.com/@renswick.d/rag-series-part-3-agentic-rag-with-langgraph-tools-routing-and-control-7b0b3e15eb43)\n- [RAG Series: Part 4 - Multi-Modal RAG with CLIP and GPT-4.1](https://medium.com/@renswick.d/rag-series-part-4-multi-modal-rag-retrieval-with-text-images-using-clip-gpt-4-1-fdbb2b3e5467)\n- [RAG Series: Part 5 - Cache-RAG: Intelligent Caching for RAG](https://medium.com/@renswick.d/rag-series-part-5-cache-augmented-rag-faster-and-cost-optimized-retrieval-aa912d3f49bc)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frenswickd%2Frag-master-repo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frenswickd%2Frag-master-repo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frenswickd%2Frag-master-repo/lists"}