{"id":23028772,"url":"https://github.com/sunnybedi990/rag-with-llm","last_synced_at":"2025-04-23T13:30:08.554Z","repository":{"id":262214072,"uuid":"886553632","full_name":"sunnybedi990/RAG-with-LLM","owner":"sunnybedi990","description":"\"A Retrieval-Augmented Generation (RAG) system for document query and summarization using vector-based search and language models.","archived":false,"fork":false,"pushed_at":"2024-12-03T07:41:50.000Z","size":2123,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-23T13:29:57.806Z","etag":null,"topics":["docker","document-query","faiss","fastapi","flask","langugae-models","machine-learning","milvus","natural-language-processing","nlp","pdf-parsing","pinecone","python","qdrant-vector-database","rag","react","retrieval-augmented-generation","summarization","vector-search","weaviate"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sunnybedi990.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-11T07:23:56.000Z","updated_at":"2025-03-29T12:32:55.000Z","dependencies_parsed_at":"2024-11-11T08:29:48.161Z","dependency_job_id":"84966ecd-ca48-4ea2-9263-1c1bc673b48f","html_url":"https://github.com/sunnybedi990/RAG-with-LLM","commit_stats":null,"previous_names":["sunnybedi990/rag-with-llm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunnybedi990%2FRAG-with-LLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunnybedi990%2FRAG-with-LLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunnybedi990%2FRAG-with-LLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunnybedi990%2FRAG-with-LLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sunnybedi990","download_url":"https://codeload.github.com/sunnybedi990/RAG-with-LLM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250440063,"owners_count":21430944,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","document-query","faiss","fastapi","flask","langugae-models","machine-learning","milvus","natural-language-processing","nlp","pdf-parsing","pinecone","python","qdrant-vector-database","rag","react","retrieval-augmented-generation","summarization","vector-search","weaviate"],"created_at":"2024-12-15T14:12:39.516Z","updated_at":"2025-04-23T13:30:08.394Z","avatar_url":"https://github.com/sunnybedi990.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# **RAG System for Document Query and Summarization with Flask or FastAPI**\n\nThis project is a Retrieval-Augmented Generation (RAG) system designed for querying and summarizing documents. It supports GPU and CPU environments, and the backend can run using either Flask (`RAG.py`) or FastAPI (`RAG_fastapi.py`). The frontend provides a user-friendly interface for uploading documents and interacting with the system.\n\n\n\n---\n\n## **Table of Contents**\n\n1. [Overview](#overview)\n2. [Setup and Installation](#setup-and-installation)\n   - [Backend Setup](#backend-setup)\n   - [Frontend Setup](#frontend-setup)\n   - [Docker Setup](#docker-setup)\n3. [Usage](#usage)\n   - [Running the Backend](#running-the-backend)\n   - [Running the Frontend](#running-the-frontend)\n   - [GPU and CPU Configurations](#gpu-and-cpu-configurations)\n   - [Switching Between Flask and FastAPI](#switching-between-flask-and-fastapi)\n4. [Project Structure](#project-structure)\n5. [Key Features](#key-features)\n6. [Troubleshooting](#troubleshooting)\n\n---\n\n## **Overview**\n\nThe RAG system uses vector-based search and large language models (LLMs) to enable:\n- Document uploads and storage.\n- Advanced querying using embeddings.\n- Document summarization.\n\nIt supports various embedding providers (e.g., OpenAI, Llama, Groq) and works in GPU and CPU environments for optimized performance.\n\nThe RAG system now supports multiple vector databases for efficient storage and retrieval of embeddings. A dedicated `Database_Readme.md` file provides detailed instructions for configuring and using each supported database.\n\n---\n\n\n### **Database Configurations**\n\nFor detailed information on supported vector databases (e.g., FAISS, Milvus, Pinecone, Qdrant, Weaviate) and their configurations, see [**Database_Readme.md**](./backend/Database_Readme.md).\n\n---\n\n### **Preview**\n\nBelow is a visual representation of the RAG system architecture and user interface:\n\n#### **User Interface**\n![User Interface Preview](./assets/tesla_query.png)\n\n---\n\n## **Setup and Installation**\n\n### **Prerequisites**\n- **Python 3.11** (with Conda for backend environment management)\n- **Node.js** (for the frontend)\n- **Docker** (optional for containerized deployment)\n- **Git** (for cloning the repository)\n\n---\n\n## **Environment Configuration**\n\nBefore running the backend, create a `.env` file in the `backend` folder and add the following environment variables:\n\n```env\n# Authentication Keys\nLLAMA_CLOUD_API_KEY=          # API key for LLama Cloud\nOPENAI_API_KEY=               # API key for OpenAI\nGROQ_API_KEY=                 # API key for Groq\n\n# Vector Database Configuration\nVECTOR_DB_PATH=vector_dbs/    # Path to store vector databases\nUSE_GPU=true                  # Set to \"true\" for GPU usage or \"false\" for CPU\n\n# Pinecone Configuration\nPINECONE_API_KEY=             # API key for Pinecone\n\n# Weaviate Configuration\nWEAVIATE_API_KEY=             # API key for Weaviate\nWEAVIATE_CLUSTER_URL=         # Cluster URL for Weaviate\n\n# Qdrant Configuration\nQDRANT_API_KEY=               # API key for Qdrant (optional)\nQDRANT_ADMIN_API_KEY=         # Admin API key for Qdrant (optional)\nQDRANT_CLUSTER_URL=           # Cluster URL for Qdrant (optional)\n```\n\n- **Note**: Replace the placeholder values with your actual API keys and paths.\n- Make sure the `VECTOR_DB_PATH` matches the directory where vector databases will be stored.\n\n---\n\n### **Backend Setup**\n\n1. **Navigate to the backend directory**:\n   ```bash\n   cd backend\n   ```\n\n2. **Create and activate a Conda environment**:\n   ```bash\n   conda env create -f src/environment.yml\n   conda activate rag_env\n   ```\n\n3. **Install dependencies**:\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n4. **Run the backend**:\n   - For Flask:\n     ```bash\n     python src/RAG.py\n     ```\n   - For FastAPI:\n     ```bash\n     python src/RAG_fastapi.py\n     ```\n\n---\n\n### **Frontend Setup**\n\n1. **Navigate to the frontend directory**:\n   ```bash\n   cd frontend\n   ```\n\n2. **Install dependencies**:\n   ```bash\n   npm install\n   ```\n\n3. **Start the development server**:\n   ```bash\n   npm start\n   ```\n\nThe frontend will be available at `http://localhost:3000`.\n\n---\n\n### **Docker Setup**\n\nThe project includes Docker support for both GPU and CPU environments.\n\n1. **Build the Docker image**:\n   - **GPU Environment**:\n     ```bash\n     docker build --build-arg USE_GPU=true -f backend/Backend.gpu.dockerfile -t rag-system-gpu .\n     ```\n   - **CPU Environment**:\n     ```bash\n     docker build --build-arg USE_GPU=false -f backend/Backend.cpu.dockerfile -t rag-system-cpu .\n     ```\n\n2. **Run the Docker container**:\n   - **GPU**:\n     ```bash\n     docker-compose -f docker-compose.gpu.yml up\n     ```\n   - **CPU**:\n     ```bash\n     docker-compose -f docker-compose.cpu.yml up\n     ```\n\n---\n\n## **Usage**\n\n### **Running the Backend**\n\nTo start the backend directly without Docker:\n- Flask:\n  ```bash\n  python backend/src/RAG.py\n  ```\n- FastAPI:\n  ```bash\n  python backend/src/RAG_fastapi.py\n  ```\n\n---\n\n### **GPU and CPU Configurations**\n\nThe backend dynamically supports both GPU and CPU setups:\n- For GPU-based setups, ensure `faiss-gpu` is installed and the `USE_GPU` environment variable is set to `true`.\n- For CPU setups, `faiss-cpu` should be installed instead.\n\nIf using Docker, the provided `docker-compose` files will handle these configurations automatically.\n\n---\n\n### **Switching Between Flask and FastAPI**\n\nTo switch between Flask and FastAPI when using Docker:\n- Set the `RAG_SERVER` environment variable:\n  - **Flask**:\n    ```bash\n    docker run -e RAG_SERVER=flask -p 5000:5000 rag-system\n    ```\n  - **FastAPI**:\n    ```bash\n    docker run -e RAG_SERVER=fastapi -p 5000:5000 rag-system\n    ```\n\n---\n\n## **Project Structure**\n\n```\nproject-root/\n│\n├── README.md                       # Main project documentation\n├── LICENSE                         # Project license\n├── .gitignore                      # Git ignore file\n├── Database_Readme.md              # Detailed documentation for database setup\n├── docker-compose.yml              # Default Docker Compose configuration\n├── docker-compose.gpu.yml          # Docker Compose configuration for GPU\n├── docker-compose.cpu.yml          # Docker Compose configuration for CPU\n│\n├── assets/                         # Folder for project assets (images, diagrams, etc.)\n│\n├── data/                           # General data folder\n│\n├── frontend/                       # Frontend (React) directory\n│   ├── rag-ui/                     # React application\n│   │   ├── public/                 # Static assets\n│   │   ├── src/                    # Source code\n│   │   ├── package.json            # Frontend dependencies\n│   │   ├── .env                    # Frontend environment configuration\n│\n├── backend/                        # Backend (Python) directory\n│   ├── __pycache__/                # Python cache files\n│   ├── data/                       # Data-related files\n│   ├── vector_dbs/                 # Vector DB index files\n│   ├── src/                        # Source code\n│   │   ├── __pycache__/            # Python cache files for `src`\n│   │   ├── adapters/               # Vector database adapters\n│   │   │   ├── __pycache__/        # Python cache files for `adapters`\n│   │   │   ├── __init__.py         # Package initialization\n│   │   │   ├── faiss_adapter.py    # FAISS vector DB adapter\n│   │   │   ├── milvus_adapter.py   # Milvus vector DB adapter\n│   │   │   ├── pinecone_adapter.py # Pinecone vector DB adapter\n│   │   │   ├── qdrant_adapter.py   # Qdrant vector DB adapter\n│   │   │   ├── weaviate_adapter.py # Weaviate vector DB adapter\n│   │   ├── add_to_vector_db.py     # Script to add documents to vector DB\n│   │   ├── config.yaml             # Configuration file for databases\n│   │   ├── embedding_config.py     # Embedding model configuration\n│   │   ├── embedding_initializer.py # Embedding model initialization logic\n│   │   ├── environment.yml         # Conda environment file\n│   │   ├── id_map.pkl              # ID mapping for vectors\n│   │   ├── main.py                 # Unified entry point for the backend\n│   │   ├── RAG_fastapi.py          # FastAPI-based backend\n│   │   ├── rag_models.py           # RAG models and processing logic\n│   │   ├── RAG.py                  # Flask-based backend\n│   │   ├── VectorDB.py             # Vector database management logic\n│   ├── .env                        # Backend environment configuration\n│   ├── Backend.cpu.dockerfile      # Dockerfile for CPU-specific setup\n│   ├── Backend.gpu.dockerfile      # Dockerfile for GPU-specific setup\n│   ├── id_map.pkl                  # ID mapping file\n│   ├── requirements.txt            # Backend dependencies\n│   ├── start_server.sh             # Script to dynamically select Flask or FastAPI\n\n```\n\n---\n\n## **Key Features**\n\n- **Flask and FastAPI Support**: Run the backend with Flask (`RAG.py`) or FastAPI (`RAG_fastapi.py`).\n- **GPU and CPU Compatibility**: Dynamically handles GPU or CPU setups based on the environment.\n- **Document Querying**: Upload documents and perform advanced queries using embeddings.\n- **Summarization**: Generate document summaries using vector-based retrieval.\n\n---\n\n## **Troubleshooting**\n\n1. **Permission Errors**:\n   - Ensure the `data/` and `backend/vector_dbs/` directories are writable:\n     ```bash\n     chmod -R 755 backend/vector_dbs\n     chmod -R 755 data\n     ```\n\n2. **Missing Dependencies**:\n   - Install missing Python packages:\n     ```bash\n     pip install -r backend/requirements.txt\n     ```\n\n3. **Docker Issues**:\n   - Check container logs:\n     ```bash\n     docker logs \u003ccontainer_id\u003e\n     ```\n\n4. **CORS Issues**:\n   - Ensure the backend allows requests from the frontend:\n     ```python\n     from fastapi.middleware.cors import CORSMiddleware\n\n     app.add_middleware(\n         CORSMiddleware,\n         allow_origins=[\"http://localhost:3000\"],\n         allow_methods=[\"*\"],\n         allow_headers=[\"*\"],\n     )","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunnybedi990%2Frag-with-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsunnybedi990%2Frag-with-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunnybedi990%2Frag-with-llm/lists"}