{"id":27854166,"url":"https://github.com/analyticace/retrieval-augmented-generation-workshop","last_synced_at":"2025-05-04T09:00:17.315Z","repository":{"id":287914524,"uuid":"966222848","full_name":"AnalyticAce/Retrieval-Augmented-Generation-Workshop","owner":"AnalyticAce","description":"This workshop demonstrates how to build an intelligent document Q\u0026A system using Retrieval-Augmented Generation (RAG).","archived":false,"fork":false,"pushed_at":"2025-04-26T07:46:54.000Z","size":6124,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-26T08:32:00.491Z","etag":null,"topics":["deepseek","llm-inference","rag-chatbot"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AnalyticAce.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-14T15:34:44.000Z","updated_at":"2025-04-26T07:46:57.000Z","dependencies_parsed_at":"2025-04-14T16:46:36.067Z","dependency_job_id":"5bb8e10e-40e0-4075-b43f-cf35ecbbd90a","html_url":"https://github.com/AnalyticAce/Retrieval-Augmented-Generation-Workshop","commit_stats":null,"previous_names":["analyticace/retrieval-augmented-generation-workshop"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnalyticAce%2FRetrieval-Augmented-Generation-Workshop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnalyticAce%2FRetrieval-Augmented-Generation-Workshop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnalyticAce%2FRetrieval-Augmented-Generation-Workshop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnalyticAce%2FRetrieval-Augmented-Generation-Workshop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AnalyticAce","download_url":"https://codeload.github.com/AnalyticAce/Retrieval-Augmented-Generation-Workshop/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252310952,"owners_count":21727516,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deepseek","llm-inference","rag-chatbot"],"created_at":"2025-05-04T09:00:16.618Z","updated_at":"2025-05-04T09:00:17.239Z","avatar_url":"https://github.com/AnalyticAce.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Retrieval-Augmented Generation Workshop\n\nThis workshop demonstrates how to build an intelligent document Q\u0026A system using Retrieval-Augmented Generation (RAG).\n\n## What is RAG and Why It's Important ?\nRAG is a framework that combines information retrieval with text generation to create AI systems that produce more accurate, factual, and contextually relevant responses.\n\n### How RAG Works\n1. Retrieval Phase: When a user asks a question, RAG first searches through a knowledge base (documents, databases, etc.) to find the most relevant information.\n\n2. Augmentation Phase: The retrieved information is injected as context into the prompt sent to a large language model (LLM).\n\n3. Generation Phase: The LLM generates a response based on both its pre-trained knowledge and the specific retrieved context.\n\n![Diagram](assets/diagram1.png)\n\n## Why RAG is Important\n\n1. Improved Accuracy: By grounding responses in specific source material, RAG reduces hallucinations and factual errors common in pure LLM responses.\n\n2. Up-to-date Information: RAG can access the latest information from your documents, overcoming the limitation of LLMs being trained on older data.\n\n3. Domain Specificity: Enables LLMs to answer questions about specialized or proprietary information not present in their training data.\n\n4. Source Attribution: RAG can provide references to the source documents, increasing transparency and trustworthiness.\n\n5. Lower Costs: Smaller, more efficient models can be used since they're augmented with external knowledge rather than needing to memorize everything.\n\n6. Privacy \u0026 Control: Organizations can keep sensitive information in their own knowledge base rather than sending it to external LLM providers.\n\nRAG represents an important evolution in AI systems, creating a bridge between traditional search and modern generative AI to deliver more reliable and useful responses.\n\n\n## Workshop/Project Overview\n\nThis project implements a RAG-based system that allows users to:\n1. Upload PDF documents\n2. Process them into searchable vector stores\n3. Ask questions about the documents using natural language\n4. Receive contextually relevant answers based on the document content\n\n## Features\n\n- **PDF Document Processing**: Upload and process PDF documents \n- **Vector Embedding**: Convert document chunks into embeddings using Hugging Face models\n- **Intelligent Retrieval**: Find the most relevant document sections for user queries\n- **Natural Language Responses**: Generate human-like responses using Large Language Models\n- **Interactive UI**: Easy-to-use Streamlit interface for document uploading and querying\n\n## Architecture\n\nThe project consists of three main components:\n\n1. **Core RAG Engine** (core.py):\n   - Document loading and chunking\n   - Vector embedding generation\n   - Retrieval system\n   - LLM integration\n\n2. **Web Interface** (streamlit_app.py):\n   - Document upload interface\n   - Chat interface for asking questions\n   - Session state management\n\n3. **Colab Notebook** (RAG_(Retrieval_Augmented_Generation).ipynb):\n   - Step-by-step tutorial implementation\n   - Detailed explanations of each RAG component\n   - Executable demonstration\n\n## Getting Started\n\n### Prerequisites\n\n- Python 3.13+\n- HuggingFace account and API token\n- DeepSeek API key (or compatible LLM API)\n\n### Installation\n\n1. Clone the repository:\n   ```\n   git clone https://github.com/AnalyticAce/Retrieval-Augmented-Generation-Workshop.git\n   cd Retrieval-Augmented-Generation-Workshop\n   ```\n\n2. Install dependencies:\n   ```\n   pip install -r requirements.txt\n   ```\n\n3. Create a .env file with your API keys:\n   ```\n   HF_TOKEN=\"your_huggingface_token\"\n   DEEPSEEK_API_KEY=\"your_deepseek_api_key\"\n   DEEPSEEK_API_BASE=\"https://api.deepseek.com/v1\"\n   ```\n\n### Running the Application\n\nStart the Streamlit web interface:\n```\nstreamlit run streamlit_app.py\n```\n\n## Usage\n\n1. Open the web interface in your browser (typically http://localhost:8501)\n2. Upload a PDF document using the sidebar\n3. Click \"Process Document\" to analyze the document\n4. Ask questions in the chat interface\n5. View the AI-generated answers based on document content\n\n### Streamlit UI\n![UI](assets/interphase.png)\n\n## Technical Details\n\n- **Document Chunking**: Uses RecursiveCharacterTextSplitter with chunk size of 10,000 characters\n- **Embeddings**: Generated using Hugging Face's \"all-MiniLM-L6-v2\" model\n- **Vector Database**: FAISS for efficient similarity search\n- **LLM**: DeepSeek Chat model (can be configured to use other LLMs)\n- **Framework**: Built on LangChain for component orchestration\n\n## Workshop Content\n\nThe colab notebook.ipynb) walks through:\n\n1. Environment setup and dependency installation\n2. Data ingestion and preprocessing techniques\n3. Vector store creation with different embedding options\n4. LLM configuration and RAG chain setup\n5. Query processing and response generation\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanalyticace%2Fretrieval-augmented-generation-workshop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fanalyticace%2Fretrieval-augmented-generation-workshop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fanalyticace%2Fretrieval-augmented-generation-workshop/lists"}