{"id":24765626,"url":"https://github.com/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub","last_synced_at":"2025-10-11T14:31:25.584Z","repository":{"id":273904210,"uuid":"921259235","full_name":"RobinMillford/Multi-Model-RAG-Powered-Article-Chatbot","owner":"RobinMillford","description":"This project creates a Retrieve-and-Generate (RAG) powered chatbot for summarizing and interacting with articles. The system processes articles provided as PDFs or URLs, extracts text, splits the content into chunks, generates embeddings, and stores them in a vector database","archived":false,"fork":false,"pushed_at":"2025-01-28T16:43:53.000Z","size":349,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-28T17:36:06.255Z","etag":null,"topics":["article-extractor","chatbot","llama3","llm","pdf-document-processor","rag","streamlit","summarizer","vector-database"],"latest_commit_sha":null,"homepage":"https://multi-model-rag-powered-article-chatbot.streamlit.app/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RobinMillford.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-23T16:25:01.000Z","updated_at":"2025-01-28T16:43:57.000Z","dependencies_parsed_at":"2025-01-28T17:36:10.337Z","dependency_job_id":"40d73cbe-8c87-44b0-b055-655ee5c6465c","html_url":"https://github.com/RobinMillford/Multi-Model-RAG-Powered-Article-Chatbot","commit_stats":null,"previous_names":["robinmillford/llama3-rag-powered-article-chatbot","robinmillford/multi-model-rag-powered-article-chatbot"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinMillford%2FMulti-Model-RAG-Powered-Article-Chatbot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinMillford%2FMulti-Model-RAG-Powered-Article-Chatbot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinMillford%2FMulti-Model-RAG-Powered-Article-Chatbot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinMillford%2FMulti-Model-RAG-Powered-Article-Chatbot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RobinMillford","download_url":"https://codeload.github.com/RobinMillford/Multi-Model-RAG-Powered-Article-Chatbot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236102623,"owners_count":19095206,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["article-extractor","chatbot","llama3","llm","pdf-document-processor","rag","streamlit","summarizer","vector-database"],"created_at":"2025-01-28T23:14:33.571Z","updated_at":"2025-10-11T14:31:25.578Z","avatar_url":"https://github.com/RobinMillford.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cortex AI: Multi-Model Insights Hub\n\n🤖 **Advanced AI-Powered Document Analysis with Multimodal RAG Capabilities**\n\nCortex AI Hub integrates multiple Large Language Models (LLMs) with a sophisticated **Multimodal Retrieve-and-Generate (RAG)** system, enabling you to extract insights from both **text and visual content** in documents.\n\n**✨ NEW: Multimodal Capabilities** - Now with support for images, charts, graphs, and infographics!\n\n---\n\n## 🌟 **Key Features**\n\n### 🖼️ **Multimodal RAG**\n\n- **📊 Visual Content Understanding**: Analyze images, charts, graphs, and infographics\n- **🔗 Unified Text-Image Search**: Search across both textual and visual content\n- **🎯 Context-Aware Analysis**: Enhanced understanding with specialized prompts\n- **💾 Persistent Storage**: Efficient FAISS-based multimodal embeddings\n- **🆓 Free \u0026 Local**: Uses open-source models (BLIP, BLIP-2, GIT, CLIP)\n\n### 🔍 **Advanced Search \u0026 RAG**\n\n- **🧠 Hybrid Search**: Combines semantic vector search with BM25 keyword search\n- **📂 Multi-Document Support**: Upload PDFs or provide URLs\n- **💾 Persistent Vector Database**: ChromaDB-powered storage\n- **✅ Accurate Citations**: Source-linked responses with references\n\n### 🤖 **AI-Powered Search Agent**\n\n- **🌐 Real-Time Research**: ArXiv, Wikipedia, and web search tools\n- **📰 Current Information**: Up-to-date news and research insights\n- **⚡ Instant Responses**: Fast, context-aware answers\n\n---\n\n## 🚀 **Supported AI Models**\n\n| Model                         | Provider | Best For                      |\n| ----------------------------- | -------- | ----------------------------- |\n| llama-3.3-70b-versatile       | Meta     | Complex reasoning, analysis   |\n| llama-3.1-8b-instant          | Meta     | Quick queries, fast responses |\n| deepseek-r1-distill-llama-70b | DeepSeek | Extended conversations        |\n| qwen/qwen3-32b                | Alibaba  | Document summarization        |\n| openai/gpt-oss-120b           | OpenAI   | Complex analysis tasks        |\n\n### 🖼️ **Vision Models**\n\n| Model  | Description            | Best For                     |\n| ------ | ---------------------- | ---------------------------- |\n| BLIP   | Quick image captioning | Speed, basic analysis        |\n| BLIP-2 | Advanced understanding | Complex visual content       |\n| GIT    | Detailed descriptions  | Charts, graphs, infographics |\n\n---\n\n## 📸 **Application Screenshots**\n\n### 🤖 **RAG Chatbot Interface**\n\n![RAG Chatbot Interface](images/Ragbot_interface.png)\n_Traditional RAG chatbot with document upload and multi-LLM selection_\n\n### 🖼️ **Multimodal RAG Interface**\n\n![Multimodal RAG Interface](images/MultiModel_Rag_Interface.png)\n_Enhanced multimodal interface with vision model selection and image analysis_\n\n### 🔍 **Search Agent Interface**\n\n![Search Agent Interface](images/Search_Agent_Interface.png)\n_AI-powered search agent with real-time research capabilities_\n\n---\n\n## 🔄 **System Architecture**\n\n### 📊 **RAG Chatbot Workflow**\n\n![RAG Chatbot Workflow](images/Ragchotbot_diagram.png)\n_Complete RAG chatbot workflow with document processing, hybrid search, and multi-LLM response generation_\n\n### 🤖 **Search Agent Workflow**\n\n![Search Agent Workflow](images/Search_Agent_Diagram.png)\n_AI-powered search agent workflow with multi-tool research and intelligent orchestration_\n\n### 🖼️ **Multimodal RAG Workflow**\n\n![Multimodal RAG Workflow](images/Multimodel_Rag.png)\n_Enhanced multimodal workflow combining text and visual content analysis_\n\n---\n\n## 🚀 **Getting Started**\n\n### 📋 **Prerequisites**\n\n- Python 3.12+\n- Git\n- API Keys: ChatGroq and Tavily\n\n### 📥 **Installation**\n\n1. **Clone Repository**\n\n   ```bash\n   git clone https://github.com/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub.git\n   cd Cortex-AI-Multi-Model-Insights-Hub\n   ```\n\n2. **Setup Environment**\n\n   ```bash\n   python -m venv venv\n   source venv/bin/activate  # Windows: venv\\Scripts\\activate\n   pip install -r requirements.txt\n   ```\n\n3. **Configure API Keys**\n\n   ```bash\n   cp .env.template .env\n   # Add your GROQ_API_KEY and TAVILY_API_KEY to .env\n   ```\n\n4. **Run Application**\n   ```bash\n   streamlit run Main_Page.py\n   ```\n\n### 🌐 **Live Demo**\n\n**[🚀 Try it now](https://cortex-ai-multi-model-insights-app.streamlit.app/)**\n\n---\n\n## 📖 **Usage Guide**\n\n### 🖼️ **Multimodal Document Analysis**\n\n1. Navigate to **\"Multimodal RAG\"** page\n2. Choose vision model (BLIP for speed, GIT for accuracy)\n3. Upload PDF with images/charts\n4. Enable **\"Extract and analyze images\"**\n5. Ask questions about text and visual content\n\n### 📄 **Traditional Document Chat**\n\n1. Go to **\"RAG Chatbot\"** page\n2. Upload PDFs or enter URLs\n3. Configure retrieval parameters\n4. Select LLM models for comparison\n5. Ask questions and get cited responses\n\n### 🔍 **Research \u0026 Web Search**\n\n1. Visit **\"Search Agent\"** page\n2. Enter research queries\n3. Choose preferred LLM model\n4. Get real-time answers with sources\n\n---\n\n## 🛠️ **Technology Stack**\n\n- **Frontend**: Streamlit with dark theme\n- **Backend**: Python, LangChain/LangGraph\n- **Vector DB**: ChromaDB (text), FAISS (multimodal)\n- **Embeddings**: HuggingFace sentence-transformers, CLIP\n- **Vision**: BLIP, BLIP-2, GIT (Hugging Face)\n- **LLMs**: Groq API\n- **Search**: Tavily, ArXiv, Wikipedia APIs\n\n### 📁 **Project Structure**\n\n```\n├── Main_Page.py                 # App entry point\n├── multimodal_helpers.py        # Multimodal processing\n├── helpers.py                   # Text utilities\n├── chain_setup.py               # LLM configuration\n├── pages/\n│   ├── 1_RAG_Chatbot.py        # Traditional RAG\n│   ├── 2_Search_Agent.py       # Web search agent\n│   └── 3_Multimodal_RAG.py     # Multimodal interface\n├── chroma_db/                   # Text vector storage\n├── multimodal_stores/           # Multimodal storage\n└── requirements.txt             # Dependencies\n```\n\n---\n\n## 🔧 **Key Technical Features**\n\n### 🧠 **Architecture Highlights**\n\n- **Two-Layer Vision**: Vision models → descriptions, CLIP → embeddings\n- **Hybrid Search**: Semantic + BM25 for optimal retrieval\n- **Model Caching**: Global cache prevents reloading\n- **Session Management**: Streamlit state for persistence\n\n### ⚡ **Performance Optimizations**\n\n- Vision models cached globally\n- Processed embeddings saved for reuse\n- Lazy loading when needed\n- Real-time progress feedback\n\n---\n\n## 🤝 **Contributing**\n\n1. Fork the repository\n2. Create feature branch: `git checkout -b feature/your-feature`\n3. Make changes and test locally\n4. Commit and push: `git commit -m \"Add feature\"`\n5. Create Pull Request\n\n### 🎯 **Areas for Contribution**\n\n- 🖼️ New vision models or analysis techniques\n- 🔍 Better retrieval algorithms\n- 🎨 UI/UX improvements\n- 📊 Analytics and metrics\n- 🧪 Testing and documentation\n\n---\n\n## 📝 **License**\n\nThis project is licensed under the **AGPL-3.0 License**.\n\n---\n\n## 🆘 **Support**\n\n- **🐛 Issues**: [GitHub Issues](https://github.com/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub/issues)\n- **💬 Discussions**: [GitHub Discussions](https://github.com/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub/discussions)\n\n---\n\n## 🙏 **Acknowledgments**\n\n- **🤗 Hugging Face**: Free open-source vision models\n- **🦙 Meta**: Llama models and CLIP\n- **🔍 Salesforce**: BLIP vision models\n- **🏢 Microsoft**: GIT vision model\n- **⚡ Groq**: Fast LLM inference\n- **🌐 Streamlit**: Amazing app framework\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRobinMillford%2FCortex-AI-Multi-Model-Insights-Hub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRobinMillford%2FCortex-AI-Multi-Model-Insights-Hub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRobinMillford%2FCortex-AI-Multi-Model-Insights-Hub/lists"}