{"id":26493319,"url":"https://github.com/robinmillford/cortex-ai-multi-model-insights-hub","last_synced_at":"2025-10-28T10:06:16.353Z","repository":{"id":281138606,"uuid":"810402283","full_name":"RobinMillford/Cortex-AI-Multi-Model-Insights-Hub","owner":"RobinMillford","description":"Cortex AI: Multi-Model Insights Hub is an advanced platform that leverages cutting-edge AI to empower your research, analysis, and data exploration. By integrating multiple Large Language Models (LLMs) with a sophisticated Retrieve-and-Generate (RAG) system","archived":false,"fork":false,"pushed_at":"2025-09-03T06:14:12.000Z","size":1436,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-10-11T14:32:24.658Z","etag":null,"topics":["article-extractor","chatbot","data-analysis","data-visualization","deepseek-chat","deepseek-r1","llama3","llm","pdf-document-processor","rag","streamlit-webapp","summarizer","vector-database"],"latest_commit_sha":null,"homepage":"https://cortex-ai-multi-model-insights-app.streamlit.app/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RobinMillford.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-06-04T16:15:05.000Z","updated_at":"2025-09-03T06:14:15.000Z","dependencies_parsed_at":"2025-10-11T14:32:26.181Z","dependency_job_id":"73ce16fb-453f-4f6e-9074-ab7eef3aec22","html_url":"https://github.com/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub","commit_stats":null,"previous_names":["robinmillford/cortex-ai-multi-model-insights-hub","robinmillford/multi-model-rag-powered-article-chatbot"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinMillford%2FCortex-AI-Multi-Model-Insights-Hub","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinMillford%2FCortex-AI-Multi-Model-Insights-Hub/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinMillford%2FCortex-AI-Multi-Model-Insights-Hub/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinMillford%2FCortex-AI-Multi-Model-Insights-Hub/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RobinMillford","download_url":"https://codeload.github.com/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RobinMillford%2FCortex-AI-Multi-Model-Insights-Hub/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281418372,"owners_count":26497783,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-28T02:00:06.022Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["article-extractor","chatbot","data-analysis","data-visualization","deepseek-chat","deepseek-r1","llama3","llm","pdf-document-processor","rag","streamlit-webapp","summarizer","vector-database"],"created_at":"2025-03-20T09:52:17.561Z","updated_at":"2025-10-28T10:06:16.348Z","avatar_url":"https://github.com/RobinMillford.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cortex AI: Multi-Model Insights Hub\n\n🤖 **Advanced AI-Powered Document Analysis with Multimodal RAG Capabilities**\n\nCortex AI Hub integrates multiple Large Language Models (LLMs) with a sophisticated **Multimodal Retrieve-and-Generate (RAG)** system, enabling you to extract insights from both **text and visual content** in documents.\n\n**✨ NEW: Multimodal Capabilities** - Now with support for images, charts, graphs, and infographics!\n\n---\n\n## 🌟 **Key Features**\n\n### 🖼️ **Multimodal RAG**\n\n- **📊 Visual Content Understanding**: Analyze images, charts, graphs, and infographics\n- **🔗 Unified Text-Image Search**: Search across both textual and visual content\n- **🎯 Context-Aware Analysis**: Enhanced understanding with specialized prompts\n- **💾 Persistent Storage**: Efficient FAISS-based multimodal embeddings\n- **🆓 Free \u0026 Local**: Uses open-source models (BLIP, BLIP-2, GIT, CLIP)\n\n### 🔍 **Advanced Search \u0026 RAG**\n\n- **🧠 Hybrid Search**: Combines semantic vector search with BM25 keyword search\n- **📂 Multi-Document Support**: Upload PDFs or provide URLs\n- **💾 Persistent Vector Database**: ChromaDB-powered storage\n- **✅ Accurate Citations**: Source-linked responses with references\n\n### 🤖 **AI-Powered Search Agent**\n\n- **🌐 Real-Time Research**: ArXiv, Wikipedia, and web search tools\n- **📰 Current Information**: Up-to-date news and research insights\n- **⚡ Instant Responses**: Fast, context-aware answers\n\n---\n\n## 🚀 **Supported AI Models**\n\n| Model                         | Provider | Best For                      |\n| ----------------------------- | -------- | ----------------------------- |\n| llama-3.3-70b-versatile       | Meta     | Complex reasoning, analysis   |\n| llama-3.1-8b-instant          | Meta     | Quick queries, fast responses |\n| deepseek-r1-distill-llama-70b | DeepSeek | Extended conversations        |\n| qwen/qwen3-32b                | Alibaba  | Document summarization        |\n| openai/gpt-oss-120b           | OpenAI   | Complex analysis tasks        |\n\n### 🖼️ **Vision Models**\n\n| Model  | Description            | Best For                     |\n| ------ | ---------------------- | ---------------------------- |\n| BLIP   | Quick image captioning | Speed, basic analysis        |\n| BLIP-2 | Advanced understanding | Complex visual content       |\n| GIT    | Detailed descriptions  | Charts, graphs, infographics |\n\n---\n\n## 📸 **Application Screenshots**\n\n### 🤖 **RAG Chatbot Interface**\n\n![RAG Chatbot Interface](images/Ragbot_interface.png)\n_Traditional RAG chatbot with document upload and multi-LLM selection_\n\n### 🖼️ **Multimodal RAG Interface**\n\n![Multimodal RAG Interface](images/MultiModel_Rag_Interface.png)\n_Enhanced multimodal interface with vision model selection and image analysis_\n\n### 🔍 **Search Agent Interface**\n\n![Search Agent Interface](images/Search_Agent_Interface.png)\n_AI-powered search agent with real-time research capabilities_\n\n---\n\n## 🔄 **System Architecture**\n\n### 📊 **RAG Chatbot Workflow**\n\n![RAG Chatbot Workflow](images/Ragchotbot_diagram.png)\n_Complete RAG chatbot workflow with document processing, hybrid search, and multi-LLM response generation_\n\n### 🤖 **Search Agent Workflow**\n\n![Search Agent Workflow](images/Search_Agent_Diagram.png)\n_AI-powered search agent workflow with multi-tool research and intelligent orchestration_\n\n### 🖼️ **Multimodal RAG Workflow**\n\n![Multimodal RAG Workflow](images/Multimodel_Rag.png)\n_Enhanced multimodal workflow combining text and visual content analysis_\n\n---\n\n## 🚀 **Getting Started**\n\n### 📋 **Prerequisites**\n\n- Python 3.12+\n- Git\n- API Keys: ChatGroq and Tavily\n\n### 📥 **Installation**\n\n1. **Clone Repository**\n\n   ```bash\n   git clone https://github.com/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub.git\n   cd Cortex-AI-Multi-Model-Insights-Hub\n   ```\n\n2. **Setup Environment**\n\n   ```bash\n   python -m venv venv\n   source venv/bin/activate  # Windows: venv\\Scripts\\activate\n   pip install -r requirements.txt\n   ```\n\n3. **Configure API Keys**\n\n   ```bash\n   cp .env.template .env\n   # Add your GROQ_API_KEY and TAVILY_API_KEY to .env\n   ```\n\n4. **Run Application**\n   ```bash\n   streamlit run Main_Page.py\n   ```\n\n### 🌐 **Live Demo**\n\n**[🚀 Try it now](https://cortex-ai-multi-model-insights-app.streamlit.app/)**\n\n---\n\n## 📖 **Usage Guide**\n\n### 🖼️ **Multimodal Document Analysis**\n\n1. Navigate to **\"Multimodal RAG\"** page\n2. Choose vision model (BLIP for speed, GIT for accuracy)\n3. Upload PDF with images/charts\n4. Enable **\"Extract and analyze images\"**\n5. Ask questions about text and visual content\n\n### 📄 **Traditional Document Chat**\n\n1. Go to **\"RAG Chatbot\"** page\n2. Upload PDFs or enter URLs\n3. Configure retrieval parameters\n4. Select LLM models for comparison\n5. Ask questions and get cited responses\n\n### 🔍 **Research \u0026 Web Search**\n\n1. Visit **\"Search Agent\"** page\n2. Enter research queries\n3. Choose preferred LLM model\n4. Get real-time answers with sources\n\n---\n\n## 🛠️ **Technology Stack**\n\n- **Frontend**: Streamlit with dark theme\n- **Backend**: Python, LangChain/LangGraph\n- **Vector DB**: ChromaDB (text), FAISS (multimodal)\n- **Embeddings**: HuggingFace sentence-transformers, CLIP\n- **Vision**: BLIP, BLIP-2, GIT (Hugging Face)\n- **LLMs**: Groq API\n- **Search**: Tavily, ArXiv, Wikipedia APIs\n\n### 📁 **Project Structure**\n\n```\n├── Main_Page.py                 # App entry point\n├── multimodal_helpers.py        # Multimodal processing\n├── helpers.py                   # Text utilities\n├── chain_setup.py               # LLM configuration\n├── pages/\n│   ├── 1_RAG_Chatbot.py        # Traditional RAG\n│   ├── 2_Search_Agent.py       # Web search agent\n│   └── 3_Multimodal_RAG.py     # Multimodal interface\n├── chroma_db/                   # Text vector storage\n├── multimodal_stores/           # Multimodal storage\n└── requirements.txt             # Dependencies\n```\n\n---\n\n## 🔧 **Key Technical Features**\n\n### 🧠 **Architecture Highlights**\n\n- **Two-Layer Vision**: Vision models → descriptions, CLIP → embeddings\n- **Hybrid Search**: Semantic + BM25 for optimal retrieval\n- **Model Caching**: Global cache prevents reloading\n- **Session Management**: Streamlit state for persistence\n\n### ⚡ **Performance Optimizations**\n\n- Vision models cached globally\n- Processed embeddings saved for reuse\n- Lazy loading when needed\n- Real-time progress feedback\n\n---\n\n## 🤝 **Contributing**\n\n1. Fork the repository\n2. Create feature branch: `git checkout -b feature/your-feature`\n3. Make changes and test locally\n4. Commit and push: `git commit -m \"Add feature\"`\n5. Create Pull Request\n\n### 🎯 **Areas for Contribution**\n\n- 🖼️ New vision models or analysis techniques\n- 🔍 Better retrieval algorithms\n- 🎨 UI/UX improvements\n- 📊 Analytics and metrics\n- 🧪 Testing and documentation\n\n---\n\n## 📝 **License**\n\nThis project is licensed under the **AGPL-3.0 License**.\n\n---\n\n## 🆘 **Support**\n\n- **🐛 Issues**: [GitHub Issues](https://github.com/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub/issues)\n- **💬 Discussions**: [GitHub Discussions](https://github.com/RobinMillford/Cortex-AI-Multi-Model-Insights-Hub/discussions)\n\n---\n\n## 🙏 **Acknowledgments**\n\n- **🤗 Hugging Face**: Free open-source vision models\n- **🦙 Meta**: Llama models and CLIP\n- **🔍 Salesforce**: BLIP vision models\n- **🏢 Microsoft**: GIT vision model\n- **⚡ Groq**: Fast LLM inference\n- **🌐 Streamlit**: Amazing app framework\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinmillford%2Fcortex-ai-multi-model-insights-hub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobinmillford%2Fcortex-ai-multi-model-insights-hub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinmillford%2Fcortex-ai-multi-model-insights-hub/lists"}