{"id":24634504,"url":"https://github.com/rag-web-ui/rag-web-ui","last_synced_at":"2025-05-14T06:05:58.895Z","repository":{"id":273539914,"uuid":"916386111","full_name":"rag-web-ui/rag-web-ui","owner":"rag-web-ui","description":"RAG Web UI is an intelligent dialogue system based on RAG (Retrieval-Augmented Generation) technology.","archived":false,"fork":false,"pushed_at":"2025-04-27T06:27:16.000Z","size":6175,"stargazers_count":2367,"open_issues_count":23,"forks_count":218,"subscribers_count":23,"default_branch":"main","last_synced_at":"2025-04-27T07:26:49.813Z","etag":null,"topics":["ai","deepseek","langchain","ollama","rag"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rag-web-ui.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["johannlai"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"lfx_crowdfunding":null,"polar":null,"buy_me_a_coffee":null,"thanks_dev":null,"custom":null}},"created_at":"2025-01-14T01:48:30.000Z","updated_at":"2025-04-27T06:27:20.000Z","dependencies_parsed_at":null,"dependency_job_id":"675ad381-315d-47ba-a768-8ca6ebe2f725","html_url":"https://github.com/rag-web-ui/rag-web-ui","commit_stats":null,"previous_names":["rag-web-ui/rag-web-ui"],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rag-web-ui%2Frag-web-ui","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rag-web-ui%2Frag-web-ui/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rag-web-ui%2Frag-web-ui/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rag-web-ui%2Frag-web-ui/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rag-web-ui","download_url":"https://codeload.github.com/rag-web-ui/rag-web-ui/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254078659,"owners_count":22011014,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","deepseek","langchain","ollama","rag"],"created_at":"2025-01-25T09:12:54.142Z","updated_at":"2025-05-14T06:05:58.884Z","avatar_url":"https://github.com/rag-web-ui.png","language":"TypeScript","funding_links":["https://github.com/sponsors/johannlai"],"categories":["TypeScript","A01_文本生成_文本对话","12. User Interfaces \u0026 Self-hosted Platforms"],"sub_categories":["大语言对话模型及数据"],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"./docs/images/github-cover-new.png\" alt=\"RAG Web UI Demo\"\u003e\n  \u003cbr /\u003e\n  \u003cp\u003e\n    \u003cstrong\u003eKnowledge Base Management Based on RAG (Retrieval-Augmented Generation)\u003c/strong\u003e\n  \u003c/p\u003e\n\n  \u003cp\u003e\n    \u003ca href=\"https://github.com/rag-web-ui/rag-web-ui/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/rag-web-ui/rag-web-ui\" alt=\"License\"\u003e\u003c/a\u003e\n    \u003ca href=\"#\"\u003e\u003cimg src=\"https://img.shields.io/badge/python-3.9+-blue.svg\" alt=\"Python\"\u003e\u003c/a\u003e\n    \u003ca href=\"#\"\u003e\u003cimg src=\"https://img.shields.io/badge/node-%3E%3D18-green.svg\" alt=\"Node\"\u003e\u003c/a\u003e\n    \u003ca href=\"#\"\u003e\u003cimg src=\"https://img.shields.io/badge/PRs-welcome-brightgreen.svg\" alt=\"PRs Welcome\"\u003e\u003c/a\u003e\n    \u003ca href=\"#\"\u003e\u003cimg src=\"https://github.com/rag-web-ui/rag-web-ui/actions/workflows/test.yml/badge.svg\" alt=\"CI\"\u003e\u003c/a\u003e\n  \u003c/p\u003e\n\n  \u003cp\u003e\n    \u003ca href=\"#features\"\u003eFeatures\u003c/a\u003e •\n    \u003ca href=\"#quick-start\"\u003eQuick Start\u003c/a\u003e •\n    \u003ca href=\"#deployment-guide\"\u003eDeployment\u003c/a\u003e •\n    \u003ca href=\"#architecture\"\u003eArchitecture\u003c/a\u003e •\n    \u003ca href=\"#development\"\u003eDevelopment\u003c/a\u003e •\n    \u003ca href=\"#contributing\"\u003eContributing\u003c/a\u003e •\n    \u003ca href=\"https://deepwiki.com/rag-web-ui/rag-web-ui\" target=\"_blank\"\u003eDeepWiki\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\n## 📖 Introduction\nRAG Web UI is an intelligent dialogue system based on RAG (Retrieval-Augmented Generation) technology that helps build intelligent Q\u0026A systems based on your own knowledge base. By combining document retrieval and large language models, it achieves accurate and reliable knowledge-based question answering services.\n\nThe system supports multiple **LLM** deployment options, including cloud services like **OpenAI** and **DeepSeek**, as well as local model deployment through **Ollama**, meeting privacy and cost requirements in different scenarios.\n\nIt also provides OpenAPI interfaces for convenient knowledge base access via API calls.\n\n## ✨ Features\n- 📚 **Intelligent Document Management**\n  - Support for multiple document formats (PDF, DOCX, Markdown, Text)\n  - Automatic document chunking and vectorization\n  - Support for async document processing and incremental updates\n\n- 🤖 **Advanced Dialogue Engine**\n  - Precise retrieval and generation based on RAG\n  - Support for multi-turn contextual dialogue\n  - Support for reference citations in conversations\n\n- 🎯 **Robust Architecture**\n  - Frontend-backend separation design\n  - Distributed file storage\n  - High-performance vector database: Support for ChromaDB, Qdrant with easy switching through Factory pattern\n\n## 🖼️ Screenshots\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"./docs/images/screenshot1.png\" alt=\"Knowledge Base Management\" width=\"800\"\u003e\n  \u003cp\u003e\u003cem\u003eKnowledge Base Management Dashboard\u003c/em\u003e\u003c/p\u003e\n  \n  \u003cimg src=\"./docs/images/screenshot2.png\" alt=\"Chat Interface\" width=\"800\"\u003e\n  \u003cp\u003e\u003cem\u003eDocument Processing Dashboard\u003c/em\u003e\u003c/p\u003e\n  \n  \u003cimg src=\"./docs/images/screenshot3.png\" alt=\"Document Processing\" width=\"800\"\u003e\n  \u003cp\u003e\u003cem\u003eDocument List\u003c/em\u003e\u003c/p\u003e\n  \n  \u003cimg src=\"./docs/images/screenshot4.png\" alt=\"System Settings\" width=\"800\"\u003e\n  \u003cp\u003e\u003cem\u003eIntelligent Chat Interface with References\u003c/em\u003e\u003c/p\u003e\n  \n  \u003cimg src=\"./docs/images/screenshot5.png\" alt=\"Analytics Dashboard\" width=\"800\"\u003e\n  \u003cp\u003e\u003cem\u003eAPI Key Management\u003c/em\u003e\u003c/p\u003e\n\n  \u003cimg src=\"./docs/images/screenshot6.png\" alt=\"Analytics Dashboard\" width=\"800\"\u003e\n  \u003cp\u003e\u003cem\u003eAPI Reference\u003c/em\u003e\u003c/p\u003e\n\u003c/div\u003e\n\n ##  Project Flowchart\n \n```mermaid\ngraph TB\n    %% Role Definitions\n    client[\"Caller/User\"]\n    open_api[\"Open API\"]\n    \n    subgraph import_process[\"Document Ingestion Process\"]\n        direction TB\n        %% File Storage and Document Processing Flow\n        docs[\"Document Input\u003cbr/\u003e(PDF/MD/TXT/DOCX)\"]\n        job_id[\"Return Job ID\"]\n        \n        nfs[\"NFS\"]\n\n        subgraph async_process[\"Asynchronous Document Processing\"]\n            direction TB\n            preprocess[\"Document Preprocessing\u003cbr/\u003e(Text Extraction/Cleaning)\"]\n            split[\"Text Splitting\u003cbr/\u003e(Segmentation/Overlap)\"]\n            \n            subgraph embedding_process[\"Embedding Service\"]\n                direction LR\n                embedding_api[\"Embedding API\"] --\u003e embedding_server[\"Embedding Server\"]\n            end\n            \n            store[(Vector Database)]\n            \n            %% Internal Flow of Asynchronous Processing\n            preprocess --\u003e split\n            split --\u003e embedding_api\n            embedding_server --\u003e store\n        end\n        \n        subgraph job_query[\"Job Status Query\"]\n            direction TB\n            job_status[\"Job Status\u003cbr/\u003e(Processing/Completed/Failed)\"]\n        end\n    end\n    \n    %% Query Service Flow  \n    subgraph query_process[\"Query Service\"]\n        direction LR\n        user_history[\"User History\"] --\u003e query[\"User Query\u003cbr/\u003e(Based on User History)\"]\n        query --\u003e query_embed[\"Query Embedding\"]\n        query_embed --\u003e retrieve[\"Vector Retrieval\"]\n        retrieve --\u003e rerank[\"Re-ranking\u003cbr/\u003e(Cross-Encoder)\"]\n        rerank --\u003e context[\"Context Assembly\"]\n        context --\u003e llm[\"LLM Generation\"]\n        llm --\u003e response[\"Final Response\"]\n        query -.-\u003e rerank\n    end\n    \n    %% Main Flow Connections\n    client --\u003e |\"1.Upload Document\"| docs\n    docs --\u003e |\"2.Generate\"| job_id\n    docs --\u003e |\"3a.Trigger\"| async_process\n    job_id --\u003e |\"3b.Return\"| client\n    docs --\u003e nfs\n    nfs --\u003e preprocess\n\n    %% Open API Retrieval Flow\n    open_api --\u003e |\"Retrieve Context\"| retrieval_service[\"Retrieval Service\"]\n    retrieval_service --\u003e |\"Access\"| store\n    retrieval_service --\u003e |\"Return Context\"| open_api\n\n    %% Status Query Flow\n    client --\u003e |\"4.Poll\"| job_status\n    job_status --\u003e |\"5.Return Progress\"| client\n    \n    %% Database connects to Query Service\n    store --\u003e retrieve\n\n    %% Style Definitions (Adjusted to match GitHub theme colors)\n    classDef process fill:#d1ecf1,stroke:#0077b6,stroke-width:1px\n    classDef database fill:#e2eafc,stroke:#003566,stroke-width:1px\n    classDef input fill:#caf0f8,stroke:#0077b6,stroke-width:1px\n    classDef output fill:#ffc8dd,stroke:#d00000,stroke-width:1px\n    classDef rerank fill:#cdb4db,stroke:#5a189a,stroke-width:1px\n    classDef async fill:#f8edeb,stroke:#7f5539,stroke-width:1px,stroke-dasharray: 5 5\n    classDef actor fill:#fefae0,stroke:#606c38,stroke-width:1px\n    classDef jobQuery fill:#ffedd8,stroke:#ca6702,stroke-width:1px\n    classDef queryProcess fill:#d8f3dc,stroke:#40916c,stroke-width:1px\n    classDef embeddingService fill:#ffe5d9,stroke:#9d0208,stroke-width:1px\n    classDef importProcess fill:#e5e5e5,stroke:#495057,stroke-width:1px\n\n    %% Applying classes to nodes\n    class docs,query,retrieval_service input\n    class preprocess,split,query_embed,retrieve,context,llm process\n    class store,nfs database\n    class response,job_id,job_status output\n    class rerank rerank\n    class async_process async\n    class client,open_api actor\n    class job_query jobQuery\n    style query_process fill:#d8f3dc,stroke:#40916c,stroke-width:1px\n    style embedding_process fill:#ffe5d9,stroke:#9d0208,stroke-width:1px\n    style import_process fill:#e5e5e5,stroke:#495057,stroke-width:1px\n    style job_query fill:#ffedd8,stroke:#ca6702,stroke-width:1px\n```\n\n## 🚀 Quick Start\n\n### Prerequisites\n\n- Docker \u0026 Docker Compose v2.0+\n- Node.js 18+\n- Python 3.9+\n- 8GB+ RAM\n\n### Installation\n\n1. Clone the repository\n```bash\ngit clone https://github.com/rag-web-ui/rag-web-ui.git\ncd rag-web-ui\n```\n\n2. Configure environment variables\n\nYou can check the details in the configuration table below.\n\n```bash\ncp .env.example .env\n```\n\n3. Start services(development server)\n```bash\ndocker compose up -d --build\n```\n\n### Verification\n\nAccess the following URLs after service startup:\n\n- 🌐 Frontend UI: http://127.0.0.1.nip.io\n- 📚 API Documentation: http://127.0.0.1.nip.io/redoc\n- 💾 MinIO Console: http://127.0.0.1.nip.io:9001\n\n## 🏗️ Architecture\n\n### Backend Stack\n\n- 🐍 **Python FastAPI**: High-performance async web framework\n- 🗄️ **MySQL + ChromaDB**: Relational + Vector databases\n- 📦 **MinIO**: Distributed object storage\n- 🔗 **Langchain**: LLM application framework\n- 🔒 **JWT + OAuth2**: Authentication\n\n### Frontend Stack\n\n- ⚛️ **Next.js 14**: React framework\n- 📘 **TypeScript**: Type safety\n- 🎨 **Tailwind CSS**: Utility-first CSS\n- 🎯 **Shadcn/UI**: High-quality components\n- 🤖 **Vercel AI SDK**: AI integration\n\n## 📈 Performance Optimization\n\nThe system is optimized in the following aspects:\n\n- ⚡️ Incremental document processing and async chunking\n- 🔄 Streaming responses and real-time feedback\n- 📑 Vector database performance tuning\n- 🎯 Distributed task processing\n\n## 📖 Development Guide\n\n```bash\ndocker compose -f docker-compose.dev.yml up -d --build\n```\n\n## 🔧 Configuration\n\n### Core Configuration\n\n| Parameter                   | Description                | Default   | Required |\n| --------------------------- | -------------------------- | --------- | -------- |\n| MYSQL_SERVER                | MySQL Server Address       | localhost | ✅        |\n| MYSQL_USER                  | MySQL Username             | postgres  | ✅        |\n| MYSQL_PASSWORD              | MySQL Password             | postgres  | ✅        |\n| MYSQL_DATABASE              | MySQL Database Name        | ragwebui  | ✅        |\n| SECRET_KEY                  | JWT Secret Key             | -         | ✅        |\n| ACCESS_TOKEN_EXPIRE_MINUTES | JWT Token Expiry (minutes) | 30        | ✅        |\n\n### LLM Configuration\n\n| Parameter         | Description           | Default                   | Applicable            |\n| ----------------- | --------------------- | ------------------------- | --------------------- |\n| CHAT_PROVIDER     | LLM Service Provider  | openai                    | ✅                     |\n| OPENAI_API_KEY    | OpenAI API Key        | -                         | Required for OpenAI   |\n| OPENAI_API_BASE   | OpenAI API Base URL   | https://api.openai.com/v1 | Optional for OpenAI   |\n| OPENAI_MODEL      | OpenAI Model Name     | gpt-4                     | Required for OpenAI   |\n| DEEPSEEK_API_KEY  | DeepSeek API Key      | -                         | Required for DeepSeek |\n| DEEPSEEK_API_BASE | DeepSeek API Base URL | -                         | Required for DeepSeek |\n| DEEPSEEK_MODEL    | DeepSeek Model Name   | -                         | Required for DeepSeek |\n| OLLAMA_API_BASE   | Ollama API Base URL   | http://localhost:11434    | Required for Ollama   |\n| OLLAMA_MODEL      | Ollama Model Name     | llama2                    | Required for Ollama   |\n\n### Embedding Configuration\n\n| Parameter                   | Description                | Default                | Applicable                    |\n| --------------------------- | -------------------------- | ---------------------- | ----------------------------- |\n| EMBEDDINGS_PROVIDER         | Embedding Service Provider | openai                 | ✅                             |\n| OPENAI_API_KEY              | OpenAI API Key             | -                      | Required for OpenAI Embedding |\n| OPENAI_EMBEDDINGS_MODEL     | OpenAI Embedding Model     | text-embedding-ada-002 | Required for OpenAI Embedding |\n| DASH_SCOPE_API_KEY          | DashScope API Key          | -                      | Required for DashScope        |\n| DASH_SCOPE_EMBEDDINGS_MODEL | DashScope Embedding Model  | -                      | Required for DashScope        |\n| OLLAMA_EMBEDDINGS_MODEL     | Ollama Embedding Model     | deepseek-r1:7b         | Required for Ollama Embedding |\n\n### Vector Database Configuration\n\n| Parameter          | Description                       | Default               | Applicable            |\n| ------------------ | --------------------------------- | --------------------- | --------------------- |\n| VECTOR_STORE_TYPE  | Vector Store Type                 | chroma                | ✅                     |\n| CHROMA_DB_HOST     | ChromaDB Server Address           | localhost             | Required for ChromaDB |\n| CHROMA_DB_PORT     | ChromaDB Port                     | 8000                  | Required for ChromaDB |\n| QDRANT_URL         | Qdrant Vector Store URL           | http://localhost:6333 | Required for Qdrant   |\n| QDRANT_PREFER_GRPC | Prefer gRPC Connection for Qdrant | true                  | Optional for Qdrant   |\n\n### Object Storage Configuration\n\n| Parameter         | Description          | Default        | Required |\n| ----------------- | -------------------- | -------------- | -------- |\n| MINIO_ENDPOINT    | MinIO Server Address | localhost:9000 | ✅        |\n| MINIO_ACCESS_KEY  | MinIO Access Key     | minioadmin     | ✅        |\n| MINIO_SECRET_KEY  | MinIO Secret Key     | minioadmin     | ✅        |\n| MINIO_BUCKET_NAME | MinIO Bucket Name    | documents      | ✅        |\n\n### Other Configuration\n\n| Parameter | Description      | Default       | Required |\n| --------- | ---------------- | ------------- | -------- |\n| TZ        | Timezone Setting | Asia/Shanghai | ❌        |\n\n## 🤝 Contributing\n\nWe welcome community contributions!\n\n### Contribution Process\n\n1. Fork the repository\n2. Create a feature branch (`git checkout -b feature/AmazingFeature`)\n3. Commit changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to branch (`git push origin feature/AmazingFeature`)\n5. Create a Pull Request\n\n### Development Guidelines\n\n- Follow [Python PEP 8](https://pep8.org/) coding standards\n- Follow [Conventional Commits](https://www.conventionalcommits.org/)\n\n### 🚧 Roadmap\n\n- [x] Knowledge Base API Integration\n- [ ] Workflow By Natural Language\n- [ ] Multi-path Retrieval\n- [x] Support Multiple Models\n- [x] Support Multiple Vector Databases\n\n## 🔧 Troubleshooting\n\nFor common issues and solutions, please refer to our [Troubleshooting Guide](docs/troubleshooting.md).\n\n## 📄 License\n\nThis project is licensed under the [Apache-2.0 License](LICENSE)\n\n## Note\n\nThis project is for learning and sharing RAG knowledge only. Please do not use it for commercial purposes. It is not ready for production use and is still under active development.\n\n## 🙏 Acknowledgments\n\nThanks to these open source projects:\n\n- [FastAPI](https://fastapi.tiangolo.com/)\n- [Langchain](https://python.langchain.com/)\n- [Next.js](https://nextjs.org/)\n- [ChromaDB](https://www.trychroma.com/)\n\n\n![star history](https://api.star-history.com/svg?repos=rag-web-ui/rag-web-ui\u0026type=Date)\n\n---\n\n\u003cdiv align=\"center\"\u003e\n  If this project helps you, please consider giving it a ⭐️\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frag-web-ui%2Frag-web-ui","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frag-web-ui%2Frag-web-ui","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frag-web-ui%2Frag-web-ui/lists"}