https://github.com/hkuds/deepcode
"DeepCode: Open Agentic Coding (Paper2Code & Text2Web & Text2Backend)"
https://github.com/hkuds/deepcode
agentic-ai agentic-coding llm-agent
Last synced: about 1 month ago
JSON representation
"DeepCode: Open Agentic Coding (Paper2Code & Text2Web & Text2Backend)"
- Host: GitHub
- URL: https://github.com/hkuds/deepcode
- Owner: HKUDS
- License: mit
- Created: 2025-05-14T05:23:02.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2025-08-05T07:28:32.000Z (2 months ago)
- Last Synced: 2025-08-05T09:26:29.485Z (2 months ago)
- Topics: agentic-ai, agentic-coding, llm-agent
- Language: Python
- Homepage:
- Size: 5.71 MB
- Stars: 693
- Watchers: 18
- Forks: 87
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
██████╗ ███████╗███████╗██████╗ ██████╗ ██████╗ ██████╗ ███████╗
██╔══██╗██╔════╝██╔════╝██╔══██╗██╔════╝██╔═══██╗██╔══██╗██╔════╝
██║ ██║█████╗ █████╗ ██████╔╝██║ ██║ ██║██║ ██║█████╗
██║ ██║██╔══╝ ██╔══╝ ██╔═══╝ ██║ ██║ ██║██║ ██║██╔══╝
██████╔╝███████╗███████╗██║ ╚██████╗╚██████╔╝██████╔╝███████╗
╚═════╝ ╚══════╝╚══════╝╚═╝ ╚═════╝ ╚═════╝ ╚═════╝ ╚══════╝
# 
DeepCode: Open Agentic Coding
### *Advancing Code Generation with Multi-Agent Systems*
### 🖥️ **Interface Showcase**
#### 🖥️ **CLI Interface**
**Terminal-Based Development**
🚀 Advanced Terminal Experience
⚡ Fast command-line workflow
🔧 Developer-friendly interface
📊 Real-time progress tracking
*Professional terminal interface for advanced users and CI/CD integration*
#### 🌐 **Web Interface**
**Visual Interactive Experience**
🎨 Modern Web Dashboard
🖱️ Intuitive drag-and-drop
📱 Responsive design
🎯 Visual progress tracking
*Beautiful web interface with streamlined workflow for all skill levels*
---
### 🎬 **Introduction Video**
*🎯 **Watch our complete introduction** - See how DeepCode transforms research papers and natural language into production-ready code*
---
> *"Where AI Agents Transform Ideas into Production-Ready Code"*
---
## 📑 Table of Contents
- [🚀 Key Features](#-key-features)
- [🏗️ Architecture](#️-architecture)
- [🚀 Quick Start](#-quick-start)
- [💡 Examples](#-examples)
- [🎬 Live Demonstrations](#-live-demonstrations)
- [⭐ Star History](#-star-history)
- [📄 License](#-license)
---
## 🚀 Key Features
🚀 Paper2Code
Automated Implementation of Complex Algorithms
Effortlessly converts complex algorithms from research papers into high-quality, production-ready code, accelerating algorithm reproduction.
🎨 Text2Web
Automated Front-End Web Development
Translates plain textual descriptions into fully functional, visually appealing front-end web code for rapid interface creation.
⚙️ Text2Backend
Automated Back-End Development
Generates efficient, scalable, and feature-rich back-end code from simple text inputs, streamlining server-side development.
### 🎯 **Autonomous Multi-Agent Workflow**
**The Challenges**:
- 📄 **Implementation Complexity**: Converting academic papers and complex algorithms into working code requires significant technical effort and domain expertise
- 🔬 **Research Bottleneck**: Researchers spend valuable time implementing algorithms instead of focusing on their core research and discovery work
- ⏱️ **Development Delays**: Product teams experience long wait times between concept and testable prototypes, slowing down innovation cycles
- 🔄 **Repetitive Coding**: Developers repeatedly implement similar patterns and functionality instead of building on existing solutions
**DeepCode** addresses these workflow inefficiencies by providing reliable automation for common development tasks, streamlining your development workflow from concept to code.
```mermaid
flowchart LR
A["📄 Research Papers
💬 Text Prompts
🌐 URLs & Document
📎 Files: PDF, DOC, PPTX, TXT, HTML"] --> B["🧠 DeepCode
Multi-Agent Engine"]
B --> C["🚀 Algorithm Implementation
🎨 Frontend Development
⚙️ Backend Development"]
style A fill:#ff6b6b,stroke:#c0392b,stroke-width:2px,color:#000
style B fill:#00d4ff,stroke:#0984e3,stroke-width:3px,color:#000
style C fill:#00b894,stroke:#00a085,stroke-width:2px,color:#000
```
---
## 🏗️ Architecture
### 📊 **System Overview**
**DeepCode** is an AI-powered development platform that automates code generation and implementation tasks. Our multi-agent system handles the complexity of translating requirements into functional, well-structured code, allowing you to focus on innovation rather than implementation details.
🎯 **Technical Capabilities**:
🧬 **Research-to-Production Pipeline**
Multi-modal document analysis engine that extracts algorithmic logic and mathematical models from academic papers. Generates optimized implementations with proper data structures while preserving computational complexity characteristics.
🪄 **Natural Language Code Synthesis**
Context-aware code generation using fine-tuned language models trained on curated code repositories. Maintains architectural consistency across modules while supporting multiple programming languages and frameworks.
⚡ **Automated Prototyping Engine**
Intelligent scaffolding system generating complete application structures including database schemas, API endpoints, and frontend components. Uses dependency analysis to ensure scalable architecture from initial generation.
💎 **Quality Assurance Automation**
Integrated static analysis with automated unit test generation and documentation synthesis. Employs AST analysis for code correctness and property-based testing for comprehensive coverage.
🔮 **CodeRAG Integration System**
Advanced retrieval-augmented generation combining semantic vector embeddings with graph-based dependency analysis. Automatically discovers optimal libraries and implementation patterns from large-scale code corpus.
---
### 🔧 **Core Techniques**
- 🧠 **Intelligent Orchestration Agent**: Central decision-making system that coordinates workflow phases and analyzes requirements. Employs dynamic planning algorithms to adapt execution strategies in real-time based on evolving project complexity. Dynamically selects optimal processing strategies for each implementation step.
- 💾 **Efficient Memory Mechanism**: Advanced context engineering system that manages large-scale code contexts efficiently. Implements hierarchical memory structures with intelligent compression for handling complex codebases. This component enables instant retrieval of implementation patterns and maintains semantic coherence across extended development sessions.
- 🔍 **Advanced CodeRAG System**: Global code comprehension engine that analyzes complex inter-dependencies across repositories. Performs cross-codebase relationship mapping to understand architectural patterns from a holistic perspective. This module leverages dependency graphs and semantic analysis to provide globally-aware code recommendations during implementation.
---
### 🤖 **Multi-Agent Architecture of DeepCode**:
- **🎯 Central Orchestrating Agent**: Orchestrates entire workflow execution and makes strategic decisions. Coordinates specialized agents based on input complexity analysis. Implements dynamic task planning and resource allocation algorithms.
- **📝 Intent Understanding Agent**: Performs deep semantic analysis of user requirements to decode complex intentions. Extracts functional specifications and technical constraints through advanced NLP processing. Transforms ambiguous human descriptions into precise, actionable development specifications with structured task decomposition.
- **📄 Document Parsing Agent**: Processes complex technical documents and research papers with advanced parsing capabilities. Extracts algorithms and methodologies using document understanding models. Converts academic concepts into practical implementation specifications through intelligent content analysis.
- **🏗️ Code Planning Agent**: Performs architectural design and technology stack optimization. Dynamic planning for adaptive development roadmaps. Enforces coding standards and generates modular structures through automated design pattern selection.
- **🔍 Code Reference Mining Agent**: Discovers relevant repositories and frameworks through intelligent search algorithms. Analyzes codebases for compatibility and integration potential. Provides recommendations based on similarity metrics and automated dependency analysis.
- **📚 Code Indexing Agent**: Builds comprehensive knowledge graphs of discovered codebases. Maintains semantic relationships between code components. Enables intelligent retrieval and cross-reference capabilities.
- **🧬 Code Generation Agent**: Synthesizes gathered information into executable code implementations. Creates functional interfaces and integrates discovered components. Generates comprehensive test suites and documentation for reproducibility.
---
#### 🛠️ **Implementation Tools Matrix**
**🔧 Powered by MCP (Model Context Protocol)**
DeepCode leverages the **Model Context Protocol (MCP)** standard to seamlessly integrate with various tools and services. This standardized approach ensures reliable communication between AI agents and external systems, enabling powerful automation capabilities.
##### 📡 **MCP Servers & Tools**
| 🛠️ **MCP Server** | 🔧 **Primary Function** | 💡 **Purpose & Capabilities** |
|-------------------|-------------------------|-------------------------------|
| **🔍 brave** | Web Search Engine | Real-time information retrieval via Brave Search API |
| **🌐 bocha-mcp** | Alternative Search | Secondary search option with independent API access |
| **📂 filesystem** | File System Operations | Local file and directory management, read/write operations |
| **🌐 fetch** | Web Content Retrieval | Fetch and extract content from URLs and web resources |
| **📥 github-downloader** | Repository Management | Clone and download GitHub repositories for analysis |
| **📋 file-downloader** | Document Processing | Download and convert files (PDF, DOCX, etc.) to Markdown |
| **⚡ command-executor** | System Commands | Execute bash/shell commands for environment management |
| **🧬 code-implementation** | Code Generation Hub | Comprehensive code reproduction with execution and testing |
| **📚 code-reference-indexer** | Smart Code Search | Intelligent indexing and search of code repositories |
| **📄 document-segmentation** | Smart Document Analysis | Intelligent document segmentation for large papers and technical documents |
##### 🔧 **Legacy Tool Functions** *(for reference)*
| 🛠️ **Function** | 🎯 **Usage Context** |
|-----------------|---------------------|
| **📄 read_code_mem** | Efficient code context retrieval from memory |
| **✍️ write_file** | Direct file content generation and modification |
| **🐍 execute_python** | Python code testing and validation |
| **📁 get_file_structure** | Project structure analysis and organization |
| **⚙️ set_workspace** | Dynamic workspace and environment configuration |
| **📊 get_operation_history** | Process monitoring and operation tracking |
---
🎛️ **Multi-Interface Framework**
RESTful API with CLI and web frontends featuring real-time code streaming, interactive debugging, and extensible plugin architecture for CI/CD integration.
**🚀 Multi-Agent Intelligent Pipeline:**
### 🌟 **Intelligence Processing Flow**
💡 INPUT LAYER
📄 Research Papers • 💬 Natural Language • 🌐 URLs • 📋 Requirements
🎯 CENTRAL ORCHESTRATION
Strategic Decision Making • Workflow Coordination • Agent Management
📝 TEXT ANALYSIS
Requirement Processing
📄 DOCUMENT ANALYSIS
Paper & Spec Processing
📋 REPRODUCTION PLANNING
Deep Paper Analysis • Code Requirements Parsing • Reproduction Strategy Development
🔍 REFERENCE ANALYSIS
Repository Discovery
📚 CODE INDEXING
Knowledge Graph Building
🧬 CODE IMPLEMENTATION
Implementation Generation • Testing • Documentation
⚡ OUTPUT DELIVERY
📦 Complete Codebase • 🧪 Test Suite • 📚 Documentation • 🚀 Deployment Ready
### 🔄 **Process Intelligence Features**
🎯 Adaptive Flow
Dynamic agent selection based on input complexity
🧠 Smart Coordination
Intelligent task distribution and parallel processing
🔍 Context Awareness
Deep understanding through CodeRAG integration
⚡ Quality Assurance
Automated testing and validation throughout
---
## 🚀 Quick Start
### 📦 **Step 1: Installation**
#### ⚡ **Direct Installation (Recommended)**
```bash
# 🚀 Install DeepCode package directly
pip install deepcode-hku
# 🔑 Download configuration files
curl -O https://raw.githubusercontent.com/HKUDS/DeepCode/main/mcp_agent.config.yaml
curl -O https://raw.githubusercontent.com/HKUDS/DeepCode/main/mcp_agent.secrets.yaml
# 🔑 Configure API keys (required)
# Edit mcp_agent.secrets.yaml with your API keys and base_url:
# - openai: api_key, base_url (for OpenAI/custom endpoints)
# - anthropic: api_key (for Claude models)
# 🔑 Configure search API keys for web search (optional)
# Edit mcp_agent.config.yaml to set your API keys:
# - For Brave Search: Set BRAVE_API_KEY: "your_key_here" in brave.env section (line ~28)
# - For Bocha-MCP: Set BOCHA_API_KEY: "your_key_here" in bocha-mcp.env section (line ~74)
# 📄 Configure document segmentation (optional)
# Edit mcp_agent.config.yaml to control document processing:
# - enabled: true/false (whether to use intelligent document segmentation)
# - size_threshold_chars: 50000 (document size threshold to trigger segmentation)
```
#### 🔧 **Development Installation (From Source)**
📂 Click to expand development installation options
##### 🔥 **Using UV (Recommended for Development)**
```bash
# 🔽 Clone the repository
git clone https://github.com/HKUDS/DeepCode.git
cd DeepCode/
# 📦 Install UV package manager
curl -LsSf https://astral.sh/uv/install.sh | sh
# 🔧 Install dependencies with UV
uv venv --python=3.13
source .venv/bin/activate # On Windows: .venv\Scripts\activate
uv pip install -r requirements.txt
# 🔑 Configure API keys (required)
# Edit mcp_agent.secrets.yaml with your API keys and base_url:
# - openai: api_key, base_url (for OpenAI/custom endpoints)
# - anthropic: api_key (for Claude models)
# 🔑 Configure search API keys for web search (optional)
# Edit mcp_agent.config.yaml to set your API keys:
# - For Brave Search: Set BRAVE_API_KEY: "your_key_here" in brave.env section (line ~28)
# - For Bocha-MCP: Set BOCHA_API_KEY: "your_key_here" in bocha-mcp.env section (line ~74)
# 📄 Configure document segmentation (optional)
# Edit mcp_agent.config.yaml to control document processing:
# - enabled: true/false (whether to use intelligent document segmentation)
# - size_threshold_chars: 50000 (document size threshold to trigger segmentation)
```
##### 🐍 **Using Traditional pip**
```bash
# 🔽 Clone the repository
git clone https://github.com/HKUDS/DeepCode.git
cd DeepCode/
# 📦 Install dependencies
pip install -r requirements.txt
# 🔑 Configure API keys (required)
# Edit mcp_agent.secrets.yaml with your API keys and base_url:
# - openai: api_key, base_url (for OpenAI/custom endpoints)
# - anthropic: api_key (for Claude models)
# 🔑 Configure search API keys for web search (optional)
# Edit mcp_agent.config.yaml to set your API keys:
# - For Brave Search: Set BRAVE_API_KEY: "your_key_here" in brave.env section (line ~28)
# - For Bocha-MCP: Set BOCHA_API_KEY: "your_key_here" in bocha-mcp.env section (line ~74)
# 📄 Configure document segmentation (optional)
# Edit mcp_agent.config.yaml to control document processing:
# - enabled: true/false (whether to use intelligent document segmentation)
# - size_threshold_chars: 50000 (document size threshold to trigger segmentation)
```
#### 🪟 **Windows Users: Additional MCP Server Configuration**
If you're using Windows, you may need to configure MCP servers manually in `mcp_agent.config.yaml`:
```bash
# 1. Install MCP servers globally
npm i -g @modelcontextprotocol/server-brave-search
npm i -g @modelcontextprotocol/server-filesystem
# 2. Find your global node_modules path
npm -g root
```
Then update your `mcp_agent.config.yaml` to use absolute paths:
```yaml
mcp:
servers:
brave:
command: "node"
args: ["C:/Program Files/nodejs/node_modules/@modelcontextprotocol/server-brave-search/dist/index.js"]
filesystem:
command: "node"
args: ["C:/Program Files/nodejs/node_modules/@modelcontextprotocol/server-filesystem/dist/index.js", "."]
```
> **Note**: Replace the path with your actual global node_modules path from step 2.
#### 🔍 **Search Server Configuration (Optional)**
DeepCode supports multiple search servers for web search functionality. You can configure your preferred option in `mcp_agent.config.yaml`:
```yaml
# Default search server configuration
# Options: "brave" or "bocha-mcp"
default_search_server: "brave"
```
**Available Options:**
- **🔍 Brave Search** (`"brave"`):
- Default option with high-quality search results
- Requires BRAVE_API_KEY configuration
- Recommended for most users
- **🌐 Bocha-MCP** (`"bocha-mcp"`):
- Alternative search server option
- Requires BOCHA_API_KEY configuration
- Uses local Python server implementation
**API Key Configuration in mcp_agent.config.yaml:**
```yaml
# For Brave Search (default) - around line 28
brave:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-brave-search"]
env:
BRAVE_API_KEY: "your_brave_api_key_here"
# For Bocha-MCP (alternative) - around line 74
bocha-mcp:
command: "python"
args: ["tools/bocha_search_server.py"]
env:
PYTHONPATH: "."
BOCHA_API_KEY: "your_bocha_api_key_here"
```
> **💡 Tip**: Both search servers require API key configuration. Choose the one that best fits your API access and requirements.
### ⚡ **Step 2: Launch Application**
#### 🚀 **Using Installed Package (Recommended)**
```bash
# 🌐 Launch web interface directly
deepcode
# The application will automatically start at http://localhost:8501
```
#### 🛠️ **Using Source Code**
Choose your preferred interface:
##### 🌐 **Web Interface** (Recommended)
```bash
# Using UV
uv run streamlit run ui/streamlit_app.py
# Or using traditional Python
streamlit run ui/streamlit_app.py
```
##### 🖥️ **CLI Interface** (Advanced Users)
```bash
# Using UV
uv run python cli/main_cli.py
# Or using traditional Python
python cli/main_cli.py
```
### 🎯 **Step 3: Generate Code**
1. **📄 Input**: Upload your research paper, provide requirements, or paste a URL
2. **🤖 Processing**: Watch the multi-agent system analyze and plan
3. **⚡ Output**: Receive production-ready code with tests and documentation
---
## 💡 Examples
### 🎬 **Live Demonstrations**
#### 📄 **Paper2Code Demo**
**Research to Implementation**
**[▶️ Watch Demo](https://www.youtube.com/watch?v=MQZYpLkzsbw)**
*Transform academic papers into production-ready code automatically*
#### 🖼️ **Image Processing Demo**
**AI-Powered Image Tools**
**[▶️ Watch Demo](https://www.youtube.com/watch?v=nFt5mLaMEac)**
*Intelligent image processing with background removal and enhancement*
#### 🌐 **Frontend Implementation**
**Complete Web Application**
**[▶️ Watch Demo](https://www.youtube.com/watch?v=78wx3dkTaAU)**
*Full-stack web development from concept to deployment*
### 🆕 **Recent Updates**
#### 📄 **Smart Document Segmentation (v1.2.0)**
- **Intelligent Processing**: Automatically handles large research papers and technical documents that exceed LLM token limits
- **Configurable Control**: Toggle segmentation via configuration with size-based thresholds
- **Semantic Analysis**: Advanced content understanding with algorithm, concept, and formula preservation
- **Backward Compatibility**: Seamlessly falls back to traditional processing for smaller documents
### 🚀 **Coming Soon**
We're continuously enhancing DeepCode with exciting new features:
#### 🔧 **Enhanced Code Reliability & Validation**
- **Automated Testing**: Comprehensive functionality testing with execution verification and error detection.
- **Code Quality Assurance**: Multi-level validation through static analysis, dynamic testing, and performance benchmarking.
- **Smart Debugging**: AI-powered error detection with automatic correction suggestions
#### 📊 **PaperBench Performance Showcase**
- **Benchmark Dashboard**: Comprehensive performance metrics on the PaperBench evaluation suite.
- **Accuracy Metrics**: Detailed comparison with state-of-the-art paper reproduction systems.
- **Success Analytics**: Statistical analysis across paper categories and complexity levels.
#### ⚡ **System-wide Optimizations**
- **Performance Boost**: Multi-threaded processing and optimized agent coordination for faster generation.
- **Enhanced Reasoning**: Advanced reasoning capabilities with improved context understanding.
- **Expanded Support**: Extended compatibility with additional programming languages and frameworks.
---
## ⭐ Star History
*Community Growth Trajectory*
---
### 🚀 **Ready to Transform Development?**
---
### 📄 **License**
**MIT License** - Copyright (c) 2025 Data Intelligence Lab, The University of Hong Kong
---
