Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/AsyncFuncAI/deepwiki-open

Open Source DeepWiki: AI-Powered Wiki Generator for GitHub Repositories
https://github.com/AsyncFuncAI/deepwiki-open

Last synced: 8 months ago
JSON representation

Open Source DeepWiki: AI-Powered Wiki Generator for GitHub Repositories

Awesome Lists containing this project

README 

          
# DeepWiki-Open



![DeepWiki Banner](screenshots/Deepwiki.png)



**DeepWiki** is my own implementation attempt of DeepWiki, automatically creates beautiful, interactive wikis for any GitHub, GitLab, or BitBucket repository! Just enter a repo name, and DeepWiki will:



1. Analyze the code structure

2. Generate comprehensive documentation

3. Create visual diagrams to explain how everything works

4. Organize it all into an easy-to-navigate wiki



[!["Buy Me A Coffee"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://buymeacoffee.com/sheing)



[![Twitter/X](https://img.shields.io/badge/Twitter-1DA1F2?style=for-the-badge&logo=twitter&logoColor=white)](https://x.com/sashimikun_void)

[![Discord](https://img.shields.io/badge/Discord-7289DA?style=for-the-badge&logo=discord&logoColor=white)](https://discord.com/invite/VQMBGR8u5v)



## ✨ Features



- **Instant Documentation**: Turn any GitHub, GitLab or BitBucket repo into a wiki in seconds

- **Private Repository Support**: Securely access private repositories with personal access tokens

- **Smart Analysis**: AI-powered understanding of code structure and relationships

- **Beautiful Diagrams**: Automatic Mermaid diagrams to visualize architecture and data flow

- **Easy Navigation**: Simple, intuitive interface to explore the wiki

- **Ask Feature**: Chat with your repository using RAG-powered AI to get accurate answers

- **DeepResearch**: Multi-turn research process that thoroughly investigates complex topics



## 🚀 Quick Start (Super Easy!)



### Option 1: Using Docker



```bash

# Clone the repository

git clone https://github.com/AsyncFuncAI/deepwiki-open.git

cd deepwiki-open



# Create a .env file with your API keys

echo "GOOGLE_API_KEY=your_google_api_key" > .env

echo "OPENAI_API_KEY=your_openai_api_key" >> .env



# Run with Docker Compose

docker-compose up

```



> 💡 **Where to get these keys:**

> - Get a Google API key from [Google AI Studio](https://makersuite.google.com/app/apikey)

> - Get an OpenAI API key from [OpenAI Platform](https://platform.openai.com/api-keys)



### Option 2: Manual Setup (Recommended)



#### Step 1: Set Up Your API Keys



Create a `.env` file in the project root with these keys:



```

GOOGLE_API_KEY=your_google_api_key

OPENAI_API_KEY=your_openai_api_key

```



#### Step 2: Start the Backend



```bash

# Install Python dependencies

pip install -r api/requirements.txt



# Start the API server

python -m api.main

```



#### Step 3: Start the Frontend



```bash

# Install JavaScript dependencies

npm install

# or

yarn install



# Start the web app

npm run dev

# or

yarn dev

```



### Step 4: Use DeepWiki!



1. Open [http://localhost:3000](http://localhost:3000) in your browser

2. Enter a GitHub, GitLab, or Bitbucket repository (like `https://github.com/openai/codex`, `https://github.com/microsoft/autogen`, `https://gitlab.com/gitlab-org/gitlab`, or `https://bitbucket.org/redradish/atlassian_app_versions`)

3. For private repositories, click "+ Add access tokens" and enter your GitHub or GitLab personal access token

4. Click "Generate Wiki" and watch the magic happen!



## 🔍 How It Works



DeepWiki uses AI to:



1. Clone and analyze the GitHub, GitLab, or Bitbucket repository (including private repos with token authentication)

2. Create embeddings of the code for smart retrieval

3. Generate documentation with context-aware AI

4. Create visual diagrams to explain code relationships

5. Organize everything into a structured wiki

6. Enable intelligent Q&A with the repository through the Ask feature

7. Provide in-depth research capabilities with DeepResearch



```mermaid

graph TD

    A[User inputs GitHub/GitLab/Bitbucket repo] --> AA{Private repo?}

    AA -->|Yes| AB[Add access token]

    AA -->|No| B[Clone Repository]

    AB --> B

    B --> C[Analyze Code Structure]

    C --> D[Create Code Embeddings]

    D --> E[Generate Documentation]

    D --> F[Create Visual Diagrams]

    E --> G[Organize as Wiki]

    F --> G

    G --> H[Interactive DeepWiki]



    classDef process stroke-width:2px;

    classDef data stroke-width:2px;

    classDef result stroke-width:2px;

    classDef decision stroke-width:2px;



    class A,D data;

    class AA decision;

    class B,C,E,F,G,AB process;

    class H result;

```



## 🛠️ Project Structure



```

deepwiki/

├── api/                  # Backend API server

│   ├── main.py           # API entry point

│   ├── api.py            # FastAPI implementation

│   ├── rag.py            # Retrieval Augmented Generation

│   ├── data_pipeline.py  # Data processing utilities

│   └── requirements.txt  # Python dependencies

│

├── src/                  # Frontend Next.js app

│   ├── app/              # Next.js app directory

│   │   └── page.tsx      # Main application page

│   └── components/       # React components

│       └── Mermaid.tsx   # Mermaid diagram renderer

│

├── public/               # Static assets

├── package.json          # JavaScript dependencies

└── .env                  # Environment variables (create this)

```



## 🛠️ Advanced Setup



### Environment Variables



| Variable | Description | Required | Note |

|----------|-------------|----------|------|

| `GOOGLE_API_KEY` | Google Gemini API key for AI generation | Yes |

| `OPENAI_API_KEY` | OpenAI API key for embeddings | Yes |

| `PORT` | Port for the API server (default: 8001) | No | If you host API and frontend on the same machine, make sure change port of `NEXT_PUBLIC_SERVER_BASE_URL` accordingly |

| `NEXT_PUBLIC_SERVER_BASE_URL` | Base URL for the API server (default: http://localhost:8001) | No |



### Docker Setup



You can use Docker to run DeepWiki:



```bash

# Pull the image from GitHub Container Registry

docker pull ghcr.io/asyncfuncai/deepwiki-open:latest



# Run the container with environment variables

docker run -p 8001:8001 -p 3000:3000 \

  -e GOOGLE_API_KEY=your_google_api_key \

  -e OPENAI_API_KEY=your_openai_api_key \

  -v ~/.adalflow:/root/.adalflow \

  ghcr.io/asyncfuncai/deepwiki-open:latest

```



Or use the provided `docker-compose.yml` file:



```bash

# Edit the .env file with your API keys first

docker-compose up

```



#### Using a .env file with Docker



You can also mount a .env file to the container:



```bash

# Create a .env file with your API keys

echo "GOOGLE_API_KEY=your_google_api_key" > .env

echo "OPENAI_API_KEY=your_openai_api_key" >> .env



# Run the container with the .env file mounted

docker run -p 8001:8001 -p 3000:3000 \

  -v $(pwd)/.env:/app/.env \

  -v ~/.adalflow:/root/.adalflow \

  ghcr.io/asyncfuncai/deepwiki-open:latest

```



#### Building the Docker image locally



If you want to build the Docker image locally:



```bash

# Clone the repository

git clone https://github.com/AsyncFuncAI/deepwiki-open.git

cd deepwiki-open



# Build the Docker image

docker build -t deepwiki-open .



# Run the container

docker run -p 8001:8001 -p 3000:3000 \

  -e GOOGLE_API_KEY=your_google_api_key \

  -e OPENAI_API_KEY=your_openai_api_key \

  deepwiki-open

```



### API Server Details



The API server provides:

- Repository cloning and indexing

- RAG (Retrieval Augmented Generation)

- Streaming chat completions



For more details, see the [API README](./api/README.md).



## 🤖 Ask & DeepResearch Features



### Ask Feature



The Ask feature allows you to chat with your repository using Retrieval Augmented Generation (RAG):



- **Context-Aware Responses**: Get accurate answers based on the actual code in your repository

- **RAG-Powered**: The system retrieves relevant code snippets to provide grounded responses

- **Real-Time Streaming**: See responses as they're generated for a more interactive experience

- **Conversation History**: The system maintains context between questions for more coherent interactions



### DeepResearch Feature



DeepResearch takes repository analysis to the next level with a multi-turn research process:



- **In-Depth Investigation**: Thoroughly explores complex topics through multiple research iterations

- **Structured Process**: Follows a clear research plan with updates and a comprehensive conclusion

- **Automatic Continuation**: The AI automatically continues research until reaching a conclusion (up to 5 iterations)

- **Research Stages**:

  1. **Research Plan**: Outlines the approach and initial findings

  2. **Research Updates**: Builds on previous iterations with new insights

  3. **Final Conclusion**: Provides a comprehensive answer based on all iterations



To use DeepResearch, simply toggle the "Deep Research" switch in the Ask interface before submitting your question.



## 📱 Screenshots



![DeepWiki Main Interface](screenshots/Interface.png)

*The main interface of DeepWiki*



![Private Repository Support](screenshots/privaterepo.png)

*Access private repositories with personal access tokens*



![DeepResearch Feature](screenshots/DeepResearch.png)

*DeepResearch conducts multi-turn investigations for complex topics*



### Demo Video



[![DeepWiki Demo Video](https://img.youtube.com/vi/zGANs8US8B4/0.jpg)](https://youtu.be/zGANs8US8B4)



*Watch DeepWiki in action!*



## ❓ Troubleshooting



### API Key Issues

- **"Missing environment variables"**: Make sure your `.env` file is in the project root and contains both API keys

- **"API key not valid"**: Check that you've copied the full key correctly with no extra spaces



### Connection Problems

- **"Cannot connect to API server"**: Make sure the API server is running on port 8001

- **"CORS error"**: The API is configured to allow all origins, but if you're having issues, try running both frontend and backend on the same machine



### Generation Issues

- **"Error generating wiki"**: For very large repositories, try a smaller one first

- **"Invalid repository format"**: Make sure you're using a valid GitHub, GitLab or Bitbucket URL format

- **"Could not fetch repository structure"**: For private repositories, ensure you've entered a valid personal access token with appropriate permissions

- **"Diagram rendering error"**: The app will automatically try to fix broken diagrams



### Common Solutions

1. **Restart both servers**: Sometimes a simple restart fixes most issues

2. **Check console logs**: Open browser developer tools to see any JavaScript errors

3. **Check API logs**: Look at the terminal where the API is running for Python errors



## 🤝 Contributing



Contributions are welcome! Feel free to:

- Open issues for bugs or feature requests

- Submit pull requests to improve the code

- Share your feedback and ideas



## 📄 License



This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.



## ⭐ Star History



[![Star History Chart](https://api.star-history.com/svg?repos=AsyncFuncAI/deepwiki-open&type=Date)](https://star-history.com/#AsyncFuncAI/deepwiki-open&Date)