{"id":28840644,"url":"https://github.com/aronweiler/code-documentor","last_synced_at":"2025-07-27T00:41:39.516Z","repository":{"id":298375878,"uuid":"999663584","full_name":"aronweiler/code-documentor","owner":"aronweiler","description":"An AI assisted code documentor","archived":false,"fork":false,"pushed_at":"2025-06-10T19:13:44.000Z","size":43,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-10T20:35:03.394Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aronweiler.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-10T15:40:37.000Z","updated_at":"2025-06-10T19:13:47.000Z","dependencies_parsed_at":"2025-06-10T20:46:56.207Z","dependency_job_id":null,"html_url":"https://github.com/aronweiler/code-documentor","commit_stats":null,"previous_names":["aronweiler/code-documentor"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/aronweiler/code-documentor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aronweiler%2Fcode-documentor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aronweiler%2Fcode-documentor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aronweiler%2Fcode-documentor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aronweiler%2Fcode-documentor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aronweiler","download_url":"https://codeload.github.com/aronweiler/code-documentor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aronweiler%2Fcode-documentor/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260774884,"owners_count":23060944,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-19T15:09:54.325Z","updated_at":"2025-07-27T00:41:39.510Z","avatar_url":"https://github.com/aronweiler.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Code Documentation Toolkit with MCP Server\n\nAn intelligent, LLM-powered toolkit that automatically generates comprehensive documentation for codebases and provides an integrated Model Context Protocol (MCP) server for AI assistant integration. This tool leverages large language models (OpenAI, Anthropic, Azure OpenAI) to create high-quality, contextual documentation and offers seamless integration with AI coding assistants through MCP.\n\n## Features\n\n### Documentation Generation\n- **Automated Code Analysis**: Scans repositories and analyzes code structure\n- **LLM-Powered Documentation**: Generates contextual documentation using advanced AI models\n- **Multiple Output Types**: Creates file documentation, design documents, and user guides\n- **Incremental Processing**: Only processes changed files to save time and API costs\n- **Cleanup Operations**: Remove orphaned documentation for deleted source files\n- **Multi-Provider Support**: Works with OpenAI, Anthropic, and Azure OpenAI\n- **Configurable Pipeline**: Highly customizable through YAML configuration\n\n### MCP Server Integration\n- **AI Assistant Integration**: Provides MCP server for seamless AI assistant interaction\n- **Intelligent File Discovery**: AI-powered relevant file finding based on natural language descriptions\n- **Feature Understanding**: Deep feature analysis and documentation retrieval\n- **VS Code Integration**: Built-in tasks and configurations for VS Code development\n- **Claude Desktop Support**: Direct integration with Claude Desktop and other MCP-compatible tools\n\n### Interface Options\n- **CLI Interface**: Comprehensive command-line interface with subcommands\n- **MCP Server**: Standards-compliant Model Context Protocol server\n- **API Endpoints**: RESTful API for programmatic access\n\n## Quick Start\n\n### Installation\n\n#### Windows\n```powershell\n# Run the installation script\n.\\install.ps1\n```\n\n#### Mac/Linux\n```bash\n# Run the installation script\nchmod +x install.sh\n./install.sh\n```\n\n#### Manual Installation\n```bash\n# Create virtual environment\npython3 -m venv venv\nsource venv/bin/activate  # On Windows: venv\\Scripts\\activate\n\n# Install dependencies\npip install -r requirements.txt\n```\n\n### Configuration\n\n1. **Set up API keys** - Create a `.env` file with your API credentials:\n```env\nOPENAI_API_KEY=sk-your-openai-key\nANTHROPIC_API_KEY=your-anthropic-key\n# For Azure OpenAI:\nAZURE_OPENAI_API_KEY=your-azure-key\nAZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/\nAZURE_OPENAI_DEPLOYMENT=your-deployment\n```\n\n2. **Configure the pipeline** - Edit `config.yaml` to match your needs:\n```yaml\nmodel:\n  provider: \"openai\"  # or \"anthropic\", \"azure_openai\"\n  name: \"gpt-4\"\n  temperature: 0.2\n\nfile_processing:\n  supported_extensions:\n    - .py\n    - .js\n    - .ts\n    - .java\n    # Add more as needed\n\n```\n\n3. **Validate configuration**:\n```bash\npython main.py validate-config\n```\n\n## Usage\n\n### Documentation Generation\n\nThe toolkit supports both subcommand and direct argument syntax for backward compatibility.\n\n#### Subcommand Syntax (Recommended)\n\nGenerate all documentation types:\n```bash\npython main.py generate --repo-path /path/to/your/repo --file-docs --design-docs --guide\n```\n\nGenerate specific documentation types:\n```bash\n# File documentation only\npython main.py generate --repo-path /path/to/your/repo --file-docs\n\n# Design documents only\npython main.py generate --repo-path /path/to/your/repo --design-docs\n\n# Documentation guide only\npython main.py generate --repo-path /path/to/your/repo --guide\n\n# Force full guide regeneration (disable incremental updates)\npython main.py generate --repo-path /path/to/your/repo --guide --force-full-guide\n\n# Clean up orphaned documentation files\npython main.py generate --repo-path /path/to/your/repo --cleanup\n```\n\n#### Direct Syntax (Backward Compatible)\n```bash\n# Generate all documentation types\npython main.py --repo-path /path/to/your/repo --file-docs --design-docs --guide\n\n# File documentation only\npython main.py --repo-path /path/to/your/repo --file-docs\n```\n\n### Repository Analysis\n\nAnalyze repository structure without generating documentation:\n```bash\npython main.py analyze /path/to/your/repo\n```\n\n### Configuration Validation\n\nValidate your configuration and API keys:\n```bash\npython main.py validate-config\n```\n\n### MCP Server\n\nStart the MCP server for AI assistant integration:\n```bash\n# Point to a specific repository\npython mcp_server.py /path/to/your/repository\n\n# Use environment variable\nexport DOCUMENTATION_REPO_PATH=/path/to/your/repository\npython mcp_server.py\n\n# Use current directory (default)\ncd /path/to/your/repository\npython mcp_server.py\n```\n\nFor detailed MCP server setup and integration instructions, see [MCP_README.md](MCP_README.md).\n\n### Command Line Options\n\n#### Generate Command Options\n\n| Option | Short | Description |\n|--------|-------|-------------|\n| `--repo-path` | `-r` | Path to the code repository (required) |\n| `--docs-path` | `-d` | Path to existing documentation for context |\n| `--output-path` | `-o` | Where to save generated documentation |\n| `--config` | `-c` | Path to configuration file (default: config.yaml) |\n| `--file-docs` | `-f` | Generate individual file documentation |\n| `--design-docs` | `-D` | Generate design documentation |\n| `--guide` | `-g` | Generate documentation guide |\n| `--force-full-guide` | | Force full guide regeneration (disable incremental updates) |\n| `--cleanup` | | Clean up orphaned documentation files for deleted source files |\n| `--verbose` | `-v` | Enable verbose output |\n\n#### Analyze Command Options\n\n| Option | Description |\n|--------|-------------|\n| `repo_path` | Path to repository to analyze (positional argument) |\n| `--config` | Configuration file (default: config.yaml) |\n\n#### Validate Config Options\n\n| Option | Description |\n|--------|-------------|\n| `--config` | Configuration file to validate (default: config.yaml) |\n\n### Examples\n\n```bash\n# Full documentation generation with custom output path\npython main.py generate -r ./my-project -o ./docs -f -D -g\n\n# Quick file documentation only\npython main.py generate -r ./my-project -f\n\n# Analyze repository structure\npython main.py analyze ./my-project\n\n# Use existing docs as context\npython main.py generate -r ./my-project -d ./existing-docs -f -g\n\n# Clean up orphaned documentation\npython main.py generate -r ./my-project --cleanup\n\n# Validate configuration\npython main.py validate-config --config custom-config.yaml\n\n# Start MCP server for AI assistant integration\npython mcp_server.py ./my-project\n```\n\n## Output Structure\n\nThe toolkit generates organized documentation in your specified output directory:\n\n```\ndocumentation_output/\n├── design_documentation/ \u003c-- Design documentation generated when using `--design-docs`\n│   ├── architecture.md\n│   ├── design.md\n│   ├── project_overview.md\n│   └── user_guide.md\n├── src/ \u003c-- File documentation generated when using `--file-docs` (path may vary)\n│   ├── config_documentation.md\n│   ├── pipeline_documentation.md\n│   └── ...\n├── documentation_guide.md \u003c-- Documentation guide generated when using `--guide`\n└── documentation_report.md \u003c-- Documentation generation report\n```\n\n### Output Types\n\n- **File Documentation**: Individual Markdown files for each source file with purpose, functionality, components, dependencies, and usage examples\n- **Design Documents**: High-level architecture, design principles, and system overview documents\n- **Documentation Guide**: Consolidated guide with navigation, summaries, and cross-references\n- **Reports**: Summary reports with metrics, status, and error information\n\n## MCP Server Integration\n\nThe toolkit includes a built-in Model Context Protocol (MCP) server that enables seamless integration with AI assistants like Claude Desktop, VS Code extensions, and other MCP-compatible tools.\n\n### MCP Server Features\n\n- **Intelligent File Discovery**: Find relevant files using natural language descriptions\n- **Feature Understanding**: Get detailed explanations of specific features or functionality\n- **Real-time Repository Analysis**: AI-powered analysis of your codebase\n- **IDE Integration**: Built-in VS Code tasks and configurations\n\n### Quick MCP Setup\n\n1. **Generate documentation for your repository**:\n   ```bash\n   python main.py generate --repo-path /path/to/your/repo --guide\n   ```\n\n2. **Start the MCP server**:\n   ```bash\n   python mcp_server.py /path/to/your/repo\n   ```\n\n3. **Integrate with Claude Desktop** by adding to your configuration:\n   ```json\n   {\n     \"mcpServers\": {\n       \"documentation-server\": {\n         \"command\": \"python\",\n         \"args\": [\"/path/to/code-documentor/mcp_server.py\", \"/path/to/your/repo\"],\n         \"cwd\": \"/path/to/code-documentor\"\n       }\n     }\n   }\n   ```\n\nFor comprehensive MCP server setup instructions, including VS Code integration, troubleshooting, and advanced configurations, see [MCP_README.md](MCP_README.md).\n\n### VS Code Integration\n\nThe toolkit includes pre-configured VS Code tasks for easy development and MCP server management:\n\n#### Available Tasks\n\n- **\"Start Documentation MCP Server\"**: Launches the MCP server using the workspace virtual environment\n- **\"Install MCP Dependencies\"**: Installs required dependencies from requirements.txt\n\n#### Using VS Code Tasks\n\n1. Open the project in VS Code\n2. Press `Ctrl+Shift+P` (or `Cmd+Shift+P` on macOS)\n3. Type \"Tasks: Run Task\"\n4. Select the desired task from the list\n\nThe tasks are configured to use the proper Python environment and working directory automatically.\n\n### Available MCP Tools\n\n- **`get_relevant_files`**: Find files relevant to a natural language description\n- **`understand_feature`**: Get documentation about specific features or functionality\n\n## Configuration\n\nThe toolkit is highly configurable through `config.yaml`. Key configuration sections:\n\n### Model Configuration\n```yaml\nmodel:\n  provider: \"openai\"  # openai, anthropic, azure_openai\n  name: \"gpt-4\"\n  temperature: 0.2\n  max_tokens: 32000\n```\n\n### File Processing\n```yaml\nfile_processing:\n  supported_extensions: [.py, .js, .ts, .java, .cpp, .c, .cs, .go, .rs, .php, .rb, .swift, .kt]\n  exclude_patterns: [\"__pycache__\", \"node_modules\", \".git\", \"*.pyc\", \"dist\", \"build\", \"venv\"]\n```\n\n### Design Documents\n```yaml\ndesign_docs:\n  enabled: true\n  documents:\n    project_overview:\n      enabled: true\n    architecture:\n      enabled: true\n    design:\n      enabled: true\n    user_guide:\n      enabled: true\n```\n\n### Processing Limits\n```yaml\nprocessing:\n  max_files: 100  # Limit number of files processed\n  save_incrementally: true  # Save files as they're processed\n\ntoken_limits:\n  max_context_tokens: 50000\n  summarization_threshold: 50000\n  chunk_size: 10000\n```\n\n## Advanced Features\n\n### Incremental Processing\nThe toolkit automatically detects file changes and only processes modified files, saving time and API costs. To force a full rebuild:\n- Delete the output directory, or\n- Use the `--force-full-guide` flag for guide regeneration\n\n### Documentation Cleanup\nAutomatically removes orphaned documentation files when source files are deleted:\n```bash\npython main.py generate --repo-path /path/to/your/repo --cleanup\n```\n\n### Token Management\nAutomatically handles large files by:\n- Chunking oversized content\n- Summarizing existing documentation when it exceeds token limits\n- Managing context windows for different LLM providers\n\n### Multiple LLM Providers\nSupports switching between providers by changing the configuration:\n- **OpenAI**: GPT-4, GPT-3.5-turbo, GPT-4-turbo\n- **Anthropic**: Claude 3 models (Opus, Sonnet, Haiku)\n- **Azure OpenAI**: Enterprise-grade OpenAI models\n\n### MCP Server Integration\n- **Standards Compliant**: Follows Model Context Protocol specifications\n- **AI Assistant Ready**: Works with Claude Desktop, VS Code, and other MCP clients\n- **Real-time Analysis**: Live repository analysis through AI assistant interactions\n- **Tool-based Interaction**: Structured API for AI assistant tool use\n\n## Troubleshooting\n\n### Common Issues\n\n| Issue | Solution |\n|-------|----------|\n| Invalid API key error | Check your `.env` file and API key validity |\n| No documentation generated | Verify `file_extensions` and `exclude_patterns` in config |\n| Truncated documentation | Normal behavior - toolkit auto-chunks and continues |\n| High API costs | Use incremental mode, exclude unnecessary files |\n| Configuration errors | Run `python main.py validate-config` |\n| MCP server connection issues | Ensure documentation guide exists, check repository path |\n| \"ModuleNotFoundError: No module named 'mcp'\" | Install MCP dependencies: `pip install mcp\u003e=1.2.0` |\n| MCP tools not responding | Verify repository has `documentation_output/documentation_guide.md` |\n| VS Code task failures | Check virtual environment activation and file paths |\n\n### Getting Help\n\n```bash\n# General help\npython main.py --help\n\n# Command-specific help\npython main.py generate --help\npython main.py analyze --help\npython main.py validate-config --help\n\n# Verbose output for debugging\npython main.py generate --repo-path ./my-project --file-docs --verbose\n\n# MCP server help\npython mcp_server.py --help\n```\n\n## Contributing\n\nWe welcome contributions to improve the toolkit! Here's how to get started:\n\n### Development Setup\n\n1. **Fork and clone the repository**\n2. **Set up development environment**:\n   ```bash\n   python3 -m venv venv\n   source venv/bin/activate\n   pip install -r requirements.txt\n   ```\n\n3. **Configure for development**:\n   - Copy `.env.example` to `.env` and add your API keys\n   - Review and modify `config.yaml` as needed\n\n### Development Guidelines\n\n- **Code Style**: Follow PEP 8 for Python code\n- **Testing**: Test your changes with different repository types and configurations\n- **Documentation**: Update documentation for any new features\n- **Modular Design**: Keep components loosely coupled and highly cohesive\n\n### Areas for Contribution\n\n- **Token Efficiency**: Needs more efficient use of tokens- such as collecting and caching document summaries with the individual documents, rather than regenerating them every time we create the guide\n- **New LLM Providers**: Add support for additional AI providers, especially local models\n- **Output Formats**: Support for additional output formats beyond Markdown\n- **Templates**: Custom documentation templates and styles\n- **Integration**: CI/CD integrations and automation tools\n- **Performance**: Optimization for large codebases\n- **MCP Extensions**: Additional MCP tools and capabilities\n- **IDE Plugins**: Enhanced IDE integrations beyond VS Code\n\n### Submitting Changes\n\n1. Create a feature branch from `main`\n2. Make your changes with appropriate tests\n3. Update documentation as needed\n4. Submit a pull request with a clear description\n\n### Code Architecture\n\nThe toolkit follows a modular pipeline architecture:\n\n#### Core Components\n- **`main.py`**: CLI entry point and command routing with subcommand support\n- **`mcp_server.py`**: Model Context Protocol server for AI assistant integration\n- **`src/config.py`**: Configuration management and validation\n- **`src/pipeline.py`**: Main orchestration and workflow management using LangGraph\n- **`src/code_analyzer.py`**: Repository scanning and file analysis\n- **`src/document_processor.py`**: Document loading and processing\n- **`src/llm_manager.py`**: Multi-provider LLM abstraction and management\n- **`src/design_document_generator.py`**: Design document generation\n- **`src/guide_generator.py`**: Documentation guide assembly with incremental updates\n- **`src/models.py`**: Pydantic data models and schemas\n\n#### MCP Integration\n- **`src/mcp_manager.py`**: MCP server orchestration and tool management\n- **`src/mcp_models.py`**: MCP-specific data models and state management\n- **LangGraph Workflows**: State-based tool execution for AI assistant interactions\n\n#### Supporting Modules\n- **`src/state_manager.py`**: Pipeline state and incremental build management\n- **`src/context_manager.py`**: Token management and context optimization\n- **`src/file_processor.py`**: File processing and metadata management\n- **`src/report_generator.py`**: Documentation reporting and metrics\n\nFor detailed architecture information, see the generated design documentation in `documentation_output/design_documentation/`.\n\n## License\n\nThis project is licensed under the GPL-3 License - see the LICENSE file for details.\n\n## Support\n\nFor issues, questions, or feature requests:\n1. Check the troubleshooting section above\n2. Review existing issues in the repository\n3. Create a new issue with detailed information about your problem\n4. Include configuration, error messages, and steps to reproduce\n\n---\n\n**Happy documenting!** 🚀📚","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faronweiler%2Fcode-documentor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faronweiler%2Fcode-documentor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faronweiler%2Fcode-documentor/lists"}