{"id":30861111,"url":"https://github.com/rhecosystemappeng/mcp-validation","last_synced_at":"2025-09-07T16:48:20.337Z","repository":{"id":307277752,"uuid":"1028960315","full_name":"RHEcosystemAppEng/mcp-validation","owner":"RHEcosystemAppEng","description":"Comprehensive validation tool for Model Context Protocol (MCP) servers with security analysis and JSON reporting","archived":false,"fork":false,"pushed_at":"2025-09-05T06:55:48.000Z","size":354,"stargazers_count":2,"open_issues_count":2,"forks_count":5,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-07T14:54:45.970Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RHEcosystemAppEng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-30T10:08:35.000Z","updated_at":"2025-09-05T06:55:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"0e87cbb4-5392-4ce5-aa02-34375740d3d7","html_url":"https://github.com/RHEcosystemAppEng/mcp-validation","commit_stats":null,"previous_names":["dmartinol/mcp-validation"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/RHEcosystemAppEng/mcp-validation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fmcp-validation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fmcp-validation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fmcp-validation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fmcp-validation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RHEcosystemAppEng","download_url":"https://codeload.github.com/RHEcosystemAppEng/mcp-validation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RHEcosystemAppEng%2Fmcp-validation/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274065738,"owners_count":25216444,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-07T02:00:09.463Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-07T16:48:18.608Z","updated_at":"2025-09-07T16:48:20.328Z","avatar_url":"https://github.com/RHEcosystemAppEng.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MCP Validation Tool\n\nA comprehensive validation tool for [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) servers to ensure protocol compliance, security, and proper implementation.\n\n## Goal\n\nThis tool validates MCP servers by:\n\n- **Protocol Compliance**: Tests the complete MCP initialization handshake\n- **Standard Conformance**: Validates JSON-RPC 2.0 format and required fields  \n- **Capability Testing**: Verifies advertised capabilities (resources, tools, prompts)\n- **Security Analysis**: Integrates with [mcp-scan](https://github.com/invariantlabs-ai/mcp-scan) for vulnerability detection\n- **Registry Validation**: Ensures servers match their registry schema definitions\n- **Detailed Reporting**: Exports comprehensive JSON reports with validation checklists\n- **Automated Testing**: Provides programmatic validation for CI/CD pipelines\n\n## Features\n\n- ✅ **Protocol Validation**: Complete MCP handshake and capability testing\n- ✅ **Security Scanning**: Integrated mcp-scan vulnerability analysis\n- ✅ **JSON Reports**: Comprehensive validation reports with linked security scans\n- ✅ **Step-by-Step Logging**: Real-time validation progress with detailed feedback\n- ✅ **Tool Discovery**: Lists all available tools, prompts, and resources\n- ✅ **Environment Variables**: Configurable environment setup\n- ✅ **Timeout Handling**: Configurable validation timeouts\n- ✅ **Exit Codes**: Proper exit codes for automation\n- ✅ **Verbose Mode**: Optional detailed output\n\n## Installation\n\n```bash\n# Clone and install\ngit clone https://github.com/modelcontextprotocol/mcp-validation\ncd mcp-validation\nuv sync\n```\n\nOr install directly:\n```bash\npip install mcp-validation\n```\n\n## Usage\n\n### Basic Validation\n\n```bash\n# Validate a Python MCP server\nmcp-validate python server.py\n\n# Validate a Node.js MCP server\nmcp-validate node server.js\n\n# Validate npx packages (use -- separator for flags)\nmcp-validate -- npx -y kubernetes-mcp-server@latest\n\n# Validate servers via container runtime (podman/docker)\nmcp-validate -- podman run -i --rm hashicorp/terraform-mcp-server\n```\n\n### With Environment Variables\n\n```bash\n# IoTDB MCP server example\nmcp-validate \\\n  --env IOTDB_HOST=127.0.0.1 \\\n  --env IOTDB_PORT=6667 \\\n  --env IOTDB_USER=root \\\n  --env IOTDB_PASSWORD=root \\\n  python src/iotdb_mcp_server/server.py\n```\n\n### JSON Report Generation\n\n```bash\n# Generate comprehensive JSON report\nmcp-validate --json-report validation-report.json python server.py\n\n# With security analysis and custom timeout\nmcp-validate \\\n  --timeout 60 \\\n  --json-report full-report.json \\\n  --env API_KEY=secret \\\n  -- npx -y some-mcp-server@latest\n```\n\n### Security Analysis Options\n\n```bash\n# Skip mcp-scan for faster validation\nmcp-validate --skip-mcp-scan python server.py\n\n# Full validation with security scan\nmcp-validate --timeout 120 --json-report report.json python server.py\n```\n\n### Programmatic Usage\n\n```python\nimport asyncio\nfrom mcp_validation import validate_mcp_server_command\n\nasync def test_server():\n    result = await validate_mcp_server_command(\n        command_args=[\"python\", \"server.py\"],\n        env_vars={\"API_KEY\": \"secret\"},\n        timeout=30.0,\n        use_mcp_scan=True\n    )\n    \n    if result.is_valid:\n        print(f\"✓ Server is MCP compliant!\")\n        print(f\"Tools: {result.tools}\")\n        print(f\"Capabilities: {list(result.capabilities.keys())}\")\n        if result.mcp_scan_results:\n            print(f\"Security scan: {result.mcp_scan_file}\")\n    else:\n        print(\"✗ Validation failed:\")\n        for error in result.errors:\n            print(f\"  - {error}\")\n\nasyncio.run(test_server())\n```\n\n## CLI Options\n\n| Option | Description | Example |\n|--------|-------------|---------|\n| `command` | Command and arguments to run the MCP server | `python server.py` |\n| `--env KEY=VALUE` | Set environment variables (repeatable) | `--env HOST=localhost` |\n| `--timeout SECONDS` | Validation timeout in seconds (default: 30) | `--timeout 60` |\n| `--verbose` | Show detailed output including warnings | `--verbose` |\n| `--skip-mcp-scan` | Skip mcp-scan security analysis | `--skip-mcp-scan` |\n| `--json-report FILE` | Export detailed JSON report to file | `--json-report report.json` |\n\n## Validation Process\n\nThe tool performs these validation steps:\n\n1. **Process Execution**: Starts the server with provided arguments and environment\n2. **Initialize Handshake**: Sends MCP `initialize` request with protocol version\n3. **Protocol Compliance**: Validates JSON-RPC 2.0 format and required response fields\n4. **Capability Discovery**: Tests advertised capabilities (resources, tools, prompts)\n5. **Security Analysis**: Runs mcp-scan vulnerability detection (optional)\n6. **Report Generation**: Creates detailed JSON reports with validation checklist\n\n## Output Format\n\n```\nTesting MCP server: npx -y kubernetes-mcp-server@latest\n\n🔄 Step 1: Sending initialize request...\n✅ Initialize request successful\n🔄 Step 2: Sending initialized notification...\n✅ Initialized notification sent\n🔄 Step 3: Testing capabilities...\n  🔄 Testing tools...\n    ✅ Found 18 tools\n    📋 Names: configuration_view, events_list, helm_install, helm_list, helm_uninstall (and 13 more)\n  🔄 Testing prompts...\n    ✅ Found 0 prompts\n  🔄 Testing resources...\n    ✅ Found 0 resources\n✅ Capability testing complete\n🔄 Step 4: Running mcp-scan security analysis...\n    🔍 Running: uvx mcp-scan@latest --json...\n    📊 Scanned 18 tools\n    ✅ No security issues detected\n    💾 Scan results saved to: mcp-scan-results_20250730_120203.json\n✅ mcp-scan analysis complete\n\n✓ Valid: True\n⏱ Execution time: 10.49s\n🖥 Server: kubernetes-mcp-server vv0.0.46\n🔧 Capabilities: logging, prompts, resources, tools\n🔨 Tools (18): configuration_view, events_list, helm_install, helm_list, helm_uninstall, namespaces_list, pods_delete, pods_exec, pods_get, pods_list, pods_list_in_namespace, pods_log, pods_run, pods_top, resources_create_or_update, resources_delete, resources_get, resources_list\n🔍 Security Scan: No issues found in 18 tools\n📋 JSON report saved to: validation-report.json\n```\n\n## JSON Report Structure\n\nThe `--json-report` option generates comprehensive validation reports:\n\n```json\n{\n  \"report_metadata\": {\n    \"generated_at\": \"2025-07-30T12:02:03.456789\",\n    \"validator_version\": \"0.1.0\",\n    \"command\": \"npx -y kubernetes-mcp-server@latest\",\n    \"environment_variables\": {}\n  },\n  \"validation_summary\": {\n    \"is_valid\": true,\n    \"execution_time_seconds\": 10.49,\n    \"total_errors\": 0,\n    \"total_warnings\": 0\n  },\n  \"validation_checklist\": {\n    \"protocol_validation\": {\n      \"initialize_request\": {\"status\": \"passed\", \"details\": \"...\"},\n      \"initialize_response\": {\"status\": \"passed\", \"details\": \"...\"},\n      \"protocol_version\": {\"status\": \"passed\", \"details\": \"...\"}\n    },\n    \"capability_testing\": {\n      \"tools_capability\": {\"status\": \"passed\", \"details\": \"...\"},\n      \"resources_capability\": {\"status\": \"skipped\", \"details\": \"...\"}\n    },\n    \"security_analysis\": {\n      \"mcp_scan_execution\": {\"status\": \"passed\", \"details\": \"...\"}\n    }\n  },\n  \"server_information\": {\n    \"server_info\": {\"name\": \"kubernetes-mcp-server\", \"version\": \"v0.0.46\"},\n    \"capabilities\": {\"logging\": {}, \"tools\": {\"listChanged\": true}},\n    \"discovered_items\": {\n      \"tools\": {\"count\": 18, \"names\": [\"configuration_view\", \"...\"]}\n    }\n  },\n  \"security_analysis\": {\n    \"mcp_scan_executed\": true,\n    \"mcp_scan_file\": \"mcp-scan-results_20250730_120203.json\",\n    \"summary\": {\n      \"tools_scanned\": 18,\n      \"vulnerabilities_found\": 0,\n      \"vulnerability_types\": [],\n      \"risk_levels\": []\n    }\n  },\n  \"issues\": {\n    \"errors\": [],\n    \"warnings\": []\n  }\n}\n```\n\n## Exit Codes\n\n- `0`: Server is MCP compliant\n- `1`: Validation failed or server is non-compliant\n\n## MCP Registry Validation\n\nFor servers listed in the [MCP Registry](https://github.com/modelcontextprotocol/registry), this tool can validate:\n\n- Package installation requirements\n- Environment variable specifications\n- Argument format compliance\n- Protocol implementation correctness\n\n## Development\n\n### Prerequisites\n\nThis project uses [uv](https://docs.astral.sh/uv/) for dependency management and development workflows.\n\n```bash\n# Install uv (if not already installed)\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Clone the repository\ngit clone https://github.com/modelcontextprotocol/mcp-validation\ncd mcp-validation\n```\n\n### Quick Start with Makefile\n\nFor convenience, this project includes a Makefile with common development tasks:\n\n```bash\n# Setup development environment\nmake install\n\n# Run the full pre-commit workflow (format, lint, test)\nmake pre-commit\n\n# Run tests\nmake test\n\n# Format code\nmake format\n\n# See all available commands\nmake help\n```\n\n### Manual Setup\n\n```bash\n# Install all dependencies including dev extras\nuv sync --extra dev\n\n# Alternatively, install the package in development mode\nuv pip install -e \".[dev]\"\n```\n\n### Available Make Commands\n\n| Command | Description |\n|---------|-------------|\n| `make help` | Show all available commands |\n| `make install` | Install dependencies with dev extras |\n| `make dev-setup` | Complete development environment setup |\n| `make test` | Run all tests (excluding partner repos) |\n| `make test-cov` | Run tests with coverage report |\n| `make test-fast` | Run tests with fail-fast (-x flag) |\n| `make debug-test` | Run tests with debug output and registry logging |\n| `make format` | Format code with Black |\n| `make check` | Check formatting without making changes |\n| `make lint` | Check code with Ruff (no fixes) |\n| `make lint-fix` | Check and fix code issues with Ruff |\n| `make pre-commit` | Run full pre-commit workflow (format, lint, test) |\n| `make ci` | Run CI-like checks (no automatic fixes) |\n| `make clean` | Clean up cache and temporary files |\n\n### Manual Testing Commands\n\n```bash\n# Run all tests\nmake test\n# OR manually:\nuv run --extra dev pytest tests/ -v\n\n# Run tests with coverage\nmake test-cov\n# OR manually:\nuv run --extra dev pytest tests/ --cov=mcp_validation --cov-report=term-missing\n\n# Run specific test file\nuv run --extra dev pytest tests/test_enhanced_registry.py -v\n\n# Run tests and stop on first failure\nmake test-fast\n# OR manually:\nuv run --extra dev pytest tests/ -x\n```\n\n### Code Formatting and Linting\n\n```bash\n# Format code with Black\nmake format\n# OR manually:\nuv run --extra dev black mcp_validation/\n\n# Check code formatting (without making changes)\nmake check\n# OR manually:\nuv run --extra dev black --check mcp_validation/\n\n# Lint with Ruff (with fixes)\nmake lint-fix\n# OR manually:\nuv run --extra dev ruff check --fix mcp_validation/\n\n# Lint with Ruff (check only)\nmake lint\n# OR manually:\nuv run --extra dev ruff check mcp_validation/\n\n# Type checking with mypy\nuv run --extra dev mypy mcp_validation/\n```\n\n### Workflows\n\n```bash\n# Pre-commit workflow (format, lint, test)\nmake pre-commit\n\n# CI-style checks (no automatic fixes)\nmake ci\n\n# Manual pre-commit workflow\nuv run --extra dev black mcp_validation/ \u0026\u0026 \\\nuv run --extra dev ruff check --fix mcp_validation/ \u0026\u0026 \\\nuv run --extra dev pytest tests/ -v\n```\n\n### Development Guidelines\n\n1. **Testing**: All new features must include tests\n2. **Code Style**: Use Black for formatting and Ruff for linting\n3. **Type Hints**: Add type hints for all public APIs\n4. **Documentation**: Update README and docstrings for new features\n\n### Test Configuration\n\nThe project uses pytest with the following configuration in `pyproject.toml`:\n\n- **Test Discovery**: Looks for tests in the `tests/` directory\n- **Async Support**: Configured for async/await testing\n- **Exclusions**: Automatically excludes partner repositories and build directories\n- **Markers**: Strict marker checking enabled\n\n### Debugging Tests\n\n```bash\n# Run tests with debug output and registry logging\nmake debug-test\n\n# Run tests with verbose output and debug information\nuv run --extra dev pytest -v -s\n\n# Run specific test with debugging\nuv run --extra dev pytest tests/test_enhanced_registry.py::test_enhanced_registry_validator -v -s\n\n# Run registry tests with debug output\nmcp-validate --debug -- npm test\n```\n\n### Debugging MCP Validation\n\nThe tool provides comprehensive debug output to track server execution progress:\n\n```bash\n# Enable debug output for detailed execution tracking\nmcp-validate --debug -- python server.py\n```\n\n**Debug output includes:**\n- **Execution Context**: Working directory, Python version, platform, user, shell\n- **Command Details**: Full command, arguments, executable path\n- **Environment Variables**: Custom variables (with sensitive value masking)\n- **Process Information**: PID, process lifecycle events\n- **Validator Progress**: Individual validator execution with timing and results\n- **Validation Summary**: Overall statistics and execution time\n\n**Example debug output:**\n```\n[10:19:29.872] [EXEC-INFO] 🚀 Starting MCP Server Process\n[10:19:29.872] [EXEC-INFO] 📁 Working Directory: /path/to/project\n[10:19:29.872] [EXEC-INFO] 🐍 Python: /usr/bin/python3 (v3.11.0)\n[10:19:29.872] [EXEC-INFO] 🔧 Command: npx @dynatrace-oss/dynatrace-mcp-server\n[10:19:29.872] [EXEC-INFO] 🌍 Environment Variables:\n[10:19:29.872] [EXEC-INFO]    API_KEY=ab*****ef\n[10:19:29.877] [VALIDATOR-INFO] 🔍 [registry] STARTING: (1/6)\n[10:19:30.727] [VALIDATOR-INFO] 🔍 [registry] PASSED: Time: 0.85s\n```\n\n## Examples\n\n### Validate Registry Server\n\n```bash\n# Apache IoTDB MCP Server from registry\nmcp-validate \\\n  --env IOTDB_HOST=127.0.0.1 \\\n  --env IOTDB_PORT=6667 \\\n  --env IOTDB_USER=root \\\n  --env IOTDB_PASSWORD=root \\\n  --env IOTDB_DATABASE=test \\\n  --env IOTDB_SQL_DIALECT=table \\\n  python src/iotdb_mcp_server/server.py\n```\n\n### CI/CD Integration\n\n```yaml\n# GitHub Actions example\n- name: Validate MCP Server\n  run: |\n    mcp-validate --json-report validation-report.json python server.py\n  env:\n    DATABASE_URL: sqlite:///test.db\n\n- name: Upload validation report\n  uses: actions/upload-artifact@v3\n  if: always()\n  with:\n    name: mcp-validation-report\n    path: |\n      validation-report.json\n      mcp-scan-results_*.json\n```\n\n### Security Analysis\n\nThe tool integrates with [mcp-scan](https://github.com/invariantlabs-ai/mcp-scan) for comprehensive security analysis:\n\n- **Automatic Detection**: Checks for `uvx` or `mcp-scan` availability\n- **Vulnerability Scanning**: Analyzes tools for potential security issues\n- **Separate Reports**: Security results saved to timestamped JSON files\n- **Linked Reports**: Main validation report references security scan files\n- **Skip Option**: Use `--skip-mcp-scan` for faster validation without security analysis\n\n## Contributing\n\nContributions are welcome! Please see our [contributing guidelines](CONTRIBUTING.md) for details.\n\n## License\n\nMIT License - see [LICENSE](LICENSE) file for details.\n\n## Related Projects\n\n- [MCP Specification](https://spec.modelcontextprotocol.io/)\n- [MCP Registry](https://github.com/modelcontextprotocol/registry)\n- [mcp-scan](https://github.com/invariantlabs-ai/mcp-scan) - Security vulnerability scanner for MCP servers\n- [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk)\n- [MCP TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhecosystemappeng%2Fmcp-validation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frhecosystemappeng%2Fmcp-validation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frhecosystemappeng%2Fmcp-validation/lists"}