Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/kznr02/OpenTester

AI-Drived end-to-end test platform
https://github.com/kznr02/OpenTester

Last synced: 3 months ago
JSON representation

AI-Drived end-to-end test platform

Awesome Lists containing this project

README 

          
# OpenTester



**MCP-First Testing Execution Infrastructure**



[![PyPI version](https://badge.fury.io/py/opentester.svg)](https://badge.fury.io/py/opentester)

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

[![kznr02/OpenTester MCP server](https://glama.ai/mcp/servers/kznr02/open-tester/badges/score.svg)](https://glama.ai/mcp/servers/kznr02/open-tester)



OpenTester is a testing execution engine designed for AI coding tools (Claude Code, Cursor, OpenCode, etc.). It provides a unified DSL format and MCP interface, enabling Agents to generate, execute, and manage test cases, achieving an automated "code-test-fix" workflow.



[![kznr02/OpenTester MCP server](https://glama.ai/mcp/servers/kznr02/open-tester/badges/card.svg)](https://glama.ai/mcp/servers/kznr02/open-tester)



## Core Positioning



```

┌─────────────────────────────────────────┐

│  AI Agent (Claude Code / Cursor / ...)  │

│  ├─ Generate DSL test cases             │

│  ├─ Decide testing strategies           │

│  └─ Analyze failure reasons             │

├─────────────────────────────────────────┤

│  OpenTester (MCP Server)                │

│  ├─ Validate DSL syntax                 │

│  ├─ Execute tests (CLI/Web)             │

│  ├─ Store cases/projects                │

│  └─ Return structured results           │

├─────────────────────────────────────────┤

│  Web UI (Auxiliary Observation Panel)   │

│  ├─ View execution progress             │

│  ├─ Debug cases (create/edit)           │

│  └─ View history reports                │

└─────────────────────────────────────────┘

```



**Design Principles**:

- **Agent Intelligence**: Test generation and failure analysis are handled by the Agent

- **OpenTester Execution**: Focuses on DSL validation and test execution

- **MCP-First**: All core features exposed through MCP

- **Web UI Auxiliary**: Visual monitoring and debugging, not required



## Installation



### Option 1: Install from PyPI (Recommended)



```bash

pip install opentester



# Or with uv

uv pip install opentester

```



### Option 2: Install from Source



```bash

# Clone repository

git clone https://github.com/kznr02/OpenTester.git

cd OpenTester



# Install backend

uv pip install ./backend



# Install frontend dependencies (optional)

cd frontend

npm install

```



### Option 3: Direct Run (Development)



```bash

cd backend

uv run opentester start

```



## Quick Start



### Start Services



```bash

# Start both FastAPI + MCP (foreground mode with prefixed logs)

opentester start



# Daemon mode (detached background with log files)

opentester start --daemon



# Start only API

opentester start --api



# Start only MCP

opentester start --mcp



# Custom ports

opentester start --api-port 8080 --mcp-port 8081



# Check status

opentester status



# Stop services

opentester stop



# Environment check

opentester doctor

```



**Foreground Mode Log Output:**

```

[API] INFO:     Started server process [12345]

[API] INFO:     Waiting for application startup.

[MCP] INFO:     Started MCP server on port 8001

[API] INFO:     Application startup complete.

```



### Configure Claude Code (Streamable HTTP)



Add MCP configuration in `.claude/settings.json`:

```json

{

  "mcpServers": {

    "opentester": {

      "url": "http://localhost:8001/mcp"

    }

  }

}

```



> **Note**: Current runtime transport is Streamable HTTP.



### Usage Example



Conversation with Claude Code:



```

You: Help me test the login functionality



Claude: I'll create tests for the login feature...

        [Generate DSL test case]

        [Call MCP: validateDSL] ✓ Validation passed

        [Call MCP: createProject] ✓ Project created

        [Call MCP: saveCase] ✓ Case saved

        [Call MCP: runCase]

        Executing...

        ✓ All passed

```



## Project Structure



```

OpenTester/

├── backend/              # FastAPI + Python

│   ├── opentester/

│   │   ├── api/         # REST API (for Web UI)

│   │   ├── core/        # Execution engine, storage

│   │   ├── models/      # Pydantic models (DSL, Project)

│   │   └── mcp/         # MCP Server (core)

│   └── pyproject.toml

├── frontend/            # React + TypeScript + Vite

│   └── src/             # Web UI (observation panel)

├── docs/

│   ├── SKILL_PROMPT.md  # Agent Skill Prompt template

│   ├── DSL_SPEC.md      # DSL syntax specification

│   └── MCP.md           # MCP interface documentation

├── README.md            # This document

└── LICENSE              # MIT License

```



## Core Features



### 1. DSL Validation



After Agent generates DSL, OpenTester validates syntax correctness:



```yaml

version: "1.0"

meta:

  name: "Login Test"

steps:

  - action: exec

    command: "curl http://localhost:3000/login"

  - action: assert

    assertion:

      type: stdout_contains

      expected: "token"

```



### 2. Test Execution



Supports execution targets:

- **CLI**: subprocess command execution (implemented)

- **WEB**: Playwright-based browser automation (implemented)

- **GUI**: experimental executor routing (disabled by default)

- **TUI**: experimental executor routing (disabled by default)



### 3. Web Testing MVP + AI DOM Analysis



- Web actions are executed by `WebExecutor` (Playwright)

- Browser console, page errors, and failed requests are normalized into `diagnostic_events` for each execution step

- Supports AI-assisted locator flow via `ai_locator` in DSL steps

- Execution can pause in `paused_waiting_for_ai` status and wait for AI selector submission

- Supports DOM snapshot + optional screenshot capture for AI analysis

- Web UI and REST clients can retrieve persisted browser diagnostics from the execution diagnostics endpoints for post-run investigation



### 3. Project Management



- Projects stored in XDG data directory (`~/.local/share/opentester/` on Linux, `~/Library/Application Support/opentester/` on macOS, `%LOCALAPPDATA%\opentester\` on Windows)

- PRD content persisted with projects (provided by Agent)

- Test case version management

- Template library for reusable DSL patterns



### 4. Real-time Monitoring



- WebSocket real-time execution progress push

- Web UI visual display

- Execution history traceability



## MCP Tools



| Tool | Description |

|------|-------------|

| `list_projects` | List all test projects |

| `get_project` | Get project details (including cases) |

| `create_project` | Create project |

| `delete_project` | Delete project |

| `validate_dsl` | Validate DSL syntax |

| `save_case` | Save test case |

| `delete_case` | Delete test case |

| `run_case` | Execute single case |

| `run_project` | Execute project cases |

| `stop_execution` | Stop execution |

| `get_execution_status` | Get execution status |

| `get_execution_log` | Get detailed logs |

| `request_dom_analysis` | Get DOM snapshot for paused AI step |

| `submit_ai_selector` | Submit selector to resume paused execution |

| `list_paused_executions` | List executions waiting for AI analysis |

| `list_templates` | List templates |

| `create_template` | Create DSL template |

| `instantiate_template` | Create case from template |



See [MCP Interface Documentation](docs/MCP.md) for details.



## Documentation



See [documentation](docs/) for detailed guides:

- [Architecture](docs/ARCHITECTURE.md) - System architecture and design decisions

- [MCP Interface](docs/MCP.md) - MCP tool specifications

- [DSL Specification](docs/DSL_SPEC.md) - YAML-based test definition language

- [Development Guide](docs/DEVELOPMENT.md) - Development setup and contribution



Agents using OpenTester need to include DSL generation specifications. Refer to [SKILL_PROMPT.md](docs/SKILL_PROMPT.md)



## DSL Specification



YAML-based test definition language. See [DSL_SPEC.md](docs/DSL_SPEC.md) for details.



## Architecture



See [ARCHITECTURE.md](docs/ARCHITECTURE.md) for system architecture, design principles, and architectural decisions.



## Development Guide



Developers refer to [DEVELOPMENT.md](docs/DEVELOPMENT.md)



## Web UI



Auxiliary features:

- View project list and details

- Edit DSL cases (Monaco editor)

- Monitor execution progress

- View history reports



**Note**: Web UI is not the main entry point. All core features are provided through MCP.



## Ports



- FastAPI (Web UI / REST API): http://localhost:8000

- MCP Server: http://localhost:8001/mcp

- API Docs: http://localhost:8000/docs

- Web UI Dev Server: http://localhost:5173



## Data Storage



OpenTester follows the XDG Base Directory Specification:



- **Projects**: `/opentester/projects/{project_id}.json`

  - Linux: `~/.local/share/opentester/projects/`

  - macOS: `~/Library/Application Support/opentester/projects/`

  - Windows: `%LOCALAPPDATA%\opentester\projects\`

- **Executions**: `/opentester/executions/`

- **Templates**: `/opentester/templates/`

- **Logs**: `/opentester/logs/daemon/` (daemon mode service logs)

- **Config**: `~/.config/opentester/` (or `$XDG_CONFIG_HOME/opentester/`)



## Distribution



Refer to [DISTRIBUTION.md](docs/DISTRIBUTION.md) for:

- PyPI publishing

- PyInstaller packaging

- Docker images

- System package managers



Quick build executable:

```bash

cd backend

pip install pyinstaller

pyinstaller opentester.spec

# Output: dist/opentester.exe (Windows) or dist/opentester (Linux/Mac)

```



## Contributing



We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.



## Documentation



- [English Documentation](docs/)

- [中文文档](docs/zh/)



## License



[MIT License](LICENSE)



---



**OpenTester** - MCP-First Testing Execution Platform