https://github.com/mountain/python-harness
A harness toolkit for Python projects
https://github.com/mountain/python-harness
Last synced: about 1 month ago
JSON representation
A harness toolkit for Python projects
- Host: GitHub
- URL: https://github.com/mountain/python-harness
- Owner: mountain
- License: mit
- Created: 2026-03-31T09:55:15.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2026-03-31T15:23:18.000Z (3 months ago)
- Last Synced: 2026-03-31T15:30:42.464Z (3 months ago)
- Language: Python
- Size: 43.9 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Python Harness
An agentic codebase evaluation and evolution tool for Python projects.
`python-harness` is designed to be a universal standard tool—just like `pytest` or `ruff`—but instead of just checking syntax or running tests, it evaluates the **architecture, readability, and governance** of your codebase using both static analysis and LLMs (DeepSeek/OpenAI).
## Features
1. **Hard Evaluation (First Fence)**: Enforces strict rules using `ruff`, `mypy`, and `ty`. Evaluates Cyclomatic Complexity (CC) and Maintainability Index (MI) via `radon`.
2. **Governance QC (Second Fence)**: Checks if the changes violate core project governance or attempt to bypass the evaluation rules themselves.
3. **Soft Evaluation (Third Fence)**:
- Calculates architecture metrics like Fan-out (coupling).
- Generates a holistic package understanding using LLMs.
- Performs "Blind QA": Randomly samples functions/classes and tests the LLM's ability to understand them without context.
4. **Actionable Output**: Synthesizes the evaluation into a final `Pass/Fail` verdict with exactly 3 concrete, actionable refactoring suggestions.
## Installation
You can install `python-harness` using `uv` or `pip`:
```bash
uv pip install python-harness
```
## Configuration
`python-harness` requires an LLM to perform its soft evaluation. Create a `.env` file in the root of your project:
```env
LLM_API_KEY=your_api_key_here
LLM_BASE_URL=https://api.deepseek.com/v1
LLM_MODEL_NAME=deepseek-reasoner
LLM_MINI_MODEL_NAME=deepseek-chat
```
*(Note: If you don't provide an API key, the harness will safely run in Mock mode).*
## Usage
### 1. Measure
To evaluate your codebase, run the `measure` command in your project directory:
```bash
harness measure .
```
This will run the full 3-fence evaluation and output a report with a final verdict and top 3 improvement suggestions.
### 2. Refine (Evolution Loop - WIP)
The `refine` command is an Agentic Edit-Test-Improve loop. It takes the suggestions generated by `measure`, automatically creates branches (variants), applies the changes, runs the tests (`pytest`), and picks the best variant.
```bash
harness refine . --steps 1 --max-retries 3
```
## License
MIT License. See [LICENSE](LICENSE) for more details.
A harness toolkit for Python projects