https://github.com/agent-sh/perf
Rigorous performance investigation workflow with baselines, profiling, and evidence-backed decisions
https://github.com/agent-sh/perf
ai automation benchmarking code-analysis devtools javascript llm optimization performance profiling
Last synced: 3 months ago
JSON representation
Rigorous performance investigation workflow with baselines, profiling, and evidence-backed decisions
- Host: GitHub
- URL: https://github.com/agent-sh/perf
- Owner: agent-sh
- Created: 2026-02-21T03:29:30.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2026-03-22T23:52:56.000Z (4 months ago)
- Last Synced: 2026-03-23T08:42:33.264Z (4 months ago)
- Topics: ai, automation, benchmarking, code-analysis, devtools, javascript, llm, optimization, performance, profiling
- Language: JavaScript
- Size: 1.69 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
Awesome Lists containing this project
README
# perf
Structured performance investigation with baselines, profiling, and evidence-backed decisions - run by AI agents inside your editor.
## Why
Performance work without structure leads to random micro-optimizations, unmeasured changes, and wasted time. `/perf` enforces a rigorous 10-phase methodology - derived from recorded real investigation sessions - that ensures every optimization is baselined, isolated, measured, and documented.
Use cases:
- Investigating a reported performance regression
- Establishing baselines before and after a refactor
- Finding breaking points under load
- Profiling CPU and memory hotspots with language-native tools
- Making evidence-backed continue/stop decisions on optimization efforts
## Installation
```bash
agentsys install perf
```
Requires [agentsys](https://github.com/agent-sh/agentsys) to be set up in your project.
## Quick Start
```
/perf --phase setup --scenario "API response time regression" --command "npm run bench" --version v1.2.0
```
This initializes an investigation, records the scenario, and prepares for baselining. The orchestrator walks you through each phase sequentially, checkpointing progress after every step.
To resume an in-progress investigation:
```
/perf --resume
```
## How It Works
Every investigation follows 10 phases in order. The orchestrator enforces strict rules: sequential benchmarks only, 60-second minimum run durations, one change per experiment, and checkpoint commits after each phase.
1. **Setup** - Confirm the scenario, success criteria, and benchmark command. The benchmark must emit metrics between `PERF_METRICS_START` / `PERF_METRICS_END` markers.
2. **Baseline** - Run the benchmark for 60+ seconds with a 10-second warmup. Results are stored as `baselines/.json`. This is the reference point for all comparisons.
3. **Breaking point** - Binary search across a parameterized range to find the threshold where performance degrades. Uses 30-second runs (the only exception to the 60-second rule).
4. **Constraints** - Run the benchmark under CPU and memory limits (default: 1 CPU, 1GB RAM). Compare against the unconstrained baseline to identify resource sensitivity.
5. **Hypotheses** - Generate up to 5 hypotheses about the root cause, each backed by evidence from git history and code analysis. No guessing - every hypothesis needs a file path or commit reference.
6. **Code paths** - Use repo-map to identify entry points, hot files, and call chains relevant to the scenario. Narrows the search space before profiling.
7. **Profiling** - Run language-specific profilers: `--cpu-prof` for Node.js, JFR for Java, cProfile for Python, pprof for Go. Capture file:line hotspots and flame graphs.
8. **Optimization** - Apply one change at a time. Validate with 2+ benchmark runs. Revert between experiments to keep the baseline clean.
9. **Decision** - Based on measured improvement, decide whether to continue optimizing or stop. The verdict and rationale are recorded in the investigation log.
10. **Consolidation** - Write the final baseline, close the evidence log, and mark the investigation complete.
## Usage
```bash
/perf --phase setup --scenario "startup time" --command "node bench.js" --version v2.0.0
/perf --resume # continue where you left off
/perf --resume --phase profiling # jump to a specific phase
/perf --resume --phase baseline --runs 5 --aggregate median
/perf --resume --phase optimization --change "replaced O(n^2) loop with hash lookup"
/perf --resume --phase decision --verdict continue --rationale "20% improvement"
```
### Key Flags
| Flag | Description |
|------|-------------|
| `--resume` | Continue the active investigation |
| `--phase ` | Target a specific phase |
| `--command ` | Benchmark command (must emit PERF_METRICS markers) |
| `--version ` | Baseline version label |
| `--duration ` | Override benchmark duration (default 60s) |
| `--runs ` | Number of runs for multi-run benchmarks |
| `--aggregate ` | median, mean, min, or max (default median) |
| `--change ` | Description of the optimization being tested |
| `--verdict ` | continue or stop |
### Investigation Artifacts
State is persisted under `{state-dir}/perf/`: `investigation.json` (active state), `investigations/.md` (evidence log), and `baselines/.json` (metrics per version).
## Architecture
| Component | Type | Model | Role |
|-----------|------|-------|------|
| `perf-orchestrator` | agent | opus | Coordinates all phases, enforces rules |
| `perf-theory-gatherer` | agent | sonnet | Generates hypotheses from git history and code |
| `perf-theory-tester` | agent | sonnet | Runs controlled experiments for hypotheses |
| `perf-analyzer` | agent | sonnet | Synthesizes findings into recommendations |
| `perf-code-paths` | agent | sonnet | Identifies hot files and entry points |
| `perf-investigation-logger` | agent | sonnet | Writes structured evidence log entries |
| `perf-baseline-manager` | skill | - | Baseline storage, one JSON per version |
| `perf-benchmarker` | skill | - | Sequential benchmark execution |
| `perf-profiler` | skill | - | Language-specific profiling (CPU, memory, flame graphs) |
## Requirements
- [agentsys](https://github.com/agent-sh/agentsys) runtime
- A benchmark command that emits `PERF_METRICS_START` / `PERF_METRICS_END` markers
- Language-specific profiler tools installed for the profiling phase (e.g., Node.js `--cpu-prof`, Java JFR, Python cProfile)
## Related Plugins
- [repo-map](https://github.com/agent-sh/repo-map) - AST-based repository map used during the code-paths phase
- [git-map](https://github.com/agent-sh/git-map) - Git history analysis for identifying recent changes that may correlate with regressions
- [enhance](https://github.com/agent-sh/enhance) - Code analysis that can complement perf findings
## License
MIT