Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/A-EVO-Lab/a-evolve

The official repository of "Position: Agentic Evolution is the Path to Evolving LLMs".
https://github.com/A-EVO-Lab/a-evolve

agents continual-learning llm-agents recursive-self-improvement self-evolving self-improving self-improving-ai

Last synced: 3 days ago
JSON representation

The official repository of "Position: Agentic Evolution is the Path to Evolving LLMs".

Awesome Lists containing this project

README 

          
# A-Evolve 🧬: The Universal Infrastructure for Self-Improving Agents



[![GitHub stars](https://img.shields.io/github/stars/A-EVO-Lab/a-evolve?style=social)](https://github.com/A-EVO-Lab/a-evolve)

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)

[![arXiv](https://img.shields.io/badge/arXiv-2602.00359-b31b1b.svg)](https://arxiv.org/abs/2602.00359)



> **The PyTorch for Agentic AI.**

> A-Evolve is an open-source infrastructure that evolves *any* agent, across *any* domain, using *any* evolution algorithm β€” with zero human intervention.



[Quick Start](#quick-start) | [News](#news) | [Benchmark Highlights](#benchmark-highlights) | [Architecture & Design](#architecture--design) | [Contribution](#community--contributing)



![A-Evolve Teaser](figs/teaser.png)



---



## What Does A-Evolve Do?



You provide a Base Agent. A-Evolve returns a SOTA Agent. **3 lines of code. 0 hours of manual harness 

engineering.** One infra, any domain, any evolution algorithm.



```python

import agent_evolve as ae



evolver = ae.Evolver(agent="./my_agent", benchmark="swe-verified")

results = evolver.run(cycles=10)

```



### Benchmark Highlights



By applying our open-source **reference evolution algorithms** to a base Claude Opus-4.6 model with **zero manual harness engineering**, A-Evolve pushed agents into top-tier performance across four diverse benchmarks:



🟒 MCP-Atlas








πŸ₯‡ #1


Baseline β†’ 79.4% (+3.4pp)



πŸ”΅ SWE-bench Verified








~#5


Baseline β†’ 76.8% (+2.6pp)



🟣 Terminal-Bench 2.0








~#7


Baseline β†’ 76.5% (+13.0pp)



🟑 SkillsBench








#2


Baseline β†’ 34.9% (+15.2pp)



🟒 ARC-AGI








πŸ₯‡ #2 Community Leaderboard


Baseline β†’ 12.3% (+2.2pp)



πŸ”΅ OSWorld








β€”


Baseline β†’ 69.6% (+3.9pp)



🟣 SWE-bench Lite








Evolved


63.7 β†’ 67.0% (+3.3pp)



🟑 Ο„-bench








Evolved


72.7 β†’ 77.0% (+4.3pp)



🟒 CL-Bench








Evolved


29.5 β†’ 34.0% (+4.5pp)



πŸ”΅ WebArena-Infinity








Evolved


72.5 β†’ 76.3% (+3.8pp)



![A-Evolve Benchmarks](figs/a_evolve_benchmarks.png)



> *All results achieved with a single Claude Opus-4.6 base model, evolved using A-Evolve's sample algorithms. 0 hours of human harness engineering. Data checked March 2026.*



### News

- **6/11** **New Tech Report**, [*A-Evolve-Training: Autonomous Post-Training of a 30B Model

*](https://arxiv.org/abs/2606.20657) (arXiv 2606.20657). We present an autonomous system that runs this loop with no human in the loop, **post-training a 30B Nemotron** across four rounds over multiple weeks. The autonomously produced model reaches a held-out score of **0.86 against the top human submission's 0.87** on the public NVIDIA Nemotron-Reasoning Challenge leaderboard, placing 8th of ~4000 at the time of writing. To the best of our knowledge, this is the first publicly reported autonomous post-training run at this scale, where prior public autonomous-ML-research demonstrations sit at GPT-2-class (~124M) budgets. The same system also post-trains the **120B and 550B** Nemotron models.

- **6/1** **New Research Paper**, [*Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams*](https://arxiv.org/abs/2606.01770) (arXiv 2606.01770). We address the brittleness of traditional auto-harness systems when moving from fixed benchmarks to open-ended, shifting task streams. We introduce **Adaptive Auto-Harness**, a framework that significantly outperforms five existing auto-harness baselines across prediction-market, security-competition, and event-forecasting streams. 

- **5/30** **New Research Paper**, [*Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents*](https://arxiv.org/abs/2605.30621) (arXiv 2605.30621).Tested across 7 evolver models (Opus-4.6, Sonnet-4.6, Qwen-3.5-9B, GPT-OSS-120B, etc.) Γ— 6 solver agents Γ— 3 agentic benchmarks (SWE-bench Verified, MCP-Atlas, SkillsBench), we answered **which model produced the best harness update and which models benefits the most from harness update**.  

- **05/04** **New Benchmark Results**, A-Evolve added [results](https://x.com/HenryL_AI/status/2051711038618480816?s=20) on ARC-AGI-3, evolving a multi-agent system to be more powerful on solving difficult tasks like [ARC-AGI-3](https://arcprize.org/arc-agi/3). Improving performance from 10% to 12%.

- **04/20** **New Algorithm Drop**, A-Evolve added new evolutionary algorithm [GEPA](https://x.com/HenryL_AI/status/2046326722912739713?s=20), submitted by the [GEPA](https://gepa-ai.github.io/gepa/blog/) team.

- **04/10** **Integration**, A-Evolve is officially integrated into [Orch-Research Skills Library](https://x.com/HenryL_AI/status/2042688465855488476), along with others including AutoResearch, OpenRLHF, DeepSpeed, SGLang

- **04/07** **New Agent Drop**, We added recently leaked public ClawCode (Claude Code), took the evolution harness + skills we learned on Terminal-Bench 2.0 (TB2) and directly transplanted them onto the ClawCode. [Result](https://x.com/HenryL_AI/status/2041621538580132280) on TB2: baseline **67.8%** β†’ **72.9%** (+5.1pp uplift)

- **04/03** **New Algorithm Drop**, A-Evolve added new evolutionary algorithm [Meta-Harness](https://x.com/HenryL_AI/status/2040218374458974715)

- **03/30** **Integration**, A-Evolve is officially integrated into [AutoResearchClaw](https://github.com/aiming-lab/AutoResearchClaw) 

- **03/25** πŸš€ **Open-source A-Evolve**, the universal infrastructure for developing and testing evolving algorithms.

- **03/25** πŸ“Š **Open-source 4 evolving algorithms** developed with A-Evolve, achieving SOTA **(#1, ~#5, ~#7, #2)** on MCP-Atlas, SWE-bench Verified, Terminal-Bench 2.0, and SkillsBench.

- **02/17** πŸ“„ Release the official implementation of [*Position: Agentic Evolution is the Path to Evolving LLMs*](https://arxiv.org/abs/2602.00359) (arXiv 2602.00359).



We are evolving fast! Support our research by leaving a ⭐.



### What Does an Evolved Agent Look Like?



A-Evolve mutates real files in the workspace. Here's a before/after from our MCP-Atlas evolution:



Before (Seed Workspace)

After (Evolved β€” 79.4% on MCP-Atlas)



```

mcp_agent/

β”œβ”€β”€ manifest.yaml

β”œβ”€β”€ prompts/system.md      ← 20 lines, generic

β”œβ”€β”€ skills/                ← empty

└── memory/                ← empty

```



```

mcp_agent/

β”œβ”€β”€ manifest.yaml

β”œβ”€β”€ prompts/system.md      ← 20 lines, unchanged

β”œβ”€β”€ skills/

β”‚   β”œβ”€β”€ entity-verification/SKILL.md   ← NEW

β”‚   β”œβ”€β”€ search-iteration/SKILL.md      ← NEW

β”‚   β”œβ”€β”€ multi-requirement/SKILL.md     ← NEW

β”‚   β”œβ”€β”€ code-execution/SKILL.md        ← NEW

β”‚   └── conditional-handler/SKILL.md   ← NEW

└── memory/

    └── episodic.jsonl     ← 6 entries

```



5 targeted skills outperformed 10 generic ones. Every mutation is git-tagged (`evo-1`, `evo-2`, …) for full reproducibility.



---



## Quick Start



### 1. Install



```bash

# PyPI (recommended)

pip install a-evolve              # core

pip install a-evolve[anthropic]   # Claude support

pip install a-evolve[mcp]         # MCP-Atlas benchmark

pip install a-evolve[swe]         # SWE-bench benchmark

pip install a-evolve[all]         # everything



# From source (for development)

git clone https://github.com/A-EVO-Lab/a-evolve.git && cd a-evolve

pip install -e ".[all,dev]"

```



### 2. Evolve β€” 3 Lines of Code



```python

import agent_evolve as ae



evolver = ae.Evolver(

    agent="swe-verified",           # built-in seed workspace (or path to yours)

    benchmark="swe-verified",       # built-in benchmark adapter

)

results = evolver.run(cycles=10)



print(f"Final score: {results.final_score:.3f}")

print(f"Converged:   {results.converged}")

```



A-Evolve ships with built-in seed workspaces (`swe`, `mcp`, `terminal`, `skillbench`) and benchmark adapters (`swe-verified`, `mcp-atlas`, `terminal-bench 2.0`, `skill-bench`). Point `agent=` at any of them β€” or at your own workspace directory.



### 3. Bring Your Own Agent (BYOA)



To make any agent evolvable, implement one method β€” `solve()`:



```python

from agent_evolve.protocol.base_agent import BaseAgent

from agent_evolve.types import Task, Trajectory



class MyAgent(BaseAgent):

    def solve(self, task: Task) -> Trajectory:

        return Trajectory(task_id=task.id, output="result")

```



Then evolve it:



```python

evolver = ae.Evolver(agent=MyAgent("./my_workspace"), benchmark="mcp-atlas")

results = evolver.run(cycles=10)

```



Your agent's evolvable state (prompts, skills, memory) lives as a standard directory β€” the [Agent Workspace](#the-agent-workspace-a-file-system-contract). A-Evolve mutates these files; your agent reloads. See [Architecture & Design](#architecture--design) for the full picture.



For benchmark-specific walkthroughs, see [SWE-bench Demo Guide](docs/swe-bench-demo.md), [MCP-Atlas Demo Guide](docs/mcp-atlas-demo.md), and [SkillBench Setup Guide](docs/skillbench-setup.md).



---



## Architecture & Design



![A-Evolve Framework](figs/A-EVOLVE-FRAMEWORK.png)



### The Agent Workspace: A File System Contract



A-Evolve's core insight: **all evolvable agent state lives on the file system as a standard directory structure.** This lets the evolution engine mutate any agent via LLM-driven file operations β€” without knowing the agent's internals.



```

my_agent/

β”œβ”€β”€ manifest.yaml          # identity, entrypoint, evolvable layers

β”œβ”€β”€ prompts/system.md      # system prompt

β”œβ”€β”€ skills/                # SKILL.md files (dynamic skill library)

β”œβ”€β”€ tools/                 # tool configurations

└── memory/                # episodic + semantic memory (JSONL)

```



The evolution engine reads these files, analyzes performance logs, and writes mutations back. The agent reloads. That's the entire contract.



### The Evolution Loop



Every cycle follows five phases:



```

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”

β”‚  Solve  │───▢│ Observe │───▢│ Evolve  │───▢│ Gate │───▢│ Reload β”‚

β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”˜

```



1. **Solve** β€” Agent processes a batch of tasks (black-box execution).

2. **Observe** β€” Collect trajectories + benchmark feedback into structured logs.

3. **Evolve** β€” Evolution engine analyzes observations and mutates workspace files (prompts, skills, memory).

4. **Gate** β€” Validate mutations on holdout tasks. Regressed mutations are rolled back via git.

5. **Reload** β€” Agent reloads from the (possibly rolled-back) workspace.



The loop converges when EGL (Evolutionary Generality Loss) stabilizes or `max_cycles` is reached. Every accepted mutation is git-tagged (`evo-1`, `evo-2`, …), providing a full audit trail.



### Built-in Adapters



A-Evolve ships with ready-to-use benchmark adapters and seed workspaces:



| Adapter | Domain | Seed Workspace | Best Result |

| :--- | :--- | :--- | :--- |

| [`swe-verified`](docs/swe-bench-demo.md) | Real-world GitHub issues (Python repos) | `seed_workspaces/swe/` | **76.8%** (~#5) |

| [`mcp-atlas`](docs/mcp-atlas-demo.md) | Tool-calling via MCP (16+ servers) | `seed_workspaces/mcp/` | **79.4%** (πŸ₯‡ #1) |

| [`terminal-bench`](docs/terminal-bench-demo.md) | Terminal/CLI ops in Docker | `seed_workspaces/terminal/` | **76.5%** (~#7) |

| [`skill-bench`](docs/skillbench-setup.md) | Agentic skill discovery | `seed_workspaces/skillbench/` | **34.9%** (~#2)|

| [`cl-bench`](examples/cl_bench_examples/) | Continual-learning rubric evaluation | β€” | **38.0%** |



### Pluggability: Bring Your Own Everything



A-Evolve is a **framework**, not a standalone agent. Every axis is pluggable:



| Axis | Interface | You Provide | Built-in Examples |

| :--- | :--- | :--- | :--- |

| **Agent (BYOA)** | `BaseAgent.solve()` | Any agent architecture β€” ReAct, Plan-and-Solve, custom | `SweAgent`, `McpAgent` |

| **Benchmark (BYOE)** | `BenchmarkAdapter.get_tasks()` / `.evaluate()` | Any domain with task + evaluation signal | SWE-bench, MCP-Atlas, Terminal-Bench 2.0, SkillsBench, CL-bench |

| **Algorithm (BYO-Algo)** | `EvolutionEngine.step()` | Any evolution strategy | `AEvolveEngine` (LLM-driven mutation) |

| **LLM Provider** | `LLMProvider.complete()` | Any model API | Anthropic, OpenAI, AWS Bedrock |



### Built-in Evolution Algorithms



A-Evolve ships with 4 reference evolution algorithms, each targeting different domains and strategies:



| Algorithm | Strategy | Best For | Docs |

| :--- | :--- | :--- | :--- |

| [`adaptive_evolve`](docs/algorithms/adaptive-evolve.md) | Per-claim feedback analysis + meta-learning | MCP-Atlas (πŸ₯‡ #1, 79.4%) | [Guide](docs/algorithms/adaptive-evolve.md) |

| [`adaptive_skill`](docs/algorithms/adaptive-skill.md) | LLM-driven workspace mutation with bash tool access | Terminal-Bench 2.0 (~#7, 76.5%)  | [Guide](docs/algorithms/adaptive-skill.md) |

| [`skillforge`](docs/algorithms/skillforge.md) | LLM-driven workspace mutation with EGL gating | SkillsBench (#2, 34.9%) | [Guide](docs/algorithms/skillforge.md) |

| [`guided_synth`](docs/algorithms/guided-synth.md) | Memory-first evolution + LLM-guided intervention synthesis |  General-purpose, SWE-bench (~#5, 76.8%) | [Guide](docs/algorithms/guided-synth.md) |



#### Plugging in a custom evolution algorithm



Each algorithm lives in its own directory under `algorithms/`. Implement a single method:



```python

from agent_evolve.engine.base import EvolutionEngine

from agent_evolve.types import StepResult



class MyEvolutionEngine(EvolutionEngine):

    def step(self, workspace, observations, history, trial) -> StepResult:

        # Analyze observations, mutate workspace files, optionally run trial tasks

        ...

        return StepResult(accepted=True, score=new_score)

```



Then pass it to the Evolver:



```python

evolver = ae.Evolver(

    agent="swe-verified",

    benchmark="swe-verified",

    engine=MyEvolutionEngine(config),

)

```



The engine has full access to shared primitives β€” `TrialRunner` (on-demand validation), `EvolutionHistory` (observation + version queries), and `VersionControl` (git-based rollback) β€” but is never forced to use them. Minimal contract, maximum freedom.



---



## Community & Contributing



A-Evolve is built for the research community. We welcome contributions across every axis of the framework.



### For Algorithm Researchers



If you work in LLM self-optimization, reinforcement learning, or agent architectures β€” implement the `EvolutionEngine` interface and your algorithm instantly gains access to:



- Diverse environments (SWE-bench, MCP-Atlas, Terminal-Bench 2.0, SkillsBench, and more).

- Standardized agent workspace representations.

- Rigorous evaluation, gating, and logging infrastructure.



Drop your algorithm into `agent_evolve/algorithms/your_algo/` and open a PR.



### For Benchmark Authors



Implement `BenchmarkAdapter` to plug any new evaluation domain into A-Evolve. The interface is two methods: `get_tasks()` and `evaluate()`.



### Get Involved



- ⭐ **Star this repo** to support our research β€” we are evolving fast.

- πŸ› **[Open an issue](https://github.com/A-EVO-Lab/a-evolve/issues)** to report bugs or request features.

- πŸ”€ **[Submit a PR](https://github.com/A-EVO-Lab/a-evolve/pulls)** β€” new evolution algorithms, benchmark adapters, agent implementations, and documentation improvements are all welcome.

- πŸ’¬ **[Join our Discord]()** to discuss research directions, share results, and collaborate.



---



## Citation



If you use A-Evolve in your research, please cite our position paper:



```bibtex

@article{lin2026position,

  title={Position: Agentic Evolution is the Path to Evolving LLMs},

  author={Lin, Minhua and Lu, Hanqing and Shi, Zhan and He, Bing and Mao, Rui and Zhang, Zhiwei and Wu, Zongyu and Tang, Xianfeng and Liu, Hui and Dai, Zhenwei and others},

  journal={arXiv preprint arXiv:2602.00359},

  year={2026}

}

```



---



## License



[MIT](https://opensource.org/licenses/MIT)