{"id":24324145,"url":"https://github.com/thehandsomedev/evolverl","last_synced_at":"2025-10-25T07:08:14.441Z","repository":{"id":271424628,"uuid":"913349459","full_name":"TheHandsomeDev/evolveRL","owner":"TheHandsomeDev","description":"Empower Truly Autonomous AI Agents through our Experimental Adversarial and Evolutionary Reinforcement Learning Framework. Join our Community: https://t.me/evolveRL_portal","archived":false,"fork":false,"pushed_at":"2025-01-15T15:23:09.000Z","size":5309,"stargazers_count":13,"open_issues_count":0,"forks_count":4,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-01-15T17:27:50.228Z","etag":null,"topics":["agents","ai","artificial-intelligence","attention-mechanism","chatgpt","gpt4","gpt4o","huggingface","llama3","llama3-2","machine-learning","prompt-engineering","prompt-learning","python","reinforcement-learning","reinforcement-learning-agent","swarms","transformers"],"latest_commit_sha":null,"homepage":"https://www.evolverl.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TheHandsomeDev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-07T14:12:18.000Z","updated_at":"2025-01-15T15:23:10.000Z","dependencies_parsed_at":"2025-01-07T17:57:25.209Z","dependency_job_id":"a428d4b6-47f9-44db-9030-bba31850a55e","html_url":"https://github.com/TheHandsomeDev/evolveRL","commit_stats":null,"previous_names":["thehandsomedev/evolverl"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheHandsomeDev%2FevolveRL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheHandsomeDev%2FevolveRL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheHandsomeDev%2FevolveRL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheHandsomeDev%2FevolveRL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TheHandsomeDev","download_url":"https://codeload.github.com/TheHandsomeDev/evolveRL/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234809015,"owners_count":18890103,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agents","ai","artificial-intelligence","attention-mechanism","chatgpt","gpt4","gpt4o","huggingface","llama3","llama3-2","machine-learning","prompt-engineering","prompt-learning","python","reinforcement-learning","reinforcement-learning-agent","swarms","transformers"],"created_at":"2025-01-17T19:32:52.771Z","updated_at":"2025-10-01T01:31:33.558Z","avatar_url":"https://github.com/TheHandsomeDev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# evolveRL: Evolutionary Reinforcement Learning for LLMs\n\n\u003e Empowering Truly Autonomous AI Agents Through Adversarial Evolutionary Reinforcement Learning\n\n## Links\n\n- [Website](https://www.evolverl.com/)\n- [Whitepaper](https://github.com/TheHandsomeDev/evolveRL/blob/main/evolverl-whitepaper.pdf)\n- [Techpaper](https://github.com/TheHandsomeDev/evolveRL/blob/main/evolverl-techpaper.pdf)\n- [Docs](https://docs.evolverl.com/)\n- [Telegram](https://t.me/evolveRL_portal)\n- [Twitter/X](https://x.com/evolveRL_sol)\n\n## 📚 Table of Contents\n\n- 🎯 [Overview](#overview)\n- ⭐ [Features](#features)\n- 🚀 [Getting Started](#getting-started)\n- 🔧 [Installation](#installation)\n- 🛠️ [Components](#components)\n- 🧬 [Evolutionary Loop](#evolutionary-loop)\n- 📊 [Detailed Walkthrough](#detailed-walkthrough)\n- 📄 [License](#license)\n- 🤝 [Contributing](#contributing)\n- 💬 [Citation](#citation)\n\n## Overview\n\nevolveRL is a groundbreaking framework that enables AI agents to self-improve through evolutionary and adversarial mechanisms. Unlike traditional approaches that rely heavily on manual prompt engineering, evolveRL allows agents to systematically generate, test, and refine their own prompts and configurations, bridging the gap between theoretical autonomy and actual self-reliance.\n\n### The Challenge\n\nIn the emerging AI agent economy, many envision a future where agents run autonomously with minimal human oversight. However, if humans must constantly update AI prompts to handle new tasks or edge cases, the agents aren't truly sovereign. evolveRL solves this by enabling continuous self-improvement through:\n\n1. **Autonomous Evolution**: Agents detect gaps and update their own prompts\n2. **Adversarial Testing**: Robust validation against challenging scenarios\n3. **Performance-Based Selection**: Natural emergence of optimal configurations\n4. **Continuous Adaptation**: Real-time response to changing conditions\n\n## Features\n\n- **🧬 Evolutionary Optimization**: Evolve prompts and behaviors using genetic algorithms\n- **🎯 Domain Agnostic**: Specialization for any domain\n- **⚖️ Robust Evaluation**: Comprehensive judging and evaluation\n- **🔥 Adversarial Testing**: Generate challenging scenarios to ensure robustness\n- **💾 State Management**: Save and load evolved models and their states\n- **🔄 Multiple Model Support**: Use OpenAI's GPT or Anthropic's Claude, or run LLaMA locally (coming soon)\n- **🤖 Self-Improvement Loop**: Continuous evolution without human intervention\n\n## Installation\n\n```bash\n# Basic installation\npip install evolverl\n\n# Install with all dependencies\npip install evolverl[all]\n```\n\n## Quick Start\n\n```python\nfrom evolverl.evolution import Evolution, EvolutionConfig\nfrom evolverl.llm import LLMConfig\nfrom evolverl.agent import Agent, AgentConfig\n\n# Configure LLM backend\nllm_config = LLMConfig(\n    model_name=\"gpt-4\",\n    model_type=\"openai\",  # or \"anthropic\"\n    openai_api_key=\"your-api-key\"  # or anthropic_api_key for Claude\n)\n\n# Create agent with system prompt\nagent_config = AgentConfig(llm_config=llm_config)\nagent = Agent(agent_config)\nagent.set_default_prompt(\"\"\"You are an expert AI agent specialized in mathematics.\nYou break down complex problems step by step and show your work clearly.\"\"\")\n\n# Configure evolution process\nconfig = EvolutionConfig(\n    population_size=5,\n    generations=10,\n    mutation_rate=0.1,\n    crossover_rate=0.8,\n    output_dir=\"agents\"\n)\n\n# Create evolution instance\nevolution = Evolution(config, experiment_id=\"math_solver\")\n\n# Run evolution process\nawait evolution.evolve(\n    domain=\"mathematics\",\n    description=\"Solve complex math problems with detailed explanations\"\n)\n```\n\n### Direct Agent Usage\n\nYou can also use agents directly without evolution:\n\n```python\n# Create and configure agent\nagent = Agent(AgentConfig(llm_config=llm_config))\nagent.set_default_prompt(\"You are a helpful AI assistant...\")\n\n# Send messages\nresponse = await agent.send_message(\"What is 2+2?\")\nprint(response)\n```\n\n## CLI Usage\n\n`train_agent.py` is a single file CLI that runs the evolution process. Be sure to update the config file `default_config.json` first, as well as keep your OpenAI or Anthropic API key as environment variables or in the `.env`. \n```bash\n# Basic usage with OpenAI\npython train_agent.py --domain math --description \"Solve math problems\" -v\n\n# Use Anthropic's Claude\npython train_agent.py --provider anthropic --domain math --description \"Solve math problems\"\n\n# Load domain from file\npython train_agent.py --domain-file domains/math_solver.json\n\n# Custom output directory\npython train_agent.py --domain math --description \"...\" --output-dir ./my_agents\n\n# Increase verbosity (up to -vvvvv)\npython train_agent.py --domain math --description \"...\" -vvv\n```\nCurrent domain examples are in natural language. You can add more details when building your own use cases. In addition, you may include any examples you believe are important for the agent to know. \n\n## Output Structure\n\n```\nagents/\n├── {experiment_id}_gen0.json           # Best agent from generation 0\n├── {experiment_id}_gen0_full.json      # All variants and scores from generation 0\n├── {experiment_id}_gen1.json           # Best agent from generation 1\n├── {experiment_id}_gen1_full.json      # All variants and scores from generation 1\n└── {experiment_id}_best.json           # Best agent overall\n```\nThe individual `.json` (not the `*_full.json`) contains the `AgentConfig` for the best agent of the generation or overall. You may initiate an agent directly from its `AgentConfig` file by calling `agent.load_config(PATH_TO_CONFIG_FILE)`. Be sure to update the API key as it will not be stored in the `AgentConfig` file.\n\n### Generation Output Format\n\n```json\n{\n    \"population_size\": 5,\n    \"generations\": 10,\n    \"mutation_rate\": 0.1,\n    \"crossover_rate\": 0.8,\n    \"min_score_threshold\": 0.7,\n    \"tournament_size\": 2,\n    \"max_interaction_attempts\": 5,\n    \"output_dir\": \"agents\",\n    \"llm_config\": {\n        \"model_name\": \"gpt-4o-mini\",\n        \"model_type\": \"openai\",\n        \"max_tokens\": 500,\n        \"temperature\": 0.7\n    }\n}\n```\n\n### Output Structure\n\n```\nagents/\n├── {experiment_id}_gen0.json           # Best agent from generation 0\n├── {experiment_id}_gen0_full.json      # All variants from generation 0\n├── {experiment_id}_gen1.json           # Best agent from generation 1\n├── {experiment_id}_gen1_full.json      # All variants from generation 1\n└── {experiment_id}_best.json           # Best agent overall\n```\n\n### Progress Tracking\n\nThe evolution process shows real-time progress with nested progress bars:\n```\nGeneration 2/10: 100%|██████████| 6/6 [00:15\u003c00:00, best_score=0875, avg_score=0834]\nOverall Progress:  15%|██        | 12/80 [00:30\u003c02:45, generation=2/10, best_overall=0875]\n```\nThis may take a while depending on the number of generations and population size per generation.\n\n## License\n\nMIT License - see LICENSE file for details\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Commit your changes\n4. Push to the branch\n5. Create a Pull Request\n\n## Citation\n\n```bibtex\n@software{evolverl2024,\n    title={evolveRL: Evolutionary Reinforcement Learning for LLMs},\n    author={TheHandsomeDev},\n    year={2025},\n    url={https://www.evolverl.com/}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthehandsomedev%2Fevolverl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthehandsomedev%2Fevolverl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthehandsomedev%2Fevolverl/lists"}