{"id":32867779,"url":"https://github.com/nshkrdotcom/gepa_ex","last_synced_at":"2026-01-20T16:50:26.141Z","repository":{"id":321407726,"uuid":"1085674539","full_name":"nshkrdotcom/gepa_ex","owner":"nshkrdotcom","description":"Elixir implementation of GEPA: LLM-driven evolutionary optimization using Pareto-efficient search for text-based systems. Features OpenAI/Gemini integration, BEAM concurrency, OTP supervision, 218 tests with 75% coverage.","archived":false,"fork":false,"pushed_at":"2025-10-29T16:39:51.000Z","size":345,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-11-06T21:55:26.204Z","etag":null,"topics":["ai-optimization","artificial-intelligence","beam","concurrent-programming","elixir","erlang","erlang-otp","evolutionary-algorithms","functional-programming","gemini","genetic-algorithms","llm","machine-learning","metaheuristics","multi-objective-optimization","openai","optimization","otp","pareto-optimization","prompt-optimization"],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nshkrdotcom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-29T11:09:52.000Z","updated_at":"2025-10-29T16:39:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"9cacfd44-b5d4-4958-a439-13756474c174","html_url":"https://github.com/nshkrdotcom/gepa_ex","commit_stats":null,"previous_names":["nshkrdotcom/gepa_ex"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nshkrdotcom/gepa_ex","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fgepa_ex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fgepa_ex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fgepa_ex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fgepa_ex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nshkrdotcom","download_url":"https://codeload.github.com/nshkrdotcom/gepa_ex/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nshkrdotcom%2Fgepa_ex/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":283264496,"owners_count":26807274,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-07T02:00:06.343Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-optimization","artificial-intelligence","beam","concurrent-programming","elixir","erlang","erlang-otp","evolutionary-algorithms","functional-programming","gemini","genetic-algorithms","llm","machine-learning","metaheuristics","multi-objective-optimization","openai","optimization","otp","pareto-optimization","prompt-optimization"],"created_at":"2025-11-09T07:00:36.134Z","updated_at":"2026-01-20T16:50:26.132Z","avatar_url":"https://github.com/nshkrdotcom.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/gepa_ex.svg\" alt=\"GEPA Elixir Logo\" width=\"200\" height=\"200\"\u003e\n\u003c/p\u003e\n\n# GEPA for Elixir\n\n[![Hex.pm](https://img.shields.io/hexpm/v/gepa_ex.svg)](https://hex.pm/packages/gepa_ex)\n[![Elixir](https://img.shields.io/badge/elixir-1.18.3-purple.svg)](https://elixir-lang.org)\n[![OTP](https://img.shields.io/badge/otp-27.3.3-blue.svg)](https://www.erlang.org)\n[![Tests](https://img.shields.io/badge/tests-218%2F218%20passing-brightgreen)]()\n[![Coverage](https://img.shields.io/badge/coverage-75.4%25-brightgreen)]()\n[![License](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/nshkrdotcom/gepa_ex/blob/main/LICENSE)\n\nAn Elixir implementation of GEPA (Genetic-Pareto), a framework for optimizing text-based system components using LLM-based reflection and Pareto-efficient evolutionary search.\n\n## Installation\n\nAdd `gepa_ex` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:gepa_ex, \"~\u003e 0.1.2\"}\n  ]\nend\n```\n\n## About GEPA\n\nGEPA optimizes arbitrary systems composed of text components—like AI prompts, code snippets, or textual specs—against any evaluation metric. It employs LLMs to reflect on system behavior, using feedback from execution traces to drive targeted improvements.\n\nThis is an Elixir port of the [Python GEPA library](https://github.com/gepa-ai/gepa), designed to leverage:\n- 🚀 **BEAM concurrency** for 5-10x evaluation speedup (coming in Phase 4)\n- 🛡️ **OTP supervision** for fault-tolerant external service integration\n- 🔄 **Functional programming** for clean, testable code\n - 📊 **Telemetry** event schema for lifecycle, iteration, proposal, and evaluation metrics\n- ✨ **Production LLMs** - OpenAI GPT-4o-mini \u0026 Google Gemini Flash Lite (`gemini-flash-lite-latest`)\n\n## Production Ready\n\n### Core Features\n\n**Optimization System:**\n- ✅ `GEPA.optimize/1` - Public API (working!)\n- ✅ `GEPA.Engine` - Full optimization loop with stop conditions\n- ✅ `GEPA.Proposer.Reflective` - Mutation strategy\n- ✅ LLM-based instruction proposal via `reflection_llm` and custom templates\n- ✅ `GEPA.State` - State management with automatic Pareto updates (96.5% coverage)\n- ✅ `GEPA.Utils.Pareto` - Multi-objective optimization (93.5% coverage, property-verified)\n- ✅ `GEPA.Result` - Result analysis (100% coverage)\n- ✅ `GEPA.Adapters.Basic` - Q\u0026A adapter (92.1% coverage)\n- ✅ Stop conditions with budget control\n- ✅ State persistence (save/load)\n- ✅ Telemetry event emitters for runs, iterations, proposals, and evaluation batches\n- ✅ End-to-end integration tested\n\n### Phase 1 Additions - NEW! 🎉\n\n**Production LLM Integration:**\n- ✅ `GEPA.LLM` - Unified LLM behavior\n- ✅ `GEPA.LLM.ReqLLM` - Production implementation via ReqLLM\n  - OpenAI support (GPT-4o-mini default)\n  - Google Gemini support (gemini-flash-lite-latest)\n  - Error handling, retries, timeouts\n  - Configurable via environment or runtime\n- ✅ `GEPA.LLM.Mock` - Testing implementation with flexible responses\n\n**Advanced Batch Sampling:**\n- ✅ `GEPA.Strategies.BatchSampler.EpochShuffled` - Epoch-based training with shuffling\n- ✅ Reproducible with seed control\n- ✅ Better training dynamics than simple sampling\n\n**Working Examples:**\n- ✅ 4 .exs script examples (quick start, math, custom adapter, persistence)\n- ✅ 3 Livebook notebooks (interactive learning)\n- ✅ Comprehensive examples/README.md guide\n- ✅ Livebook guide with visualizations\n\n**Phase 2 Additions - NEW! 🎉**\n\n**Merge Proposer:**\n- ✅ `GEPA.Proposer.Merge` - Genealogy-based candidate merging\n- ✅ `GEPA.Utils` - Pareto dominator detection (93.3% coverage)\n- ✅ `GEPA.Proposer.MergeUtils` - Ancestry tracking (92.3% coverage)\n- ✅ Engine integration with merge scheduling\n- ✅ 44 comprehensive tests (34 unit + 10 properties)\n\n**Incremental Evaluation:**\n- ✅ `GEPA.Strategies.EvaluationPolicy.Incremental` - Progressive validation\n- ✅ Configurable sample sizes and thresholds\n- ✅ Reduces computation on large validation sets\n- ✅ 12 tests\n\n**Advanced Stop Conditions:**\n- ✅ `GEPA.StopCondition.Timeout` - Time-based stopping\n- ✅ `GEPA.StopCondition.NoImprovement` - Early stopping\n- ✅ Flexible time units and patience settings\n- ✅ 9 tests\n\n**Test Quality:**\n- 201 tests (185 unit + 16 properties + 1 doctest)\n- 100% passing ✅\n- 75.4% coverage (excellent!)\n- Property tests with 1,600+ runs\n- Zero Dialyzer errors\n- TDD methodology throughout\n\n## What's Next?\n\n**✅ Phase 1: Production Viability** - COMPLETE!\n- ✅ Real LLM integration (OpenAI, Gemini)\n- ✅ Quick start examples (4 scripts + 3 livebooks)\n- ✅ EpochShuffledBatchSampler\n\n**✅ Phase 2: Core Completeness** - COMPLETE!\n- ✅ Merge proposer (genealogy-based recombination)\n- ✅ IncrementalEvaluationPolicy (progressive validation)\n- ✅ Additional stop conditions (Timeout, NoImprovement)\n- ✅ Engine integration for merge proposer\n\n**Phase 3: Production Hardening** - in progress\n- ✅ Telemetry event schema and helpers\n- 🎨 Progress tracking (planned)\n- 🛡️ Robust error handling (planned)\n\n**Phase 4: Ecosystem Expansion** - 12-14 weeks\n- 🔌 Additional adapters (Generic, RAG)\n- 🚀 Performance optimization (parallel evaluation)\n- 🌟 Community infrastructure\n\n## Quick Start\n\n### With Mock LLM (No API Key Required)\n\n```elixir\n# Define training data\ntrainset = [\n  %{input: \"What is 2+2?\", answer: \"4\"},\n  %{input: \"What is 3+3?\", answer: \"6\"}\n]\n\nvalset = [%{input: \"What is 5+5?\", answer: \"10\"}]\n\n# Create adapter with mock LLM (for testing)\nadapter = GEPA.Adapters.Basic.new(llm: GEPA.LLM.Mock.new())\n\n# Run optimization\n{:ok, result} = GEPA.optimize(\n  seed_candidate: %{\"instruction\" =\u003e \"You are a helpful assistant.\"},\n  trainset: trainset,\n  valset: valset,\n  adapter: adapter,\n  max_metric_calls: 50\n)\n\n# Access results\nbest_program = GEPA.Result.best_candidate(result)\nbest_score = GEPA.Result.best_score(result)\n\nIO.puts(\"Best score: #{best_score}\")\nIO.puts(\"Iterations: #{result.i}\")\n```\n\n### With Production LLMs (NEW!)\n\n```elixir\n# OpenAI (GPT-4o-mini) - Requires OPENAI_API_KEY\nllm = GEPA.LLM.ReqLLM.new(provider: :openai)\nadapter = GEPA.Adapters.Basic.new(llm: llm)\n\n# Or Gemini (`gemini-flash-lite-latest`) - Requires GEMINI_API_KEY\nllm = GEPA.LLM.ReqLLM.new(provider: :gemini)\nadapter = GEPA.Adapters.Basic.new(llm: llm)\n\n# Then run optimization as above\n{:ok, result} = GEPA.optimize(\n  seed_candidate: %{\"instruction\" =\u003e \"...\"},\n  trainset: trainset,\n  valset: valset,\n  adapter: adapter,\n  max_metric_calls: 50\n)\n```\n\nSee [Examples overview](examples/README.md) for complete working examples!\n\n### Candidate Selection Strategies (NEW)\n\nGEPA includes multiple candidate selectors to balance exploration vs. exploitation:\n\n- `GEPA.Strategies.CandidateSelector.Pareto` (default): frequency-weighted sampling from Pareto front\n- `GEPA.Strategies.CandidateSelector.CurrentBest`: always pick the best-scoring program\n- `GEPA.Strategies.CandidateSelector.EpsilonGreedy`: configurable exploration with optional epsilon decay\n\nStateful selectors (like epsilon-greedy) are carried forward automatically so decay persists across iterations.\n\nTo enable epsilon-greedy with decay:\n\n```elixir\nselector =\n  GEPA.Strategies.CandidateSelector.EpsilonGreedy.new(\n    epsilon: 0.3,\n    epsilon_decay: 0.95,\n    epsilon_min: 0.05\n  )\n\n{:ok, result} =\n  GEPA.optimize(\n    seed_candidate: %{\"instruction\" =\u003e \"...\"},\n    trainset: trainset,\n    valset: valset,\n    adapter: adapter,\n    max_metric_calls: 50,\n    candidate_selector: selector\n  )\n```\n\n### LLM-Based Instruction Proposal (NEW!)\n\nUse an LLM to propose improved component instructions based on reflective feedback. You can also provide a custom proposal template.\n\n```elixir\nreflection_llm = GEPA.LLM.ReqLLM.new(provider: :openai, model: \"gpt-4o-mini\")\n\ncustom_template = \"\"\"\nImprove {component_name}:\nCurrent: {current_instruction}\nFeedback: {reflective_dataset}\nNew instruction:\n\"\"\"\n\n{:ok, result} = GEPA.optimize(\n  seed_candidate: %{\"instruction\" =\u003e \"You are a concise math tutor.\"},\n  trainset: trainset,\n  valset: valset,\n  adapter: adapter,\n  max_metric_calls: 50,\n  reflection_llm: reflection_llm,\n  proposal_template: custom_template\n)\n```\n\nWhen `reflection_llm` is not provided, GEPA falls back to a simple testing-only improvement marker (`\"[Optimized]\"`).\n\n### Interactive Livebooks (NEW!)\n\nFor interactive learning and experimentation:\n\n```bash\n# Install Livebook\nmix escript.install hex livebook\n\n# Open a livebook\nlivebook server livebooks/01_quick_start.livemd\n```\n\nAvailable Livebooks:\n- `01_quick_start.livemd` - Interactive introduction\n- `02_advanced_optimization.livemd` - Parameter tuning and visualization\n- `03_custom_adapter.livemd` - Build adapters interactively\n\nSee [livebooks/README.md](livebooks/README.md) for details!\n\n### With State Persistence\n\n```elixir\n{:ok, result} = GEPA.optimize(\n  seed_candidate: seed,\n  trainset: trainset,\n  valset: valset,\n  adapter: GEPA.Adapters.Basic.new(),\n  max_metric_calls: 100,\n  run_dir: \"./my_optimization\"  # State saved here, can resume\n)\n```\n\n## Development\n\n```bash\n# Get dependencies\nmix deps.get\n\n# Run tests\nmix test\n\n# Run with coverage\nmix test --cover\n\n# Run specific tests\nmix test test/gepa/utils/pareto_test.exs\n\n# Format code\nmix format\n\n# Type checking\nmix dialyzer\n```\n\n## Architecture\n\nBased on behavior-driven design with functional core:\n\n```\nGEPA.optimize/1\n  ↓\nGEPA.Engine ← Behaviors → User Implementations\n  ├─→ Adapter (evaluate, reflect, propose)\n  ├─→ Proposer (reflective, merge)\n  ├─→ Strategies (selection, sampling, evaluation)\n  └─→ StopCondition (budget, time, threshold)\n```\n\n## Documentation\n\n### Technical Documentation\n- [Technical Design](docs/TECHNICAL_DESIGN.md)\n- [LLM Adapter Design](docs/llm_adapter_design.md) - Design for real LLM integration\n- [Completing the Port (Plans)](docs/20251129/completing-the-port/README.md)\n\n## Changelog\n\n### v0.1.2 (2025-11-29)\n- Epsilon-greedy candidate selector with decay/reset and stateful selector support in engine/proposer\n- Telemetry event schema and LLM-backed instruction proposal with custom templates\n- Reflective proposer consumes instruction proposals with fallback marker when no LLM is provided\n- Docs for completing the port and telemetry-first experiment tracking\n\n### v0.1.1 (2025-11-29)\n- Documentation cleanup and release tagging\n\n### v0.1.0 (2025-10-29)\n- Initial release with Phase 1 \u0026 2 complete\n- Production LLM integration (OpenAI GPT-4o-mini, Google Gemini Flash Lite)\n- Core optimization engine with reflective and merge proposers\n- Incremental evaluation and advanced stop conditions\n- 218 tests passing with 75.4% coverage\n\n## Related Projects\n\n- [GEPA Python](https://github.com/gepa-ai/gepa) - Original implementation\n- [GEPA Paper](https://arxiv.org/abs/2507.19457) - Research paper\n\n## License\n\n[MIT License](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnshkrdotcom%2Fgepa_ex","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnshkrdotcom%2Fgepa_ex","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnshkrdotcom%2Fgepa_ex/lists"}