{"id":34787906,"url":"https://github.com/shobrook/promptimal","last_synced_at":"2025-12-25T09:33:46.218Z","repository":{"id":266754872,"uuid":"886682265","full_name":"shobrook/promptimal","owner":"shobrook","description":"A very fast, very minimal prompt optimizer","archived":false,"fork":false,"pushed_at":"2025-01-05T16:56:53.000Z","size":5057,"stargazers_count":288,"open_issues_count":3,"forks_count":12,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-09-30T21:13:04.487Z","etag":null,"topics":["evolutionary-algorithm","genetic-algorithm","prompt-engineering","prompt-learning","prompt-optimization","prompt-tuning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shobrook.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-11T12:18:15.000Z","updated_at":"2025-09-28T10:16:02.000Z","dependencies_parsed_at":"2024-12-06T00:30:37.098Z","dependency_job_id":null,"html_url":"https://github.com/shobrook/promptimal","commit_stats":null,"previous_names":["shobrook/promptimal"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/shobrook/promptimal","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shobrook%2Fpromptimal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shobrook%2Fpromptimal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shobrook%2Fpromptimal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shobrook%2Fpromptimal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shobrook","download_url":"https://codeload.github.com/shobrook/promptimal/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shobrook%2Fpromptimal/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28025635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-25T02:00:05.988Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["evolutionary-algorithm","genetic-algorithm","prompt-engineering","prompt-learning","prompt-optimization","prompt-tuning"],"created_at":"2025-12-25T09:33:37.487Z","updated_at":"2025-12-25T09:33:46.211Z","avatar_url":"https://github.com/shobrook.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# promptimal\n\n**CLI for quickly improving your AI prompts. No dataset needed.**\n\nJust submit your prompt and a description of what you want to improve. Promptimal will then use a genetic algorithm to iteratively refine the prompt until it's better than the original. An LLM evaluates the modified prompts to guide the process, but you can also define your own evaluation function.\n\n![Demo](./assets/demo.gif)\n\n## Installation\n\n```bash\n\u003e pipx install promptimal\n```\n\nOnce installed, make sure you have your OpenAI API key added to your environment:\n\n```bash\n\u003e export OPENAI_API_KEY=\"...\"\n```\n\n## Quickstart\n\nOpen the tool from your terminal:\n\n```bash\n\u003e promptimal\n```\n\nYou'll be asked to input your initial prompt and what you want to improve. Alternatively, you can specify these inputs as command-line arguments:\n\n```bash\n\u003e promptimal \\\n    --prompt \"You will be provided with a piece of code, and your task is to explain it in a concise way.\" \\\n    --improve \"Summaries need to include less code references and be more high-level.\"\n```\n\nOnce you're done, a UI will open in your terminal for monitoring the optimization process:\n\n\u003cimg src=\"./assets/demo.png\" width=\"720\" /\u003e\n\n## Advanced usage\n\n### Hyperparameters\n\nYou can control the optimization parameters by passing additional command-line arguments:\n\n```bash\n\u003e promptimal --num_iters=10 --num_samples=20 --threshold=0.7\n```\n\n1. `num_iters`: Number of iterations to run the optimization loop for. Equivalent to the number of \"generations\" in an evolutionary algorithm.\n2. `num_samples`: Number of candidate prompts to generate in each iteration. Equivalent to the \"population size\" in an evolutionary algorithm.\n3. `threshold`: Termination threshold for the loop. If a candidate prompt gets a score higher than this threshold, the optimization loop will stop. Default is 1.0.\n\n### Custom evaluators\n\nBy default, promptimal uses an LLM-as-judge approach (with self-consistency) to evaluate prompt candidates. But to boost performance, you may want to evaluate prompts against a dataset or use some other evaluation technique. To do this, first create a Python file called `evaluator.py`. Then copy/paste the code below into that file and define your own evaluation function:\n\n```python\nimport argparse\n\ndef evaluator(prompt: str) -\u003e float:\n    # Your code goes here\n    # Must return value between 0 and 1\n\ndef main():\n    parser = argparse.ArgumentParser()\n    parser.add_argument(\"--prompt\", required=True, type=str)\n    args = parser.parse_args()\n\n    score = evaluator(args.prompt)\n    print(score)\n\nif __name__ == \"__main__\":\n    main()\n```\n\nOnce finished, specify the path to `evaluator.py` when you run promptimal:\n\n```bash\n\u003e promptimal --evaluator=\"path/to/evaluator.py\"\n```\n\nThis file will effectively serve as a script that promptimal uses to evaluate prompts.\n\n## Roadmap\n\n1. Support for other LLM providers, like Anthropic, Groq, etc. And ollama for local models.\n2. Evolve not only the prompts, but the meta-prompts (based on the [PromptBreeder paper](https://arxiv.org/pdf/2309.16797)).\n3. Pre-define some mutation operators.\n4. Generate synthetic tests as part of the evaluation process.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshobrook%2Fpromptimal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshobrook%2Fpromptimal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshobrook%2Fpromptimal/lists"}