{"id":13624259,"url":"https://github.com/google-deepmind/opro","last_synced_at":"2025-06-17T00:39:53.238Z","repository":{"id":206610101,"uuid":"699169039","full_name":"google-deepmind/opro","owner":"google-deepmind","description":"official code for \"Large Language Models as Optimizers\"","archived":false,"fork":false,"pushed_at":"2024-12-04T21:04:50.000Z","size":4725,"stargazers_count":536,"open_issues_count":6,"forks_count":60,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-15T20:46:47.194Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2309.03409","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-deepmind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-10-02T04:30:31.000Z","updated_at":"2025-04-15T03:23:03.000Z","dependencies_parsed_at":"2023-11-11T04:23:14.319Z","dependency_job_id":"9dbf52a1-1fb8-47bb-9fef-e311e7e947f0","html_url":"https://github.com/google-deepmind/opro","commit_stats":null,"previous_names":["googlestaging/opro"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/google-deepmind/opro","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fopro","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fopro/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fopro/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fopro/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-deepmind","download_url":"https://codeload.github.com/google-deepmind/opro/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fopro/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260268635,"owners_count":22983601,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T21:01:40.695Z","updated_at":"2025-06-17T00:39:53.217Z","avatar_url":"https://github.com/google-deepmind.png","language":"Python","funding_links":[],"categories":["Python","A01_文本生成_文本对话","3. Prompt Optimization","Tools and Code"],"sub_categories":["大语言对话模型及数据","Rust","Prompt Optimization Tools"],"readme":"# Large Language Models as Optimizers\n\nThis repository contains the code for the paper\n\n\u003e [Large Language Models as Optimizers](https://arxiv.org/abs/2309.03409)\\\n\u003e Chengrun Yang*, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, Xinyun Chen* [* Equal Contribution]\\\n\u003e _arXiv: 2309.03409_\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"img/workflow.png\" alt=\"workflow\" width=\"48%\"\u003e\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n  \u003cimg src=\"img/gpt_meta_prompt.png\" alt=\"workflow\" width=\"40%\"\u003e\n\u003c/p\u003e\n\n## Dependency requirements\n\nThe code has been verified to work under `Python 3.10.13` with the following dependencies:\n\n```\n- absl-py (2.0.0)\n- google.generativeai (0.1.0)\n- immutabledict (3.0.0)\n- openai (0.27.2)\n```\n\n## Usage\n\n### Prompt optimization \nUse `opro/optimization/optimize_instructions.py`, follow the steps at the top. \n\nA quickstarter:\n\n`\npython optimize_instructions.py --optimizer=\"gpt-3.5-turbo\" --scorer=\"text-bison\"\n--instruction_pos=\"Q_begin\" --dataset=\"gsm8k\" --task=\"train\" --palm_api_key=\"\u003cyour_palm_api_key\u003e\" --openai_api_key=\"\u003cyour_openai_api_key\u003e\"\n`\n\n### Prompt evaluation\nUse `opro/evaluation/evaluate_instructions.py`, follow the steps at the top.\n\nA quickstarter:\n\n`\npython evaluate_instructions.py --scorer=\"text-bison\" --dataset=\"gsm8k\" --task=\"test\" --instruction_pos=\"Q_begin\" --evaluate_training_fold=false --evaluate_test_fold=true --palm_api_key=\"\u003cyour_palm_api_key\u003e\"\n`\n\n### Linear regression\nUse `opro/optimization/optimize_linear_regression.py`, follow the steps at the top.\n\n\n### Traveling salesman problem\nUse `opro/optimization/optimize_tsp.py`, follow the steps at the top.\n\n\n## Supported models\n\nThe code in this repository currently supports [text-bison](https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/text) and [GPT models](https://platform.openai.com/docs/api-reference/introduction). Alternatively, you may serve your own model and plug it in here, similar to the existing prompting APIs in `opro/prompt_utils.py`.\n\n\n## Precaution on API costs\n\nCalling the PaLM or GPT APIs for prompt optimization and evaluation may incur unexpectedly large costs. Please carefully estimate the cost and/or start with lighter use (e.g., evaluate on a smaller portion of the benchmark dataset or run optimization for fewer steps) before the formal experimentations, or prompt self-served models instead.\n\n## Citation\n\nIf you have used our code in your research, please cite our [paper](https://arxiv.org/abs/2309.03409):\n\n```\n@article{yang2023large,\n  title={Large language models as optimizers},\n  author={Yang, Chengrun and Wang, Xuezhi and Lu, Yifeng and Liu, Hanxiao and Le, Quoc V and Zhou, Denny and Chen, Xinyun},\n  journal={arXiv preprint arXiv:2309.03409},\n  year={2023}\n}\n```\n\n\n*Disclaimer: this is not an officially supported Google product.*\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fopro","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-deepmind%2Fopro","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fopro/lists"}