{"id":27948317,"url":"https://github.com/pku-alignment/proagent","last_synced_at":"2025-05-07T14:57:34.093Z","repository":{"id":190305494,"uuid":"682354120","full_name":"PKU-Alignment/ProAgent","owner":"PKU-Alignment","description":"AAAI24(Oral) ProAgent: Building Proactive Cooperative Agents with Large Language Models","archived":false,"fork":false,"pushed_at":"2025-03-04T13:38:55.000Z","size":2407,"stargazers_count":81,"open_issues_count":0,"forks_count":9,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-05-07T14:57:27.556Z","etag":null,"topics":["cooperative","cooperative-ai","human-ai","human-ai-interaction","language-model","llm-agent","overcooked"],"latest_commit_sha":null,"homepage":"https://pku-proagent.github.io/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PKU-Alignment.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-08-24T01:56:07.000Z","updated_at":"2025-04-26T02:57:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"a5bfbda8-8817-48f0-93da-ed3e96cef29e","html_url":"https://github.com/PKU-Alignment/ProAgent","commit_stats":null,"previous_names":["ceyaozhang/proagent","pku-alignment/proagent"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-Alignment%2FProAgent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-Alignment%2FProAgent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-Alignment%2FProAgent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PKU-Alignment%2FProAgent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PKU-Alignment","download_url":"https://codeload.github.com/PKU-Alignment/ProAgent/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252902623,"owners_count":21822257,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cooperative","cooperative-ai","human-ai","human-ai-interaction","language-model","llm-agent","overcooked"],"created_at":"2025-05-07T14:57:33.351Z","updated_at":"2025-05-07T14:57:34.069Z","avatar_url":"https://github.com/PKU-Alignment.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ProAgent: Building Proactive Cooperative Agents with Large Language Models\n\n## Getting Started\n\n### How to setup\nWe highly recommend to use Linux or WSL to run instead of Windows, especially you should VPN to link OpenAI API.\nIt is useful to setup a conda environment with Python 3.7 (python3.8 not support tf1 which is the precondition package used in the baselines):\n- Install requirements\n    - Do it directly by `conda env create --file environment.yaml`, which will automatically install the conda env, and all Python dependencies in it.\n    - Another choice:\n        ```\n        conda create -n proagent python=3.7\n        conda activate proagent\n\n        pip install -r requirements.txt \n        conda install mpi4py==3.1.4 # pip install often fails\n        ```\n\n    Notes: Here we use `tf1-cpu`. If you are only familiar with `pytorch`. Don't worry! The baseline models are trained with tf1, so we have to use tf1 to load the model and we ONLY use tf1 when loading the models.\n\n- Install the game environment `overcooked_ai` and another supporting package `stable_baselines` locally.\n    ```\n    cd ./lib/overcooked_ai\n    pip install -e .\n\n    \n    cd ./lib/stable_baselines\n    pip install -e .\n    ```\n    Notes 1: Overcooked-AI is a benchmark environment for fully cooperative human-AI task performance, based on the wildly popular video game Overcooked. https://github.com/HumanCompatibleAI/overcooked_ai.\n    Notes 2: `stable_baselines` package is only used for supporting `BC` method, which is trained through the GAIL in `stable_baselines`. \n\n\n- Download the baselines. We now support baselines: Greedy, Random, Behavior Cloning(BC), Self-Play(SP),Population-Based Training(PBT), [Fictitious Co-Play (FCP)](https://arxiv.org/abs/2110.08176), [Maximum Entropy Population-Based Training (MEP)](https://arxiv.org/abs/2112.11701), [Cooperative Open-ended Learning (COLE)](https://arxiv.org/abs/2302.04831).\n    - Greedy (rule-based) and Random algorithms are built-in methods in the *overcooked_ai* package.\n    - For those learning-based algorithms, COLE, FCP, MEP, PBT, SP. Thanks to [COLE-Platform](https://github.com/liyang619/COLE-Platform), we can download those trained models in [here](https://drive.google.com/drive/folders/1s88a_muyG6pVlfcKDKop6R1Fhxr8dcGH) directly, with the structure `layout/model`. \n\n        - Note: The layout names in code and google drive are not aligned with the layout names in recent papers. Here is the mapping:\n            ```\n            PYTHON_LAYOUT_NAME_TO_ENV_NAME = {\n                \"unident_s\": \"Asymmetric Advantages\",\n                \"simple\": \"Cramped Room\",\n                \"random1\": \"Coordination Ring\",\n                \"random0\": \"Forced Coordination\",\n                \"random3\": \"Counter Circuit\"\n            }\n            ```\n            You need to download those model file and put in the dir ./models.\n\n### How to Run\n\n- Build a `./src/openai_key.txt` file and put your openai key in it. \n- Run `./src/main.py`. For example\n\n    ```\n    python main.py --layout cramped_room --p0 Greedy --p1 Greedy --horizon 20\n    python main.py --layout coordination_ring --p0 ProAgent --p1 Greedy\n    ```\n    We support five classical layouts: ['cramped_room', 'asymmetric_advantages', 'forced_coordination', 'coordination_ring', 'counter_circuit'].\n\n- To run many experiments simultaneously, you can use `./src/run.py` where you can select the layouts and algorithms you want to use.\n\n\n\n## GPT parser choice\n\n### GPT Model \n\n- Our main experiments were completed between June and August 2023 based on `gpt-3.5-turbo-0301`, which will be deprecated on June 13th 2024, see: https://platform.openai.com/docs/models/gpt-3-5.\n\n- We also test the `text-davinci-003` and `gpt-4-0314` as the comparsions in our extention version. \n    - However, the model `text-davinci-003` has already been deprecated on Januray 4th 2024 and the official recommended replacement is `gpt-3.5-turbo-instruct`, but we haven't test it yet. \n    - To use GPT-4, please insure your openai key has the right first. As to the first GPT-4 version, model `gpt-4-0314` may be shutdown at earliest 2024-06-13, and the recommended replacement is model `gpt-4-0613`.\n    - Learn more here: https://platform.openai.com/docs/deprecations. \n\n### Prompts level \n\n- `l1-p`: make plans directly without CoT \n- `l2-ap`: plans with analysis \n- `l3-aip`: plans with analysis and intention\n\n\n### Memory length\nWe now support retrival recent K dialogs and use BERT+cos to retrival top K similarity dialogs\n```\n# without any in-context in the query\npython main.py --layout cramped_room --p0 ProAgent --p1 Greedy --retrival_method recent_k --K 0\n\n# with recent 10 dialogs in the query\npython main.py --layout cramped_room --p0 ProAgent --p1 Greedy --retrival_method recent_k --K 10\n```\n\n\u003e Notes: Our full experiments (5 baselines, 5 layouts) are based on `--gpt_model gpt-3.5-turbo-0301 --prompt_level l2-ap --retrival_method recent_k --K 0or1`. \nWe did an ablation study on the impact of choosing different prompts based on `crampt room` layout.\nwe also use the different LLMs on five layouts only cooperate with Greedy method.\n\n\n## Q\u0026A \n\n\u003e [OPENAI ERROR]: Rate limit reached for default-gpt-3.5-turbo in organization org-xxxx tokens per min. Limit: 90000 / min. Current: 88214 / min. Contact us through our help\n\n\u003e [OPENAI ERROR]: Error communicating with OpenAI: HTTPSConnectionPool(host='api.openai.com', port=443): Max retries exceeded with url: /v1/chat/completions (Caused by       \nSSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1131)')))\n\u003e\n\u003e `Answer`: Check your VPN connection.\n\n\u003e [OPENAI ERROR]: You didn't provide an API key. You need to provide your API key in an Authorization header using Bearer\nauth (i.e. Authorization: Bearer YOUR_KEY), or as the password field (with blank username) if you're accessing the API\nfrom your browser and are prompted for a username and password.\n\u003e\n\u003e `Answer`: Please delete the blank lines in your `openai_key.txt` file. For example, if you have only one key, just make the file only one line.\n\n\u003e pkg_resources.DistributionNotFound: The 'overcooked_ai' distribution was not found and is required by the application\n\u003e\n\u003e `Answer`: You may forget to install the `overcooked_ai` package.\n\n\n## Citation\n\n```bibtex\n@inproceedings{zhang2024proagent,\n  title={Pro{A}gent: Building Proactive Cooperative Agents with Large Language Models},\n  author={Zhang, Ceyao and Yang, Kaijie and Hu, Siyi and Wang, Zihao and Li, Guanghe and Sun, Yihang and Zhang, Cheng and Zhang, Zhaowei and Liu, Anji and Zhu, Song-Chun and Chang, Xiaojun and Zhang, Junge and Yin, Feng and Liang, Yitao  and Yang, Yaodong},\n  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},\n  volume={38},\n  number={16},\n  pages={17591--17599},\n  year={2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpku-alignment%2Fproagent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpku-alignment%2Fproagent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpku-alignment%2Fproagent/lists"}