Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ShengranHu/ADAS
Automated Design of Agentic Systems
https://github.com/ShengranHu/ADAS
Last synced: about 1 month ago
JSON representation
Automated Design of Agentic Systems
- Host: GitHub
- URL: https://github.com/ShengranHu/ADAS
- Owner: ShengranHu
- License: apache-2.0
- Created: 2024-07-31T16:08:55.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2024-08-20T20:05:05.000Z (about 1 month ago)
- Last Synced: 2024-08-21T03:16:18.776Z (about 1 month ago)
- Language: Python
- Homepage:
- Size: 10.5 MB
- Stars: 253
- Watchers: 6
- Forks: 30
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-agents - ADAS
README
Automated Design of Agentic Systems
In this work, we describe a newly forming research area **A**utomated **D**esign of **A**gentic **S**ystems (**ADAS**), which aims to *automatically create powerful agentic system designs, including inventing novel building blocks and/or combining them in new ways.*
We present a simple yet effective ADAS algorithm named **Meta Agent Search** to demonstrate that **agents can invent novel and powerful agent designs**. In Meta Agent Search, a "meta" agent iteratively *programs* interesting new agents in code based on previous discoveries.
## Setup
```bash
conda create -n adas python=3.11
conda activate adas
pip install -r requirements.txt
# provide your OpenAI API key
export OPENAI_API_KEY="YOUR KEY HERE"
```
## Running Instructions
### Running Meta Agent Search
To run experiments for each domain, navigate to its respective folder. The code in each folder is self-contained. Launch experiments using the `search.py` script located in each domain's folder.
```bash
python {DOMAIN}/search.py
```
Replace `{DOMAIN}` with the specific domain folder name {`_arc`, `_drop`, `_mgsm`, ...} to run the experiment for.
### Customizing Meta Agent Search for New Domains
You can easily adapt the code to search for new domains. To do so, follow these steps:
1. Modify the `evaluate_forward_fn()` function and adjust any necessary formatting prompts (e.g. [this line](https://github.com/ShengranHu/ADAS/blob/main/_mmlu/search.py#L89)) in the `search.py` file.
2. Consider adding additional basic functions for the meta agent to utilize during the design process (similar to [this line](https://github.com/ShengranHu/ADAS/blob/main/_arc/search.py#L161)).
3. Update the domain-specific information within the prompts to match the requirements of your new domain (e.g. [this line](https://github.com/ShengranHu/ADAS/blob/main/_mmlu/mmlu_prompt.py#L229)).
4. Run the search and evaluation on your new domain.
### Safety Consideration
> [!WARNING]
> The code in this repository involves executing untrusted model-generated code. We strongly advise users to be aware of this safety concern. While it is highly unlikely that model-generated code will perform overtly malicious actions in our current settings and with the models we use, such code may still act destructively due to limitations in model capability or alignment. By using this repository, you acknowledge and accept these risks.
## Citing
If you find this project useful, please consider citing:
```
@article{hu2024ADAS,
title={Automated Design of Agentic Systems},
author={Hu, Shengran and Lu, Cong and Clune, Jeff},
journal={arXiv preprint arXiv:2408.08435},
year={2024}
}
```