https://github.com/xxyQwQ/GenAgent

https://github.com/xxyQwQ/GenAgent

Last synced: 3 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/xxyQwQ/GenAgent
• Owner: xxyQwQ
• Created: 2024-09-01T08:35:29.000Z (3 months ago)
• Default Branch: main
• Last Pushed: 2024-09-09T03:41:47.000Z (3 months ago)
• Last Synced: 2024-09-09T04:56:47.500Z (3 months ago)
• Language: Python
• Size: 8.04 MB
• Stars: 51
• Watchers: 2
• Forks: 3
• Open Issues: 2
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• ai-game-devtools - GenAgent - Case Studies on ComfyUI. |[arXiv](https://arxiv.org/abs/2409.01392) | | Agent | (<span id="game">Game (Agent)</span> / <span id="tool">Tool (AI LLM)</span>)

README

        


GenAgent: Build Collaborative AI Systems with Automated Workflow Generation - Case Studies on ComfyUI





    Implementation for the "GenAgent: Build Collaborative AI Systems with Automated Workflow Generation - Case Studies on ComfyUI" paper.

    


    Xiangyuan Xue

     

    Zeyu Lu

     

    Di Huang

     

    Wanli Ouyang

     

    Lei Bai*

    






      

      

      



![Teaser](assets/image/teaser.png)

The GenAgent framework builds collaborative AI systems by creating workflows. The workflows are converted into code so that LLM agents can better understand them. GenAgent can learn from human-designed workflows and create new ones. The generated workflows can be interpreted as collaborative systems to complete complex tasks.

## 🛠️ Configuration

First, clone the repository and navigate to the project directory:

```bash

git clone https://github.com/xxyQwQ/GenAgent

cd GenAgent

```

Then, create a new conda environment and install the dependencies:

```bash

conda create -n genagent python=3.12

conda activate genagent

pip install -r requirements.txt

```

Finally, modify the `config.yaml` file to set HTTP proxy address, OpenAI API key, and ComfyUI server address. Before executing the workflows, you should manually install all the required extensions in ComfyUI.

## 🚀 Execution

Run the following command to execute the GenAgent pipeline:

```bash

# See `main.py` for more parameter settings

python main.py \

    --query_text "input your task requirement" \

    --agent_name "gen_agent" \

    --save_path "path/to/save/result"

```

The log file together with the generated workflow will be saved in the specified path. If your ComfyUI server is working properly, the workflow will be executed automatically, and the result will also be saved in the specified path.

Run the following commands to reproduce the experiments in the paper:

```bash

# Benchmark evaluation

python inference.py --agent_name "zero_shot_agent" "few_shot_agent" "cot_agent" "rag_agent" "gen_agent"

python evaluation.py --agent_name "zero_shot_agent" "few_shot_agent" "cot_agent" "rag_agent" "gen_agent"

# Abalation study

python inference.py --agent_name "json_gen_agent" "list_gen_agent" "code_gen_agent" --num_fixes 0

python evaluation.py --agent_name "json_gen_agent" "list_gen_agent" "code_gen_agent"

```

Make sure that you have set your OpenAI API key in the `config.yaml` file. The benchmark evaluation will cost approximately 30 dollars.

## 🎬 Gallery

Here are some examples of the generated workflows. You can visit our [project page](https://xxyqwq.github.io/GenAgent) for more details.

![Example_1](assets/image/example_1_overview.png)

![Example_2](assets/image/example_2_overview.png)