https://github.com/zjunlp/synworld
[ACL 2025] SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement
https://github.com/zjunlp/synworld
agent agent-planning agentic artificial-intelligence knowledge large-language-models natural-language-processing synworld
Last synced: about 1 month ago
JSON representation
[ACL 2025] SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement
- Host: GitHub
- URL: https://github.com/zjunlp/synworld
- Owner: zjunlp
- License: mit
- Created: 2024-12-18T10:47:06.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-04-14T08:41:43.000Z (12 months ago)
- Last Synced: 2025-07-21T12:53:48.739Z (9 months ago)
- Topics: agent, agent-planning, agentic, artificial-intelligence, knowledge, large-language-models, natural-language-processing, synworld
- Language: Python
- Homepage:
- Size: 346 KB
- Stars: 7
- Watchers: 4
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
SynWorld
Virtual Scenario Synthesis for Agentic Action Knowledge Refinement
[](https://opensource.org/licenses/MIT)
## Table of Contents
- 🌻[Acknowledgement](#acknowledgement)
- 🌟[Overview](#overview)
- 🔧[Installation](#installation)
- ✏️[ToolBench-API-Deploy](#toolbench-api-deploy)
- 📝[MCTS-refine](#MCTS-refine)
- 🤔[ActionKnowledge-Evaluation](#ActionKnowledge-evaluation)
- 🚩[Citation](#citation)
---
## 🌻Acknowledgement
Our code of training module is referenced and adapted from [StableToolbench](https://github.com/THUNLP-MT/StableToolBench), [Aflow](https://github.com/FoundationAgents/AFlow), [Draft](https://github.com/quchangle1/DRAFT). And the Dataset is collected from [ToolBench](https://github.com/openbmb/toolbench?tab=readme-ov-file), [HotpotQA](). Our end-to-end evaluation module is based on [Stable ToolBench](https://github.com/THUNLP-MT/StableToolBench), [HotpotQA](https://github.com/hotpotqa/hotpot). Thanks for their great contributions!
## 🌟Overview
In the interaction between agents and their environments, agents expand their capabilities by planning and executing actions. However, LLM-based agents face substantial challenges when deployed in novel environments or required to navigate unconventional action spaces. To empower agents to autonomously explore environments, optimize workflows, and enhance their understanding of actions, we propose SynWorld, a framework that allows agents to synthesize possible scenarios with multi-step action invocation within the action space and perform Monte Carlo Tree Search (MCTS) exploration to effectively refine their action knowledge in the current environment. Our experiments demonstrate that SynWorld is an effective and general approach to learning action knowledge in new environments
## 🔧Installation
```bash
git clone https://github.com/zjunlp/SynWorld
cd SynWorld
pip install -r requirements.txt
```
## ✏️ToolBench-API-Deploy
```bash
git clone https://github.com/THUNLP-MT/StableToolBench.git
cd StableToolBench
pip install requirements.txt
cd server
python main.py
```
## 📝MCTS—refine
Generate workflow with local llm api
```bash
cd SynWrold/src
python PlanAlign.py \
--model qwen-max \
--max_rounds 15 \
--dataset Toolbench \
--validate_nums 100 \
--max_workers 16 \
--mode dev \
--overwrite
```
## 🤔ActionKnowledge-Evaluation
Evaluation the workflow of node n
```bash
cd SynWrold/src
python PlanAlign.py \
--model qwen-max \
--dataset Toolbench \
--validate_nums 100 \
--max_workers 16 \
--mode test \
--test_round the_round_num_you_select(eg. 100 in our experiment) \
--overwrite
```
## 🚩Citation
If this work is helpful, please kindly cite as:
```bibtex
@article{fang2025synworld,
title={SynWorld: Virtual Scenario Synthesis for Agentic Action Knowledge Refinement},
author={Fang, Runnan and Wang, Xiaobin and Liang, Yuan and Qiao, Shuofei and Wu, Jialong and Xi, Zekun and Zhang, Ningyu and Jiang, Yong and Xie, Pengjun and Huang, Fei and others},
journal={arXiv preprint arXiv:2504.03561},
year={2025}
}
```