{"id":23934047,"url":"https://github.com/NicerWang/ToolCommander","last_synced_at":"2025-09-11T16:33:39.710Z","repository":{"id":261377666,"uuid":"872854830","full_name":"NicerWang/ToolCommander","owner":"NicerWang","description":"Official implementation of \"From Allies to Adversaries: Manipulating LLM Tool Scheduling through Adversarial Injection\".","archived":false,"fork":false,"pushed_at":"2024-12-23T07:46:48.000Z","size":334,"stargazers_count":1,"open_issues_count":1,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2024-12-23T08:23:34.581Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2412.10198","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NicerWang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-15T07:32:28.000Z","updated_at":"2024-12-23T08:17:46.000Z","dependencies_parsed_at":"2024-12-25T19:38:07.020Z","dependency_job_id":null,"html_url":"https://github.com/NicerWang/ToolCommander","commit_stats":null,"previous_names":["anonymous-lgtm/toolcommander","nicerwang/toolcommander"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NicerWang%2FToolCommander","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NicerWang%2FToolCommander/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NicerWang%2FToolCommander/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NicerWang%2FToolCommander/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NicerWang","download_url":"https://codeload.github.com/NicerWang/ToolCommander/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232658738,"owners_count":18556992,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-06T00:30:04.413Z","updated_at":"2025-09-11T16:33:39.695Z","avatar_url":"https://github.com/NicerWang.png","language":"Python","funding_links":[],"categories":["Building"],"sub_categories":["Tools"],"readme":"# ToolCommander: Adversarial Tool Scheduling Framework\n\n[Paper Here](https://arxiv.org/abs/2412.10198)\n\nThis repository contains the official implementation of the paper, \"**From Allies to Adversaries: Manipulating LLM Tool Scheduling through Adversarial Injection**\". The paper introduces **ToolCommander**, a novel framework that identifies and exploits vulnerabilities in the tool scheduling mechanisms of Large Language Model (LLM) agents. By leveraging adversarial tool injection, ToolCommander can lead to privacy theft, denial-of-service (DoS) attacks, and the manipulation of tool-calling behaviors.\n\n![ToolCommander](./pages/src/assets/1-commander.webp)\n\n## Table of Contents\n\n- [Data](#data)\n- [Prerequisites](#prerequisites)\n- [Usage](#usage)\n- [Baselines](#baselines)\n- [Citation](#citation)\n\n---\n\n## Data\n\nThe dataset used in this project is located in the `data` directory. The files follow this naming convention:\n\n```\ng1_\u003ctrain/eval\u003e_\u003ca/b/c\u003e.json\n```\n\nWhere:\n\n- `g1` refers to the original category from the **ToolBench** dataset.\n- `train` and `eval` denote the training and evaluation sets, respectively.\n- `a`, `b`, and `c` represent different keywords used to generate the data:\n  - `a`: **YouTube**\n  - `b`: **Email**\n  - `c`: **Stock**\n\n### ToolBench Dataset\n\nIn addition to the provided data, you will need to download the **ToolBench** dataset from its [official repository](https://github.com/OpenBMB/ToolBench). Specifically, you will need the following components:\n\n- `corpus.tsv`\n- `tools` folder\n\nOnce downloaded, place the dataset in the `data/toolbench` directory. The final directory structure should look like this:\n\n```\n/data\n├── toolbench\n│   ├── corpus.tsv\n│   └── tools\n│       ├── ...\n├── g1_train_a.json\n├── g1_train_b.json\n├── g1_train_c.json\n├── g1_eval_a.json\n├── g1_eval_b.json\n├── g1_eval_c.json\n└── ...\n```\n\n---\n\n## Prerequisites\n\nTo set up the environment, first install the required dependencies:\n\n```bash\npip install -r requirements.txt\n```\n\n### OpenAI API Setup\n\nFor evaluation using OpenAI's models, you need to set the `OPENAI_API_KEY` environment variable with your OpenAI API key. Detailed instructions can be found in the [OpenAI API documentation](https://platform.openai.com/docs/quickstart#create-and-export-an-api-key).\n\n---\n\n## Usage\n\nWe provide several scripts to help reproduce the results presented in the paper.\n\n### Running the Adversarial Attack\n\nTo execute the adversarial injection attack and evaluate the results, use the following command:\n\n```bash\nbash attack_all.sh \u0026\u0026 bash eval_all.sh\n```\n\n- `attack_all.sh`: Executes the adversarial injection attack across all retrievers and datasets.\n- `eval_all.sh`: Evaluates the performance of the retrievers after the attack.\n\nThe results will be printed directly in the console.\n\n---\n\n## Baselines\n\nWe compare ToolCommander against the `PoisonedRAG` baseline. For more details, visit the [PoisonedRAG repository](https://github.com/sleeepeer/PoisonedRAG).\n\n### Baseline Data\n\nThe attack results generated by `PoisonedRAG` have been provided in the `data` directory as:\n\n```\ng1_train_{a/b/c}_poisonedRAG_generated.pkl\n```\n\n### Baseline Evaluation\n\nTo evaluate the baseline performance, run the following command:\n\n```bash\npython evaluate.py --data_path data/g1_train_{a/b/c}.json --attack_path data/g1_train_{a/b/c}_poisonedRAG_generated.pkl\n```\n\n## Citation\n\nIf you find this work useful, please consider citing the following paper:\n\n```bibtex\n@inproceedings{zhang-etal-2025-allies,\n    title = \"From Allies to Adversaries: Manipulating {LLM} Tool-Calling through Adversarial Injection\",\n    author = \"Zhang, Rupeng  and\n      Wang, Haowei  and\n      Wang, Junjie  and\n      Li, Mingyang  and\n      Huang, Yuekai  and\n      Wang, Dandan  and\n      Wang, Qing\",\n    editor = \"Chiruzzo, Luis  and\n      Ritter, Alan  and\n      Wang, Lu\",\n    booktitle = \"Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)\",\n    month = apr,\n    year = \"2025\",\n    address = \"Albuquerque, New Mexico\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2025.naacl-long.101/\",\n    doi = \"10.18653/v1/2025.naacl-long.101\",\n    pages = \"2009--2028\",\n    ISBN = \"979-8-89176-189-6\"\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNicerWang%2FToolCommander","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FNicerWang%2FToolCommander","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNicerWang%2FToolCommander/lists"}