https://github.com/servicenow/doomarena
DoomArena is a Framework for Testing AI Agents Against Evolving Security Threats
https://github.com/servicenow/doomarena
ai ai-safety attack browsergym defense llm machine machine-learning red-teaming security taubench web-agents
Last synced: 3 months ago
JSON representation
DoomArena is a Framework for Testing AI Agents Against Evolving Security Threats
- Host: GitHub
- URL: https://github.com/servicenow/doomarena
- Owner: ServiceNow
- License: apache-2.0
- Created: 2025-03-17T18:46:04.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-09-12T20:59:58.000Z (4 months ago)
- Last Synced: 2025-09-25T01:23:03.324Z (4 months ago)
- Topics: ai, ai-safety, attack, browsergym, defense, llm, machine, machine-learning, red-teaming, security, taubench, web-agents
- Language: Python
- Homepage: https://servicenow.github.io/DoomArena/
- Size: 6.46 MB
- Stars: 45
- Watchers: 1
- Forks: 6
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION.cff
- Security: SECURITY.md
Awesome Lists containing this project
README
# DoomArena: A Framework for Testing AI Agents Against Evolving Security Threats
[](https://pypi.org/project/doomarena/)
[]([https://opensource.org/licenses/MIT](http://www.apache.org/licenses/LICENSE-2.0))
[](https://pypistats.org/packages/doomarena)
[](https://star-history.com/#ServiceNow/DoomArena)
[DoomArena](https://servicenow.github.io/DoomArena/) is a modular, configurable, plug-in security testing framework for AI agents that supports many agentic frameworks including [$\tau$-bench](https://github.com/sierra-research/tau-bench), [Browsergym](https://github.com/ServiceNow/browsergym), [OSWorld](https://github.com/xlang-ai/OSWorld) and [TapeAgents](https://github.com/ServiceNow/tapeagents) (see Mail agent example). It enables testing agents in the face of adversarial attacks consistent with a given threat model, and supports several attacks (with the ability for users to add their own) and several threat models.
## 🚀 Quick Start
The [DoomArena Intro Notebook](https://colab.research.google.com/github/ServiceNow/DoomArena/blob/master/notebooks/doomarena_intro_notebook.ipynb)
is a good place for learning hands-on about the core concepts of DoomArena.
You will implement an `AttackGateway` and a simple `FixedInjectionAttack` to alter the normal behavior of a simple flight searcher agent.
If you only want to use the library just run
```bash
pip install doomarena # core library, minimal dependencies
```
If you want to run DoomArena integrated with [TauBench](https://github.com/sierra-research/tau-bench/), additionally run
```bash
pip install doomarena-taubench # optional
```
If you want to run DoomArena integrated with [Browsergym](https://github.com/ServiceNow/BrowserGym), additionally run
```bash
pip install doomarena-browsergym # optional
```
If you want to test attacks on a Mail Agent (which can summarize and send emails on your behalf) inspired by the [LLMail Challenge](https://llmailinject.azurewebsites.net/) run
```bash
pip install -e doomarena/mailinject # optional
```
If you want to run DoomArena integrated with [OSWorld](https://github.com/xlang-ai/OSWorld) run
```
pip install -e doomarena/osworld
```
and follow our setup instructions [here](doomarena/osworld/README.md).
Export relevant API keys into your environment or `.env` file.
```bash
OPENAI_API_KEY=""
OPENROUTER_API_KEY=""
```
## 🛠️ Advanced Setup
To actively develop `DoomArena`, please create a virtual environment and install the package locally in editable mode using
```bash
pip install -e doomarena/core
pip install -e doomarena/taubench
pip install -e doomarena/browsergym
pip install -e doomarena/mailinject
pip install -e doomarena/osworld
```
Once the environments are set up, run the tests to make sure everything is working.
```bash
make ci-tests
make tests # requires openai key
```
## 💻 Running Experiments
Follow the environment-specific instructions for [TauBench](doomarena/taubench/README.md) and [BrowserGym](doomarena/browsergym/README.md)
## 🌟 Contributors
[](https://github.com/ServiceNow/DoomArena/graphs/contributors)
Note: contributions made prior to the open-sourcing are not accounted for; please refer to author list for full list of contributors.
## 📝 Paper
If you found DoomArena helpful, please cite us
```
@misc{boisvert2025doomarenaframeworktestingai,
title={DoomArena: A framework for Testing AI Agents Against Evolving Security Threats},
author={Leo Boisvert and Mihir Bansal and Chandra Kiran Reddy Evuru and Gabriel Huang and Abhay Puri and Avinandan Bose and Maryam Fazel and Quentin Cappart and Jason Stanley and Alexandre Lacoste and Alexandre Drouin and Krishnamurthy Dvijotham},
year={2025},
eprint={2504.14064},
archivePrefix={arXiv},
primaryClass={cs.CR},
url={https://arxiv.org/abs/2504.14064},
}
```