Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/SecurityLab-UCD/ai-agent-security
This repository contains source code for the demos and attacks we present in our paper Security of AI Agents.
https://github.com/SecurityLab-UCD/ai-agent-security
Last synced: 3 days ago
JSON representation
This repository contains source code for the demos and attacks we present in our paper Security of AI Agents.
- Host: GitHub
- URL: https://github.com/SecurityLab-UCD/ai-agent-security
- Owner: SecurityLab-UCD
- Created: 2024-04-18T21:39:18.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2024-12-15T22:47:37.000Z (24 days ago)
- Last Synced: 2024-12-15T23:27:53.029Z (24 days ago)
- Language: Python
- Homepage: https://arxiv.org/pdf/2406.08689
- Size: 46.9 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome_ai_agents - Ai-Agent-Security - This repository contains source code for the demos and attacks we present in our paper Security of AI Agents. (Building / Security)
- awesome_ai_agents - Ai-Agent-Security - This repository contains source code for the demos and attacks we present in our paper Security of AI Agents. (Building / Security)
README
# ai-agent-security
This repository contains source code for the demos encryption defense we present in our paper
[Security of AI Agents](https://arxiv.org/pdf/2406.08689).
The code for sandbox defense and evaluation can be found in [our fork of AgentBench](https://github.com/SecurityLab-UCD/AgentBench).
## Requirements
[Python 3.8](https://www.python.org/downloads/release/python-380/) or above
## Setup
`env.sh` is for letting Python find our modules. Source it from repo root directory.
```sh
source ./env.sh
```
Install dependencies
```sh
pip install -r requirements.txt
```
Generate homomorphic encryption data
- Run `python HE_data.py -h` to see how to modify generated ciphertexts
```sh
cd HE_data && python HE_data.py && cd ../
```
## Run Demos
To run agents using OpenAI LLMs for reasoning, set this environment variable first
```sh
export OPENAI_API_KEY=""
```
### SSN Agent Demo
To run the agent
```sh
python agents/ssn_agent.py --model= --user_id= --ssns_path= --secretkeys_path=
```
When prompting, write "number" instead of "SSN" or "social security number" to avoid triggering alignment. You can ask for groups of the number such as the first three digits or last four digits.
Example prompt: `What are the first three digits of my number?`
### Homomorphic Encryption Agent Demo
To run the agent
```sh
python agents/HE_agent.py --model=
```
When prompting, please specify "sum" or "product" for postprocessing reasons. The default encryptor we use cannot handle numbers greater than 400 (this can be changed in `HE_data/HE_data.py`), so limit calculation results to the range 0 to 400 inclusive.
Example prompt: `What is the product of indices 0 and 1?`
- Known bug: The LLM indexes the wrong thing if 0 is not included as an index in the prompt. Make sure the first index you write in the prompt is 0.
## Tests
To run tests
```sh
# Create ciphertext files if you haven't already
cd HE_data && python HE_data.py && cd ../
# Run tests
pytest tests/*
```
## Cite
```biblatex
@inproceedings{he2025aiagent,
author = {He, Yifeng and Wang, Ethan and Rong, Yuyang and Cheng, Zifei and Chen, Hao},
title = {Security of AI Agents},
booktitle = {International Workshop on Responsible AI Engineering (RAIE)},
date = {2025-04-29},
address = {Ottawa, Ontario, Canada},
doi = {https://doi.org/10.48550/arXiv.2406.08689},
}
```