An open API service indexing awesome lists of open source software.

https://github.com/reaatech/prompt-injection-bench

Reproducible benchmark and test corpus for prompt-injection defenses — adapters for common sanitization libraries, scored leaderboard, CI-ready. Security-adjacent, evergreen.
https://github.com/reaatech/prompt-injection-bench

adversarial-testing agentic-ai ai-safety ai-security benchmark defense llm llm-security prompt-injection red-teaming sanitization security test-corpus typescript

Last synced: 2 days ago
JSON representation

Reproducible benchmark and test corpus for prompt-injection defenses — adapters for common sanitization libraries, scored leaderboard, CI-ready. Security-adjacent, evergreen.

Awesome Lists containing this project

README

          

# prompt-injection-bench

[![CI](https://github.com/reaatech/prompt-injection-bench/actions/workflows/ci.yml/badge.svg)](https://github.com/reaatech/prompt-injection-bench/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
[![TypeScript](https://img.shields.io/badge/TypeScript-5.8-blue)](https://www.typescriptlang.org/)

> Reproducible benchmark and test corpus for evaluating prompt-injection defenses in AI agent systems.

This monorepo provides an attack corpus, pluggable defense adapters, a scoring engine, a public leaderboard, and an MCP server for benchmarking prompt-injection defenses.

## Features

- **Attack corpus** — 8 categories, 300+ templates, variant generation with obfuscation strategies
- **Defense adapters** — 8 built-in adapters (Rebuff, Lakera Guard, LLM Guard, Garak, OpenAI/Azure/Anthropic/Cohere Moderation, Custom HTTP) with a common interface
- **Scoring engine** — Weighted scoring, Wilson confidence intervals, Cohen's h effect sizes, z-tests, ANOVA
- **Leaderboard** — Public rankings with S/A/B/C/D tiers and JSON persistence
- **MCP integration** — 4 MCP tools (`run_benchmark`, `compare_defenses`, `generate_report`, `submit_results`)
- **Reproducibility** — Deterministic seeds, SHA-256 proofs, version-locked manifests
- **Security hardening** — SSRF protection, path traversal prevention, rate limiting, input validation
- **CLI tool** — 6 subcommands for benchmarking, comparing, and reporting from the terminal

## Installation

### Using the packages

Packages are published under the `@reaatech` scope (with the umbrella package unscoped) and can be installed individually:

```bash
# Core types and schemas
pnpm add @reaatech/pi-bench-core

# Defense adapters
pnpm add @reaatech/pi-bench-adapters

# Scoring engine
pnpm add @reaatech/pi-bench-scoring

# CLI and umbrella (re-exports everything)
pnpm add prompt-injection-bench
```

### Contributing

```bash
# Clone the repository
git clone https://github.com/reaatech/prompt-injection-bench.git
cd prompt-injection-bench

# Install dependencies
pnpm install

# Build all packages
pnpm build

# Run the test suite
pnpm test

# Run linting
pnpm lint
```

## Quick Start

### CLI

```bash
# Run a benchmark against the mock adapter
npx prompt-injection-bench benchmark --defense mock

# Compare two defense results
npx prompt-injection-bench compare --results rebuff.json lakera.json

# Generate an HTML report
npx prompt-injection-bench report --results latest.json --format html
```

### Library

```typescript
import { createBenchmarkEngine, createMockAdapter, generateDefaultCorpus } from "prompt-injection-bench";

const adapter = createMockAdapter(0.95, 0.03);
const corpus = generateDefaultCorpus();
const engine = createBenchmarkEngine({ defense: adapter, parallel: 10 });

const result = await engine.runBenchmark(corpus);
console.log(`Detection rate: ${(1 - result.attackSuccessRate) * 100}%`);
```

### MCP

Connect the MCP server to any MCP client:

```json
{
"mcpServers": {
"prompt-injection-bench": {
"command": "npx",
"args": ["@reaatech/pi-bench-mcp-server"]
}
}
}
```

## Packages

| Package | Description |
|---------|-------------|
| [`@reaatech/pi-bench-core`](./packages/core) | Canonical types, Zod schemas, and attack taxonomy |
| [`@reaatech/pi-bench-observability`](./packages/observability) | Structured logging, tracing, and metrics |
| [`@reaatech/pi-bench-corpus`](./packages/corpus) | Attack corpus builder, validator, and variant engine |
| [`@reaatech/pi-bench-adapters`](./packages/adapters) | Pluggable defense adapter implementations |
| [`@reaatech/pi-bench-scoring`](./packages/scoring) | Scoring engine and statistical analysis |
| [`@reaatech/pi-bench-runner`](./packages/runner) | Benchmark execution engine |
| [`@reaatech/pi-bench-leaderboard`](./packages/leaderboard) | Leaderboard management and persistence |
| [`@reaatech/pi-bench-mcp-server`](./packages/mcp-server) | MCP server and reproducibility tools |
| [`prompt-injection-bench`](./packages/prompt-injection-bench) | CLI and umbrella package |

## Attack Taxonomy

| Category | Weight | Description |
|----------|--------|-------------|
| Direct Injection | 1.0 | Obvious malicious instruction overrides |
| Prompt Leaking | 1.2 | Attempts to extract system prompts or config |
| Role-Playing | 1.3 | DAN, developer mode, persona adoption |
| Encoding Attacks | 1.1 | Base64, ROT13, Unicode obfuscation |
| Multi-Turn Jailbreaks | 1.4 | Gradual persuasion across conversation turns |
| Payload Splitting | 1.2 | Attack split across multiple inputs |
| Translation Attacks | 1.1 | Low-resource language injection |
| Context Stuffing | 1.3 | Hiding injection in large context |

## Scoring Methodology

```
weighted_score = Σ(category_score × category_weight) / Σ(weights)
final_score = weighted_score × (1 - fpr_penalty) × consistency_bonus
```

| Metric | Formula | Interpretation |
|--------|---------|----------------|
| Attack Success Rate (ASR) | `bypassed / total` | Lower is better (0% = perfect) |
| Defense Efficacy Score (DES) | `1 - ASR` | Higher is better (100% = perfect) |
| False Positive Rate (FPR) | `false_alarms / benign` | Lower is better (0% = perfect) |
| Latency Overhead | `defense_time - baseline` | Lower is better |

## Defense Adapters

| Adapter | Type | Configuration |
|---------|------|---------------|
| `mock` | Testing | No config needed — deterministic |
| `rebuff` | Library | `REBUFF_API_KEY` env var |
| `lakera` | API | `LAKERA_API_KEY` env var |
| `llm-guard` | Library | `LLM_GUARD_API_KEY` env var |
| `garak` | Tool | `garak` in PATH |
| `moderation-openai` | API | `OPENAI_API_KEY` env var |
| `moderation-azure` | API | `AZURE_*` env vars |
| `moderation-anthropic` | API | `ANTHROPIC_API_KEY` env var |
| `moderation-cohere` | API | `COHERE_API_KEY` env var |
| `custom` | HTTP | Custom `detectUrl`, `sanitizeUrl` |

## Security Features

- **Input validation** — 10K char limit, null byte rejection, API key checks before network calls
- **SSRF protection** — Blocks `file:`, `javascript:`, `data:` protocols and cloud metadata endpoints
- **Path traversal prevention** — Validates all file paths in MCP tool operations
- **Rate limiting** — Token bucket algorithm (100 req/min per adapter, 500 req/min global)

## Documentation

- [`ARCHITECTURE.md`](./ARCHITECTURE.md) — System design, package relationships, and data flows
- [`AGENTS.md`](./AGENTS.md) — Coding conventions and development guidelines
- [`CONTRIBUTING.md`](./CONTRIBUTING.md) — Contribution workflow and release process

## License

[MIT](LICENSE)