https://github.com/mldangelo/redteam-chatbot
https://github.com/mldangelo/redteam-chatbot
Last synced: 12 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/mldangelo/redteam-chatbot
- Owner: mldangelo
- Created: 2025-05-29T22:26:14.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-05-29T23:06:58.000Z (about 1 year ago)
- Last Synced: 2025-06-16T23:53:16.698Z (about 1 year ago)
- Language: JavaScript
- Size: 98.6 KB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# redteam-chatbot (Red teaming a Multi-turn Chatbot)
You can run this example with:
```bash
npx promptfoo@latest init --example redteam-chatbot
```
## Introduction
This example demonstrates how to test a stateless chatbot for security vulnerabilities using promptfoo's multi-turn strategies. It includes a Node.js Express server that accepts a conversation history in OpenAI format and returns a response in the same format. It leverages promptfoo's [goat](https://www.promptfoo.dev/blog/jailbreaking-with-goat/) and crescendo strategies for multi-turn red teaming. You can learn more about configuring these strategies [here](https://www.promptfoo.dev/docs/red-team/strategies/multi-turn/).
## Setup
### Installation
1. Install dependencies:
```bash
npm install
```
2. Set your OpenAI API key:
```bash
export OPENAI_API_KEY=your-api-key-here
```
3. Start the server:
```bash
npm start
```
## Running Tests
```bash
# Generate test cases
promptfoo redteam generate
# Execute evaluation
promptfoo redteam eval
# View results
promptfoo view
```
## Node.js Webserver Example Usage
### Single Message Request
```bash
curl -X POST http://localhost:2345/chat \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your-token-here" \
-d '{
"api_provider": "openai",
"chat_history": [
{"role": "user", "content": "Tell me about your turboencabulator models"}
]
}'
```
### Multi-turn Conversation
```bash
curl -X POST http://localhost:2345/chat \
-H "Content-Type: application/json" \
-H "Authorization: Bearer your-token-here" \
-d '{
"api_provider": "openai",
"chat_history": [
{"role": "user", "content": "Tell me about your turboencabulator models"},
{"role": "assistant", "content": "TurboTech offers several turboencabulator models..."},
{"role": "user", "content": "What maintenance does it require?"}
]
}'
```