Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/duriantaco/ravage

AI pentesting research agent for controlled local labs, evidence gated exploit chains, memory-assisted learning, and benchmark eval.
https://github.com/duriantaco/ravage

ai-agent authorized-testing benchmarking dast pentesting python research security web-security

Last synced: 12 days ago
JSON representation

AI pentesting research agent for controlled local labs, evidence gated exploit chains, memory-assisted learning, and benchmark eval.

Awesome Lists containing this project

README 

          


  Ravage logo




# Ravage



Ravage is a source-available research workspace for autonomous web application

penetration-testing agents on controlled, authorized targets.



The current system includes:



- `ravage attack`: the model-driven `ai-web` loop with scoped tool execution,

  audit events, reports, resume support, and an optional live observer.

- `ravage scan`: deterministic DAST probes that reuse the same scope, audit,

  workspace, and report shape without spending model turns.

- `ravage lab`: deliberately vulnerable local lab boxes for reproducible

  testing.

- Tool-runtime preflight for host or Docker scanners such as `nmap`, `ffuf`,

  `katana`, `nuclei`, and `sqlmap`.

- Optional local Ravage Memory for reviewed, redacted, replay-backed lessons.

- Benchmark harnesses for local manifests and XBEN-style controlled runs.



## Responsible Use



Read [DISCLAIMER.md](DISCLAIMER.md) and [LICENSE](LICENSE) before running

anything. Ravage is provided for research under a restrictive research license;

it is not open-source software.



Only use Ravage against systems you own or have explicit written authorization

to test, inside the agreed rules of engagement. Do not run it against public

internet targets, production services, customer environments, employer systems,

or any machine outside your authorization.



The lab boxes in this repository intentionally contain serious vulnerabilities.

Run them only on localhost or an isolated private lab network. Do not expose

them to a public or shared network.



## Scope Enforcement



Ravage is localhost-first. Remote targets require `--allow-remote-target` and

must be explicitly listed in the engagement `scope.in_scope`. HTTP and browser

actions are checked against scope, redirects are rechecked before following, and

terminal tools are allowlisted to scoped network tools only.



The runtime records an `orchestrator/scope_firewall_plan_generated` audit event,

but it does not automatically install firewall rules on the host. Treat that

event as a firewall plan, not evidence that kernel-level egress filtering was

applied. The plan is intentionally IP/CIDR-only for network destinations, so

hostname entries are skipped rather than relying on DNS at rule-install time.

The Docker scope-enforcement integration test applies rules explicitly inside

its Kali container.



## Install From Source



Ravage is currently a source-checkout project. There is no published Ravage

PyPI release yet, and there are no tagged releases yet.



Use Python 3.12. The workspace is managed with `uv`:



```bash

uv sync --frozen

source .venv/bin/activate

ravage --help

ravage --version

```



For a fresh local wheel-style install without relying on the checkout venv,

install the shared schemas and runtime package together:



```bash

python3.12 -m venv .venv

.venv/bin/python -m pip install packages/schemas packages/ravage

.venv/bin/ravage --help

.venv/bin/ravage --version

```



External scanners are not Python dependencies. Check or install them where the

Ravage process runs:



```bash

ravage tools check

ravage tools install

ravage tools install --method docker --execute

```



`ravage tools install` is a dry run unless `--execute` is passed.



## Local Lab Quickstart



List the local boxes:



```bash

ravage lab list

```



- `ravage-acme-box`: Python support portal with login, invoices, catalog

  search, JWT admin access, SSRF, and 4 default flags.

- `ravage-forgeops-box`: Go release-operations portal with project BOLA,

  audit query injection, weak JWT, artifact traversal, diagnostics command

  injection, SSRF, extra non-flag findings, and 6 default flags.

- `ravage-node-market-box`: Node/Express market portal with order BOLA,

  catalog SQL injection, JWT admin confusion, prototype-pollution-style beta

  config abuse, internal admin pivot, and 5 default flags.

- `ravage-perimeter-box`: Python two-service perimeter lab with hidden edge

  backup/debug paths, a secondary ops port, default credentials, audit SQL

  injection, authenticated export traversal, and 5 default flags.



Start the Acme support portal:



```bash

ravage lab up ravage-acme-box

```



Run a deterministic scan:



```bash

ravage scan labs/ravage-acme-box/brief.yaml \

  --profile web-basic \

  --max-actions 40 \

  --observe

```



Run the model-driven agent with a hosted OpenAI-compatible profile:



```bash

export OPENAI_API_KEY=...



ravage attack labs/ravage-acme-box/brief.yaml \

  --observe \

  --model-profile hosted-openai \

  --model-tier low \

  --tool-runtime auto \

  --tool-image ravage-kali:latest \

  --memory off \

  --allow-paid-models

```



Stop the box:



```bash

ravage lab down ravage-acme-box

```



Every `ravage attack` run writes:



```text

runs/-/

  audit.db

  agent.stdout

  report.json

  workspace/

```



Every `ravage scan` run writes the same shape, with `scan.stdout` instead of

`agent.stdout`.



Audit rows are hash-chained as they are written. Verify an untouched run

directory or direct database path with:



```bash

ravage audit verify runs/-

ravage audit verify runs/-/audit.db

```



Resume a previous run by pointing at the run directory, workspace directory, or

`report.json`:



```bash

ravage attack labs/ravage-acme-box/brief.yaml \

  --resume-from runs/brief-YYYYMMDDHHMMSS/report.json

```



Attach the observer to an existing run:



```bash

ravage observe runs/brief-YYYYMMDDHHMMSS \

  --lab-manifest labs/ravage-acme-box/ravage-lab.yaml

```



To run the recon-heavy perimeter lab, substitute `ravage-perimeter-box`. Its

brief explicitly scopes both `http://127.0.0.1:8094` and

`http://127.0.0.1:8095`, and requires port scanning plus directory brute-force

capabilities:



```bash

ravage lab up ravage-perimeter-box

ravage attack labs/ravage-perimeter-box/brief.yaml \

  --observe \

  --observe-port 8790 \

  --model-profile hosted-openai \

  --model-tier low \

  --tool-runtime auto \

  --tool-image ravage-kali:latest \

  --tool-recon \

  --memory off \

  --allow-paid-models

```



The live web UI does not display credentials. Assisted-mode seed credentials

live in each lab's `OPERATOR_NOTES.md`; omit that file from the agent context

for stricter black-box testing. The dashboard masks captured flag values by

default but shows counts, proof status, agent steps, and the current

attack-chain stage.



`ai-web` tracks coverage as access state changes. A probe that is blocked by

authentication or privilege is not treated as permanently done; after a cookie,

header, browser login, default-credential success, or JWT privilege change, the

agent releases those blocked candidates for retry. Confirmed command injection

and local file inclusion also get tiny read-only flag follow-ups on localhost

capture-flag labs only. This is deliberately evidence-gated so the agent covers

more ground without learning lab-specific paths or treating memory as proof.

The same coverage ledger now records all paths attempted by bounded multi-path

probes, so failed variants do not burn later turns as if they were still fresh.

JSON account/config routes can also be checked with a bounded business-logic

probe for mass-assignment or merge-style privilege changes.



## Included Labs



| Lab | Default URL | Difficulty | Flags | Main coverage |

| --- | --- | --- | ---: | --- |

| `ravage-acme-box` | `http://127.0.0.1:8088` | medium | 4 | login, IDOR, SQLi, weak JWT, SSRF |

| `ravage-forgeops-box` | `http://127.0.0.1:8090` | hard | 6 | BOLA, query injection, JWT, traversal, command injection, SSRF |

| `ravage-node-market-box` | `http://127.0.0.1:8092` | medium | 5 | BOLA, SQLi, JWT confusion, config merge abuse, SSRF |

| `ravage-perimeter-box` | `http://127.0.0.1:8094` and `:8095` | hard | 5 | multi-port recon, hidden paths, default credentials, SQLi, traversal |



See [BENCHMARKS.md](BENCHMARKS.md) for scoring rules and latest local run

notes.



## Model Providers



`ai-web` uses OpenAI-compatible chat-completions routes. Built-in profiles cover

local Ollama, LM Studio, vLLM, LiteLLM, and hosted OpenAI-compatible APIs.



Inspect routes:



```bash

ravage --print-model-routes \

  --model-profile local-ollama \

  --model-tier mid

```



Use hosted models only with explicit spend acknowledgement:



```bash

ravage attack labs/ravage-acme-box/brief.yaml \

  --model-profile hosted-openai \

  --model-tier low \

  --allow-paid-models

```



See [docs/model-providers.md](docs/model-providers.md) for provider-specific

environment variables.



## Memory



Ravage Memory is local and optional. It stores redacted candidate lessons in

SQLite, retrieves reviewed memories as advisory hints, and only promotes

replay-backed lessons.



```bash

ravage memory review

ravage memory show 

ravage memory promote  --reason "replayed on controlled target" --replay-passed

ravage memory reject  --reason "overfit or noisy"

ravage memory export --redacted

ravage memory gc

```



Benchmarks default memory to `off` for reproducibility. Normal local/manual

runs default to `read`.



See [docs/memory.md](docs/memory.md).



## Benchmarks



Run the deterministic local SQLi benchmark:



```bash

ravage --benchmark eval/local_sqli_manifest.yaml \

  --output-dir runs/benchmarks/local-sqli

```



Run the `ai-web` benchmark with a local model route:



```bash

RAVAGE_OLLAMA_MODEL=qwen2.5-coder:32b \

OLLAMA_BASE_URL=http://localhost:11434/v1 \

ravage --benchmark eval/ai_web_manifest.yaml \

  --output-dir runs/benchmarks/ai-web-local

```



For hosted routes, preflight before spending:



```bash

OPENAI_API_KEY=... \

ravage --benchmark eval/ai_web_manifest.yaml \

  --output-dir runs/benchmarks/ai-web-hosted-preflight \

  --benchmark-model-profile hosted-openai \

  --benchmark-model-tier low \

  --benchmark-limit 1 \

  --benchmark-max-turns 4 \

  --benchmark-max-model-requests 4 \

  --benchmark-preflight

```



Fake-model tests only prove loop mechanics and scoring behavior. They are not

benchmark scores.



## Validation



Run the fast tests:



```bash

python -m pytest packages/schemas packages/ravage/tests tests

```



Run the scope-enforcement integration test when Docker is available:



```bash

python -m pytest -k test_scope_enforcement

```



## Docs



- [Responsible Use Disclaimer](DISCLAIMER.md)

- [License](LICENSE)

- [Docs Index](docs/README.md)

- [AI Web Operator Guide](docs/ai-web-operator-guide.md)

- [Model Providers](docs/model-providers.md)

- [Ravage Memory](docs/memory.md)

- [Benchmarking](docs/benchmarking.md)

- [Benchmarks And Local Test Boxes](BENCHMARKS.md)



Planning and strategy documents under `docs/` are useful project history, but

the README, operator guide, benchmark docs, and CLI help are the operational

source of truth.



## Packaging



Many established Python security tools do publish installable packages on PyPI.

Examples include `sqlmap`, `pwntools`, `impacket`, `mitmproxy`, `wapiti3`,

`wafw00f`, `dirsearch`, `wfuzz`, and `arjun`. Some still recommend git, Docker,

Kali packages, or OS package managers for particular workflows, and PyPI does

not install non-Python scanners such as `nmap`.



Ravage is now arranged so the publishable runtime package is `packages/ravage`,

with distribution name `ravage`, import package `ravage`, and console command

`ravage`. The repository root is still a development workspace named

`pentest-agent`; do not publish the workspace root. The shared schema models

are prepared as a small companion distribution, `ravage-schemas`, with the

existing Python import package `pentest_schemas`.



Before a PyPI release:



- reserve `ravage` and `ravage-schemas` on PyPI and TestPyPI;

- publish/install `ravage-schemas` before `ravage`;

- decide whether any other workspace members need separate public packages;

- keep vulnerable labs, benchmark fixtures, run artifacts, and large assets out

  of the wheel unless explicitly packaged as separate lab extras;

- keep external scanner installation in `ravage tools install`, not Python

  dependencies;

- test the final wheel on TestPyPI in a clean virtual environment.



The repository includes a GitHub Actions workflow at

`.github/workflows/publish-pypi.yml` for PyPI Trusted Publishing. Configure

pending PyPI publishers for both `ravage-schemas` and `ravage` with:



- owner: `duriantaco`

- repository: `ravage`

- workflow: `publish-pypi.yml`

- environment: `pypi`



Version bumps are automated by Release Please. When conventional commits land

on `main`, `.github/workflows/release-please.yml` opens or updates a release PR

that bumps both publishable package versions together, updates source fallback

versions, and refreshes `CHANGELOG.md`. Merging that release PR creates the

matching GitHub release tag, for example `v0.0.1`.



Configure `RELEASE_PLEASE_TOKEN` with a token or GitHub App credential if the

generated release should trigger the downstream PyPI publish workflow.



Publishing is triggered by publishing a GitHub release whose tag matches the

package version.



## Versioning



Ravage uses semantic versioning for publishable packages and tags. Release tags

should use `vMAJOR.MINOR.PATCH`, for example `v0.0.1`, and the package versions

in `packages/schemas/pyproject.toml` and `packages/ravage/pyproject.toml`

should match the tag without the leading `v`.



Conventional commit titles drive the automated bump: `fix(...)` maps to patch,

`feat(...)` maps to minor, and breaking-change markers map to major.



While Ravage is pre-1.0, minor releases may include breaking changes as the

agent APIs, report shape, and packaging surface settle. Patch releases should be

limited to compatible bug fixes, docs corrections, and packaging fixes.