https://github.com/reforcemind/netforge_rl
Multi-Agent Reinforcement Learning (MARL) cybersecurity simulator
https://github.com/reforcemind/netforge_rl
cybersecurity multi-agent-systems pettingzoo python reinforcement-learning reinforcement-learning-environments
Last synced: 16 days ago
JSON representation
Multi-Agent Reinforcement Learning (MARL) cybersecurity simulator
- Host: GitHub
- URL: https://github.com/reforcemind/netforge_rl
- Owner: reforcemind
- License: other
- Created: 2026-03-05T06:57:52.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2026-07-07T19:55:49.000Z (19 days ago)
- Last Synced: 2026-07-07T21:13:11.910Z (19 days ago)
- Topics: cybersecurity, multi-agent-systems, pettingzoo, python, reinforcement-learning, reinforcement-learning-environments
- Language: Python
- Homepage: https://reforcemind.github.io/NetForge_RL/
- Size: 2.73 MB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Citation: CITATION.cff
Awesome Lists containing this project
README
NetForge RL
Multi-agent cybersecurity environment for RL research. Red team vs Blue team on generated networks.
---
## Why NetForge
- **Standardized API** — PettingZoo `ParallelEnv`, Gymnasium spaces, conformance-tested.
A `gymnasium.Env` single-agent facade (`NetForgeSingleAgentEnv`) drops straight into
Stable-Baselines3 or CleanRL.
- **Partial observability that means something** — Blue reads a filtered, optionally
*delayed* SIEM feed; Red must recon a host before it can exploit it.
- **Real telemetry** — actions emit real Windows/Sysmon event XML, encoded into
observations by an NLP pipeline. Point an OpenAI, Anthropic, Google, or vLLM client at
the raw logs and let it play SOC analyst.
- **MITRE ATT&CK aligned** — exploits map to real CVEs (MS17-010, CVE-2019-0708,
Log4Shell); every episode reports which ATT&CK techniques Red actually exercised.
- **Reproducible to the bit** — a fixed seed replays observations, SIEM embeddings, infos,
and rewards identically. Test-guaranteed, not just claimed.
- **Tunable difficulty** — named `easy` / `medium` / `hard` presets plus a frozen held-out
evaluation split, so results are comparable across runs.
- **Diagnostic probes + capability cards** — 6 targeted probes (memory, attention,
temporal, precision, safety, generalization) score *what* a policy can and can't do,
summarized as a per-policy radar chart.
- **Deception as a mechanic** — decoys and honeytokens with a `deception_efficacy` metric
quantifying how much of Red's effort was wasted on traps.
- **Graph-native observations** — hosts as nodes, reachability as edges, fog-of-war aware,
one call from PyTorch Geometric / jraph.
- **Self-play & Elo** — a population tournament rates every red and blue policy on one
shared ladder, AlphaStar-league style.
- **Fast** — the JAX backend measures **270,795 env-steps/s (1,083,181 agent-steps/s)** at
batch 4096 on CPU, with an in-kernel SIEM signal so the vectorized path carries
telemetry end-to-end.
- **Trained, not just scripted, baselines** — a JIT-fused IPPO trainer whose entire
rollout runs on-device; a committed run learns mean reward 0.06 → 0.71 on `ransomware`.
## Install
```bash
pip install 'netforge_rl[jax] @ git+https://github.com/reforcemind/NetForge_RL'
```
Extras: `jax` (vectorized backend), `render` (visualization), `finetune` (LLM PEFT),
`rllib` (Ray multi-agent training). See the
[install guide](https://reforcemind.github.io/NetForge_RL/quickstart/) for the full list.
## Quick start
```python
from netforge_rl.environment import make_env
env = make_env('medium', scenario_type='ransomware', seed=0)
obs, infos = env.reset(seed=0)
while env.agents:
actions = {a: env.action_space(a).sample() for a in env.agents}
obs, rewards, term, trunc, infos = env.step(actions)
if all(term.values()) or all(trunc.values()):
break
```
### JAX vectorized (4096 environments)
```python
import jax
from netforge_rl.backends.jax import VectorEnvSpec
from netforge_rl.bridges.jaxmarl import JaxMARLEnv, random_action_dict
env = JaxMARLEnv(spec=VectorEnvSpec(n_hosts=100, n_red=1, n_blue=3), batch_size=4096)
key = jax.random.PRNGKey(0)
obs, state = env.reset(key)
obs, state, reward, done, info = env.step(key, state, random_action_dict(env, key))
```
### LLM as a Blue agent
```python
from netforge_rl.semantic import state_to_text, parse_action
prompt = state_to_text(env.to_envstate(), agent_id='blue_dmz')
# send prompt to your model, then:
action_idx = parse_action(model_reply, 'blue_dmz', sorted(env.global_state.all_hosts))
```
See the [Quick Start Guide](https://reforcemind.github.io/NetForge_RL/quickstart/) for
difficulty presets, baselines, diagnostics, and the Gymnasium single-agent wrapper.
## The five scenarios
| Scenario | Red objective | Blue objective | Terminal condition |
|---|---|---|---|
| `ransomware` | Compromise Corporate + Secure | Contain, restore, avoid downtime | all Corporate/Secure compromised, or PLC kinetic |
| `apt_espionage` | Stealthy persistence + exfiltration | Detect and isolate every foothold | every infected host isolated |
| `cloud_hybrid` | Breach the Secure enclave | Protect the Secure subnet SLA | every Secure host compromised |
| `iot_grid` | Take the grid controllers | Keep controllers healthy | all controllers compromised |
| `ot_stuxnet` | Drive a PLC to kinetic destruction | Prevent physical damage | any PLC kinetic destruction |
## Package layout
| Package | Purpose |
|---|---|
| `environment/` | PettingZoo `NetForgeRLEnv`, difficulty presets, curriculum, Gymnasium facade |
| `scenarios/` | 5 reward/objective families sharing one `BaseScenario` |
| `actions/red`, `actions/blue` | ATT&CK-aligned capabilities + technique mapping |
| `siem/`, `nlp/` | Sysmon/Windows log synthesis, NLP encoding, OCSF-style export |
| `backends/jax/` | Vectorized `vmap`/`jit` transition kernel + NumPy reference |
| `baselines/` | Random, heuristic, kill-chain red, JAX IPPO |
| `diagnostics/` | Capability probes, oracle information-asymmetry, capability cards |
| `bridges/` | RLlib, JaxMARL, CleanRL, DLPack adapters |
| `semantic/` | LLM SOC agents, prompt grammars, fine-tuning recipes |
## Citation
```bibtex
@misc{jankowski2026eventdriventemporalgraphnetworks,
title={Event-Driven Temporal Graph Networks for Asynchronous Multi-Agent Cyber Defense in NetForge_RL},
author={Igor Jankowski},
year={2026},
eprint={2604.09523},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2604.09523},
}
```
## License
MIT — see [LICENSE](LICENSE).