{"id":51569806,"url":"https://github.com/reforcemind/netforge_rl","last_synced_at":"2026-07-10T19:00:34.920Z","repository":{"id":342326782,"uuid":"1173253784","full_name":"reforcemind/NetForge_RL","owner":"reforcemind","description":"Multi-Agent Reinforcement Learning (MARL) cybersecurity simulator","archived":false,"fork":false,"pushed_at":"2026-07-07T19:55:49.000Z","size":2861,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-07T21:13:11.910Z","etag":null,"topics":["cybersecurity","multi-agent-systems","pettingzoo","python","reinforcement-learning","reinforcement-learning-environments"],"latest_commit_sha":null,"homepage":"https://reforcemind.github.io/NetForge_RL/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reforcemind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-05T06:57:52.000Z","updated_at":"2026-07-07T19:55:52.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/reforcemind/NetForge_RL","commit_stats":null,"previous_names":["elprofesoriqo/marl_cyborg","xaiqo/netforge_rl","reforcemind/netforge_rl"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/reforcemind/NetForge_RL","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2FNetForge_RL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2FNetForge_RL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2FNetForge_RL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2FNetForge_RL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reforcemind","download_url":"https://codeload.github.com/reforcemind/NetForge_RL/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2FNetForge_RL/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35339931,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-10T02:00:06.465Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cybersecurity","multi-agent-systems","pettingzoo","python","reinforcement-learning","reinforcement-learning-environments"],"created_at":"2026-07-10T19:00:21.533Z","updated_at":"2026-07-10T19:00:34.899Z","avatar_url":"https://github.com/reforcemind.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/version-3.1.0-blue?style=for-the-badge\" alt=\"Version 3.1.0\"/\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Python-3.12+-blue?style=for-the-badge\u0026logo=python\" alt=\"Python 3.12+\"/\u003e\n  \u003cimg src=\"https://img.shields.io/badge/JAX-vmap%20%2B%20jit-orange?style=for-the-badge\" alt=\"JAX\"/\u003e\n  \u003cimg src=\"https://img.shields.io/badge/PettingZoo-MARL-purple?style=for-the-badge\" alt=\"PettingZoo\"/\u003e\n  \u003cimg src=\"https://img.shields.io/badge/MITRE_ATT%26CK-Mapped-red?style=for-the-badge\" alt=\"MITRE ATT\u0026CK\"/\u003e\n\u003c/div\u003e\n\n\u003ch1 align=\"center\"\u003eNetForge RL\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003eMulti-agent cybersecurity environment for RL research. Red team vs Blue team on generated networks.\u003c/p\u003e\n\n---\n\n## Why NetForge\n\n- **Standardized API** — PettingZoo `ParallelEnv`, Gymnasium spaces, conformance-tested.\n  A `gymnasium.Env` single-agent facade (`NetForgeSingleAgentEnv`) drops straight into\n  Stable-Baselines3 or CleanRL.\n- **Partial observability that means something** — Blue reads a filtered, optionally\n  *delayed* SIEM feed; Red must recon a host before it can exploit it.\n- **Real telemetry** — actions emit real Windows/Sysmon event XML, encoded into\n  observations by an NLP pipeline. Point an OpenAI, Anthropic, Google, or vLLM client at\n  the raw logs and let it play SOC analyst.\n- **MITRE ATT\u0026CK aligned** — exploits map to real CVEs (MS17-010, CVE-2019-0708,\n  Log4Shell); every episode reports which ATT\u0026CK techniques Red actually exercised.\n- **Reproducible to the bit** — a fixed seed replays observations, SIEM embeddings, infos,\n  and rewards identically. Test-guaranteed, not just claimed.\n- **Tunable difficulty** — named `easy` / `medium` / `hard` presets plus a frozen held-out\n  evaluation split, so results are comparable across runs.\n- **Diagnostic probes + capability cards** — 6 targeted probes (memory, attention,\n  temporal, precision, safety, generalization) score *what* a policy can and can't do,\n  summarized as a per-policy radar chart.\n- **Deception as a mechanic** — decoys and honeytokens with a `deception_efficacy` metric\n  quantifying how much of Red's effort was wasted on traps.\n- **Graph-native observations** — hosts as nodes, reachability as edges, fog-of-war aware,\n  one call from PyTorch Geometric / jraph.\n- **Self-play \u0026 Elo** — a population tournament rates every red and blue policy on one\n  shared ladder, AlphaStar-league style.\n- **Fast** — the JAX backend measures **270,795 env-steps/s (1,083,181 agent-steps/s)** at\n  batch 4096 on CPU, with an in-kernel SIEM signal so the vectorized path carries\n  telemetry end-to-end.\n- **Trained, not just scripted, baselines** — a JIT-fused IPPO trainer whose entire\n  rollout runs on-device; a committed run learns mean reward 0.06 → 0.71 on `ransomware`.\n\n## Install\n\n```bash\npip install 'netforge_rl[jax] @ git+https://github.com/reforcemind/NetForge_RL'\n```\n\nExtras: `jax` (vectorized backend), `render` (visualization), `finetune` (LLM PEFT),\n`rllib` (Ray multi-agent training). See the\n[install guide](https://reforcemind.github.io/NetForge_RL/quickstart/) for the full list.\n\n## Quick start\n\n```python\nfrom netforge_rl.environment import make_env\n\nenv = make_env('medium', scenario_type='ransomware', seed=0)\nobs, infos = env.reset(seed=0)\nwhile env.agents:\n    actions = {a: env.action_space(a).sample() for a in env.agents}\n    obs, rewards, term, trunc, infos = env.step(actions)\n    if all(term.values()) or all(trunc.values()):\n        break\n```\n\n### JAX vectorized (4096 environments)\n\n```python\nimport jax\nfrom netforge_rl.backends.jax import VectorEnvSpec\nfrom netforge_rl.bridges.jaxmarl import JaxMARLEnv, random_action_dict\n\nenv = JaxMARLEnv(spec=VectorEnvSpec(n_hosts=100, n_red=1, n_blue=3), batch_size=4096)\nkey = jax.random.PRNGKey(0)\nobs, state = env.reset(key)\nobs, state, reward, done, info = env.step(key, state, random_action_dict(env, key))\n```\n\n### LLM as a Blue agent\n\n```python\nfrom netforge_rl.semantic import state_to_text, parse_action\n\nprompt = state_to_text(env.to_envstate(), agent_id='blue_dmz')\n# send prompt to your model, then:\naction_idx = parse_action(model_reply, 'blue_dmz', sorted(env.global_state.all_hosts))\n```\n\nSee the [Quick Start Guide](https://reforcemind.github.io/NetForge_RL/quickstart/) for\ndifficulty presets, baselines, diagnostics, and the Gymnasium single-agent wrapper.\n\n## The five scenarios\n\n| Scenario | Red objective | Blue objective | Terminal condition |\n|---|---|---|---|\n| `ransomware` | Compromise Corporate + Secure | Contain, restore, avoid downtime | all Corporate/Secure compromised, or PLC kinetic |\n| `apt_espionage` | Stealthy persistence + exfiltration | Detect and isolate every foothold | every infected host isolated |\n| `cloud_hybrid` | Breach the Secure enclave | Protect the Secure subnet SLA | every Secure host compromised |\n| `iot_grid` | Take the grid controllers | Keep controllers healthy | all controllers compromised |\n| `ot_stuxnet` | Drive a PLC to kinetic destruction | Prevent physical damage | any PLC kinetic destruction |\n\n## Package layout\n\n| Package | Purpose |\n|---|---|\n| `environment/` | PettingZoo `NetForgeRLEnv`, difficulty presets, curriculum, Gymnasium facade |\n| `scenarios/` | 5 reward/objective families sharing one `BaseScenario` |\n| `actions/red`, `actions/blue` | ATT\u0026CK-aligned capabilities + technique mapping |\n| `siem/`, `nlp/` | Sysmon/Windows log synthesis, NLP encoding, OCSF-style export |\n| `backends/jax/` | Vectorized `vmap`/`jit` transition kernel + NumPy reference |\n| `baselines/` | Random, heuristic, kill-chain red, JAX IPPO |\n| `diagnostics/` | Capability probes, oracle information-asymmetry, capability cards |\n| `bridges/` | RLlib, JaxMARL, CleanRL, DLPack adapters |\n| `semantic/` | LLM SOC agents, prompt grammars, fine-tuning recipes |\n\n## Citation\n\n```bibtex\n@misc{jankowski2026eventdriventemporalgraphnetworks,\n      title={Event-Driven Temporal Graph Networks for Asynchronous Multi-Agent Cyber Defense in NetForge_RL}, \n      author={Igor Jankowski},\n      year={2026},\n      eprint={2604.09523},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG},\n      url={https://arxiv.org/abs/2604.09523}, \n}\n```\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freforcemind%2Fnetforge_rl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freforcemind%2Fnetforge_rl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freforcemind%2Fnetforge_rl/lists"}