{"id":51570550,"url":"https://github.com/reforcemind/ct-gmarl","last_synced_at":"2026-07-10T19:02:41.211Z","repository":{"id":351028715,"uuid":"1201161722","full_name":"reforcemind/ct-gmarl","owner":"reforcemind","description":"Event-Driven Continuous-Time Graph MARL for Asynchronous Cyber Defense in NetForge_RL.  A  framework for training autonomous cyber defenders in a continuous-time POSMDP environment. Features Neural ODE temporal dynamics, GAT spatial reasoning, and a Sim2Real evaluation bridge against live CVE payloads.","archived":false,"fork":false,"pushed_at":"2026-04-14T07:27:49.000Z","size":9976,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-30T19:21:56.064Z","etag":null,"topics":["asynchronous-learning","graph-neural-networks","marl","multi-agent-systems","network-security","neural-ode","pytorch","reinforcement-learning","sim2real"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reforcemind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-04T09:41:13.000Z","updated_at":"2026-04-29T03:42:05.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/reforcemind/ct-gmarl","commit_stats":null,"previous_names":["xaiqo/ct-gmarl","reforcemind/ct-gmarl"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/reforcemind/ct-gmarl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2Fct-gmarl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2Fct-gmarl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2Fct-gmarl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2Fct-gmarl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reforcemind","download_url":"https://codeload.github.com/reforcemind/ct-gmarl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reforcemind%2Fct-gmarl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35339931,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-10T02:00:06.465Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asynchronous-learning","graph-neural-networks","marl","multi-agent-systems","network-security","neural-ode","pytorch","reinforcement-learning","sim2real"],"created_at":"2026-07-10T19:02:40.591Z","updated_at":"2026-07-10T19:02:41.199Z","avatar_url":"https://github.com/reforcemind.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Event-Driven Temporal Graph Networks for Asynchronous Multi-Agent Cyber Defense\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![arXiv](https://img.shields.io/badge/arXiv-2604.09523-b31b1b.svg)](https://arxiv.org/abs/2604.09523)\n[![Gist Science](https://img.shields.io/badge/Read-Gist_Science-1e90ff.svg)](https://gist.science/paper/2604.09523#gist)\n\nA MARL framework for bridging the Sim2Real gap in autonomous cyber defense using **NetForge_RL** (an asynchronous POSMDP simulator) and **CT-GMARL** (a Continuous-Time ODE-based Graph architecture).\n\n---\n\n## Research Overview\n\nThe transition of Multi-Agent Reinforcement Learning (MARL) policies to operational Security Operations Centers (SOCs) is bottlenecked by the **Sim2Real gap**. Legacy simulators abstract network physics into synchronous discrete turns, failing to capture the event-driven nature of real-world telemetry. \n\n### Key Contributions:\n1.  **NetForge_RL**: Transitions the domain from discrete POMDPs to continuous-time **POSMDPs** ($F(t|s,a)$).\n2.  **CT-GMARL**: A spatial-temporal architecture that marries **Multi-Head Graph Attention (GAT)** for dynamic topology reasoning with **Neural ODEs** for continuous-time hidden state drift.\n3.  **Sim2Real Bridge**: A dual-hypervisor engine allowing training in a high-throughput `MockHypervisor` ($\\approx 10,000$ SPS) and Zero-Shot evaluation against live exploits in `DockerHypervisor` using real Vulhub containers.\n4.  **Operational Fidelity**: Avoids \"scorched earth\" failure modes through precise reward shaping and NLP-encoded SIEM telemetry (TF-IDF/Transformer).\n\n---\n\n## Dual-Engine Simulation Flow\n\nNetForge_RL allows models to learn fundamental cyber physics at high speeds in a simulated environment before strictly verifying performance retention against genuine, live-fire vulnerabilities.\n\n```mermaid\ngraph TD\n    subgraph Execution Pipeline\n        A[Hydra Joblib Launcher] --\u003e|4 Parallel Workers| B(Batch Phase 1: Core Training)\n        B --\u003e B1[CT-GMARL Architecture]\n        B --\u003e B2[R-MAPPO / QMIX Baseline]\n    end\n\n    subgraph The NetForge_RL Bridge\n        B1 --\u003e D{Sim2Real Dispatcher}\n        B2 --\u003e D\n        \n        D --\u003e|High-Throughput| E[MockHypervisor - Training]\n        D --\u003e|Zero-Shot Transfer| F[DockerHypervisor - Verification]\n        \n        E --\u003e|Simulated Physics| G[(Asynchronous Event Queue)]\n        F --\u003e|Subprocess API| H[(Live Vulhub Containers)]\n    end\n\n    subgraph Model Output\n        G --\u003e I[Evaluation Metrics]\n        H --\u003e J[Operational Verification]\n    end\n```\n\n---\n\n## CT-GMARL Architecture\n\nCT-GMARL decouples the spatial constraints of Zero-Trust topology from the temporal irregularities of event-driven SIEM telemetry.\n\n```mermaid\ngraph LR\n    subgraph Spatial Layer\n        Obs[Local Telemetry 128-dim] --\u003e GAT[Multi-Head GAT]\n        Mask[ZTNA Topology Mask] -.-\u003e GAT\n    end\n\n    subgraph Integration Layer\n        GAT --\u003e|x_k+1| ODE[Neural ODE Cell]\n        DT[Normalized dt_k] --\u003e ODE\n        ODE --\u003e|Hidden Drift| Drift[h_t+1 = h_t + ∫f_θ dt]\n    end\n\n    subgraph Policy Layer\n        Drift --\u003e Actor[PPO Actor]\n        Drift --\u003e Critic[Centralized Critic]\n        Actor --\u003e Act[MultiDisc Action Space]\n    end\n\n    style ODE fill:#f96,stroke:#333,stroke-width:2px\n    style GAT fill:#1f7,stroke:#333,stroke-width:2px\n```\n\n---\n\n## Project Structure\n\n```bash\nct_gmarl/\n├── conf/                    # Hydra Experiment Configurations\n│   ├── algorithm/           # RL Algorithm hyperparams (PPO, QMIX)\n│   ├── env/                 # NetForge_RL Environment settings (100-node)\n│   ├── model/               # CT-GMARL architecture specific configs\n│   └── research/            # Pre-configured experimental matrix runs\n├── src/\n│   ├── engine/              # NetForge_RL POSMDP Engine\n│   │   ├── hypervisor/      # Mock vs. Docker Hypervisor logic\n│   │   ├── suite.py         # ForgeSuite orchestration\n│   │   └── telemetry.py     # SIEM NLP Log Encoding pipeline\n│   ├── models/              # Neural Architecture Implementations\n│   │   ├── ct_gmarl/        # Neural ODE + GAT Processor logic\n│   │   └── baselines/       # R-MAPPO and QMIX implementations\n│   └── utils/               # Metric export, logging, and seed management\n├── diagrams/                # Exported training plots and action heatmaps\n│   ├── ctg/                 # CT-GMARL specific heatmaps\n│   └── rmappo/              # R-MAPPO baseline heatmaps\n├── train.py                 # Primary entry point for training and evaluation\n└── pyproject.toml           # Dependency and packaging management\n```\n\n---\n\n## Quick Start\n\n### 1. Installation\nThis project requires Python 3.12 and a running Docker daemon for Sim2Real evaluation.\n\n```bash\ngit clone https://github.com/YourUsername/ct_gmarl.git\ncd ct_gmarl\npip install . \n# Optional: install nlp transformers for BERT encoding\npip install \".[nlp]\"\n```\n\n### 2. Running Experiments\nWe use **Hydra** for experiment configuration. You can launch the core training matrix directly from `train.py`.\n\n```bash\n# Launch CT-GMARL training on 100-node topology\npython train.py algorithm=ct_gmarl env.node_count=100\n\n# Launch competitive matrix (R-MAPPO vs CT-GMARL)\npython train.py algorithm=\"[rmappo, ct_gmarl]\" env.mode=sim\n\n# Zero-Shot Sim2Real Evaluation (Requires Docker)\npython train.py algorithm=ct_gmarl env.mode=real checkpoint=path/to/model\n```\n\n---\n\n## Zero-Shot Sim2Real Transfer\n\nWe define **Zero-Shot Sim2Real Transfer** as evaluating the converged policy directly on live container payloads (DockerHypervisor) without any gradient updates or fine-tuning in the target environment. This verifies the agent's capacity to generalize topological and temporal defense logic trained in the MockHypervisor to genuine CVEs.\n\n---\n\n## Citation\n\nIf you use **NetForge_RL** or the **CT-GMARL** architecture in your research, please cite our work:\n\n```bibtex\n@misc{jankowski2026eventdriventemporalgraphnetworks,\n      title={Event-Driven Temporal Graph Networks for Asynchronous Multi-Agent Cyber Defense in NetForge_RL}, \n      author={Igor Jankowski},\n      year={2026},\n      eprint={2604.09523},\n      archivePrefix={arXiv},\n      primaryClass={cs.LG},\n      url={https://arxiv.org/abs/2604.09523}, \n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freforcemind%2Fct-gmarl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freforcemind%2Fct-gmarl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freforcemind%2Fct-gmarl/lists"}