{"id":15415463,"url":"https://github.com/abstractionslab/idps-escape","last_synced_at":"2026-05-07T02:38:47.369Z","repository":{"id":255869355,"uuid":"849757816","full_name":"AbstractionsLab/idps-escape","owner":"AbstractionsLab","description":"IDPS-ESCAPE (Intrusion Detection and Prevention Systems for Evading Supply Chain Attacks and Post-compromise Effects), part of the CyFORT project: open-source SOAR system powered by a dedicated ML-based anomaly detection toolbox (ADBox) integrated with open-source software such as Wazuh and Suricata.","archived":false,"fork":false,"pushed_at":"2024-09-07T14:39:04.000Z","size":58625,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-06T02:06:16.827Z","etag":null,"topics":["anomaly-detection","artificial-intelligence","correlation","data-ingestion","docker","graph-attention-network","idps","intrusion-detection","machine-learning","mtad-gat","multivariate-timeseries","opensearch","pandas","python3","pytorch","siem","soar","suricata","wazuh"],"latest_commit_sha":null,"homepage":"https://abstractionslab.com/index.php/research-and-development/cyfort/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AbstractionsLab.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-30T07:29:18.000Z","updated_at":"2024-09-08T14:25:27.000Z","dependencies_parsed_at":"2024-09-07T16:19:10.258Z","dependency_job_id":null,"html_url":"https://github.com/AbstractionsLab/idps-escape","commit_stats":null,"previous_names":["abstractionslab/idps-escape"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbstractionsLab%2Fidps-escape","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbstractionsLab%2Fidps-escape/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbstractionsLab%2Fidps-escape/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AbstractionsLab%2Fidps-escape/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AbstractionsLab","download_url":"https://codeload.github.com/AbstractionsLab/idps-escape/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246547371,"owners_count":20794970,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anomaly-detection","artificial-intelligence","correlation","data-ingestion","docker","graph-attention-network","idps","intrusion-detection","machine-learning","mtad-gat","multivariate-timeseries","opensearch","pandas","python3","pytorch","siem","soar","suricata","wazuh"],"created_at":"2024-10-01T17:08:27.987Z","updated_at":"2026-05-07T02:38:47.338Z","avatar_url":"https://github.com/AbstractionsLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IDPS-ESCAPE\n\nIDPS-ESCAPE, short for Intrusion Detection and Prevention Systems for Evading Supply Chain Attacks and Post-compromise Effects, is a sub-project of the [CyFORT](https://abstractionslab.com/index.php/research-and-development/cyfort/) project, which in turn stands for Cloud Cybersecurity Fortress of Open Resources and Tools for Resilience. CyFORT is carried out in the context of the [IPCEI-CIS](https://ec.europa.eu/commission/presscorner/detail/en/ip_23_6246) project, with further details available [here](https://www.bmwk.de/Redaktion/EN/Artikel/Industry/ipcei-cis.html).\n\n\u003cimg src=\"./docs/manual/_figures/CyFORT-logo.png\" alt=\"cyfort_logo\" width=\"400\"/\u003e\n\nIDPS-ESCAPE is aimed at closely capturing the notion of MAPE-K (Monitor, Analyze, Plan, Execute and Knowledge) from autonomic computing applied to cybersecurity, which translates into providing a comprehensive package fulfilling the roles of a Security Orchestration, Automation, and Response (SOAR) system, a Security Information and Event Management (SIEM), and an Intrusion Detection and Prevention System (IDPS), with a central subsystem dealing with anomaly detection (AD) based on state-of-the-art advances in machine learning (ML). We call this AD subsystem \"**ADBox**\", which comes with out-of-the-box integration with well-known open-source solutions such as [OpenSearch](https://opensearch.org/) for search and analytics, [Wazuh](https://wazuh.com/) as our SIEM\\\u0026XDR of choice, in turn connected to [MISP](https://www.misp-project.org/) for enriching alerts, and to [Suricata](https://suricata.io/), acting both as our network-based IDPS of choice, as well as a network-level data acquisition source.\n\nOur extensible **ADBox** framework and implementation also include a Multivariate Time-series Anomaly Detection (MTAD) algorithm relying on Graph Attention Networks (GAT).\n\nThis repository contains the source code and full documentation (requirements, technical specifications, schematics, user manual, test case specifications and test reports) of IDPS-ESCAPE, based on the [C5-DEC](https://github.com/AbstractionsLab/c5dec) method and software also developed in CyFORT, which relies on storing, interlinking and processing all software development life cycle (SDLC) artifacts in a unified manner; see our [traceability web page](https://abstractionslab.github.io/idps-escape/docs/traceability/index.html) providing the technical specifications of IDPS-ESCAPE.\n\n## Table of contents\n\n- [Overview](#overview)\n- [Features](#features)\n- [User manual](#user-manual)\n- [Technical specifications](#documentation-and-technical-specifications)\n- [Getting started](#getting-started)\n- [Usage](#usage)\n- [Use case scenario example](#example-of-a-use-case-scenario)\n- [Disclaimer](#disclaimer-use-of-alphaexperimental-software)\n- [Roadmap](#roadmap)\n- [License](#license)\n- [Contact](#contact)\n\n## Overview\n\nIDPS-ESCAPE, part of the CyFORT suite of open-source cybersecurity software solutions, addresses various aspects of cybersecurity as an ensemble, targeting different user groups, ranging from public to private and from CIRT/CSIRT to system administrators.\nThe design of IDPS-ESCAPE is targeted to cloud-native deployments, with an eye on CERT/CSIRT-operated monitoring systems.\n\nMoreover, IDPS-ESCAPE is being developed in parallel with another CyFORT sub-project, namely SATRAP-DL, aimed at enhancing cyber threat intelligence (CTI) analysts' work using semi-automated reasoning over CTI. Ultimately, IDPS-ESCAPE is planned to include, among other things, mechanisms for coping with and addressing supply chain and adversarial machine learning attacks.\n \nCurrently, for the alpha release, the main bulk of this repository is dedicated to a novel open-source and extensively documented anomaly detection (AD) framework, called **ADBox**.\nThe ADBox implementation provides a modular and extensible software framework for efficiently integrating ML and AD algorithms and it already comes with a Multivariate Time-series Anomaly Detection (MTAD) algorithm relying on Graph Attention Networks (GAT). \n\nIn addition to providing security practitioners such as SOC operators or CTI analysts with anomaly detection over Wazuh indices (alerts, archives, statistics, etc.) in multiple modes (batch, real-time and historical), it can also be used to simplify and refine the work of security practitioners across several dimensions, e.g.,\n\n- rule management, \n- events correlation, \n- alert-to-incident derivation, and, \n- alert/response policy tuning and mappings to KBs such as MITRE ATT\u0026CK. \n\nADBox can also be used as a software library to deploy various ML based AD algorithms in different environments, while allowing for a high degree of tailoring thanks to its modular and extensible design. An environment-driven customization can not only contribute to reducing false positives, but it can also help detect suspicious behavior with arguably limited information, or to otherwise provide an investigation entry point dealing with adversarial patterns for which prior signatures or indicators of compromise may not be readily available.\n\nAs a consequence, ADBox provides a stepping stone towards settling various controversial statements and at times questionable findings and claims from the academic literature and those made by practitioners in the industry: plug the latest implementation of an ML-based AD algorithm into ADBox, integrate with real-world security tools such as Wazuh, to assess and (in)validate such claims.\n\nThe current version of the IDPS-ESCAPE stack consists of \n- a combined setup integrating state-of-the-art open source _signature-based network and host_ IDPS and SIEM\\\u0026XDR, along with\n- ADBox, a custom-designed and implemented _anomaly detection_ subsystem based on machine learning.\n\nAlthough the IDPS, SIEM and ADBox subsystems can be deployed independently, we recommend a fully integrated deployment. We provide automation scripts and guides for an easy deployment of such setups.\n\n## Features\n\n### Design\n\n- Free/libre and open source;\n- Cross platform: works on GNU/Linux, MacOS and Windows;\n- Extensible due to a modular design and architecture;\n- Based on open data formats such as Markdown, YAML, XML, JSON, CSV and HTML;\n- Integrated with well-known open-source security solutions such as [OpenSearch](https://opensearch.org/), [Wazuh](https://wazuh.com/), [Suricata](https://suricata.io/);\n- Thanks to building on top of Wazuh, an easy integration with other well-known [third-party solutions](https://documentation.wazuh.com/current/getting-started/use-cases/threat-hunting.html) such as [MISP](https://www.misp-project.org/) using existing mechanisms.\n\n### ADBox\n\nADBox is a custom-designed and implemented _anomaly detection_ subsystem, with its key features summarized as follows:\n\n- A data ingestion module capable of fetching data from Wazuh and OpenSearch via a REST API;\n- A data transformation module for preprocessing, data type conversions and data aggregation;\n- A configuration management module for controlling backend configurations via dedicated files and loading such data into memory, (e.g., Wazuh, ML training parameters, etc.);\n- A data management module for centralizing data storage and retrieval and managing created and stored detectors, with dedicated features for saving trained ML models and their associated parameterization data, both used by detectors;\n- An integration of a machine learning package providing a [PyTorch-based implementation](https://github.com/ML4ITS/mtad-gat-pytorch) of the [MTAD-GAT algorithm](https://arxiv.org/pdf/2009.02040);\n- A dedicated anomaly detection engine (called AD Engine), aimed at orchestrating and generalizing common AD tasks and capturing them via an abstract and extensible design and implementation (currently under active development);\n- A driver module providing the entry point to the ADBox and currently using a CLI to interact with the user;\n- A set of AD use case scenario definitions encoded as YAML files, which can be directly used by the user, but they can also easily form the basis for creating new ones, tailored to the user's preferences for adjusting the training part as well as the prediction part of the ML-based algorithm and pipeline.\n\n### Front-end\n\n- A command-line interface (**CLI**) for efficient user interactions and automation via scripting integration, currently available via the driver module (under active development);\n- A dedicated Jupyter notebook for analysis and post-processing, providing a prepared playbook with tailored plotting and operating directly on top of anomaly prediction data produced by the ADBox backend;\n\n### Network and host monitoring\n\nTo achieve comprehensive monitoring capabilities, we combine well-established open-source solutions, namely [Wazuh](https://wazuh.com/), a cybersecurity platform that integrates SIEM and XDR capabilities and [Suricata](https://suricata.io/), an open-source Network Intrusion Detection System (NIDS). We provide deployment solutions that allow centralized monitoring for coping with limited resources (network agents relaying traffic data to a central node for processing) as well as running monitoring instances on each node and only grouping the obtained monitoring data in a centralized node for analysis.\n\nSee our [Instructions for IDPS and SIEM integrated deployment](./deployment/README.md) page for further details.\n\n## User manual\n\nPlease see the [Instructions for IDPS and SIEM integrated deployment](./deployment/README.md) and the [ADBox user manual](./docs/manual/README.md) page to learn more about the installation, setup requirements, overall usage and specific modules of the ADBox. \n\n## Documentation and technical specifications\n\nYou can visit our [traceability page](https://abstractionslab.github.io/idps-escape/docs/traceability/index.html) to view the technical specifications of IDPS-ESCAPE.\n\n## Getting Started\n\n### IDPS and SIEM integrated deployment\n\nNote that if you already have a running instance of Wazuh, and do not wish to integrate Suricata, you can simply skip to the [ADBox installation section](#adbox-installation).\n\nA complete and installation of the signature-based intrusion detection and the SIEM subsystems of IDPS-ESCAPE can be done using the following guides:\n\n1. Suricata, to enable network monitoring capabilities:\n\n      a.  [installation in a containerized environment](./deployment/suricata/suricata_installation.md#installation-and-configuration-of-suricata)\n      \n      b.  [configuration to local network](./deployment/suricata/suricata_installation.md#suricata-configuration-file)\n1. Wazuh central components installation, for SIEM \\\u0026 XDR:\n\n    a. [installation of Dashboard, Manager and Indexer in a containerized environment ](./deployment/wazuh/wazuh_installation.md)\n\n    b. [configuration to local system](./deployment/wazuh/wazuh_installation.md#next-steps)\n\n1. [Installation of a Wazuh agent](./deployment/wazuh/wazuh_agents.md) to enable host monitoring capabilities.\n\n    a. Possibly, deployment of additional agents on other remote hosts (system *endpoints*), same as above.\n  \n    b. Possibly, [enable remote traffic monitoring](./deployment/remote_monitoring/remote_monitoring.md).\n\n1. Follow [integration procedure of Suricata and Wazuh](./deployment/integration.md).\n\n Details of the above steps and scripts are provided in the [Guide for IDPS and SIEM integrated deployment](./deployment/README.md).\n\n This integration guarantees:\n \n - joint monitoring of host and network events, and\n - centralized storage.\n\nAll the data ending up in the central SIEM \\\u0026 XDR can now be fed to ADBox for training ML models and anomaly detection, providing a holistic view of the system(s) under monitoring.\n\n### ADBox installation\n\nADBox can be deployed using the following methods:\n  \n- [Deployment using Docker and our shell scripts](./docs/manual/installation.md#installing-adbox-via-docker-and-shell-script) (**recommended for end-users**);\n\n- [Deployment in a development containerized environment in VS Code](./docs/manual/installation.md#installing-adbox-in-a-development-containerized-environment-in-vs-code) (**recommended for developers**);\n\nBelow we describe the deployment using Docker and shell scripts. For other installation methods, please see the [installation](./docs/manual/installation.md) page of the user manual.\n\n#### Installing ADBox via Docker and shell scripts\n\nThe easiest and recommended way to deploy and run ADBox is described in this section and can be achieved using our Docker definition file and build/execution scripts, which can be found in the repository. The instructions below work on GNU/Linux, MacOS and Windows Subsystem for Linux (WSL). ADBox is run as a service in a Docker container.\n\n##### Requirements\n\nThe following pieces of software are necessary for setting up the ADBox as a service in a Docker container.\n\n- A local installation of [Docker Engine](https://docs.docker.com/engine/install/), with the Docker service running prior to launching ADBox.\n\n##### Installation\n\n1. Simply clone the repository or download a ZIP archive of the project \n\n    ```sh\n    git clone https://github.com/AbstractionsLab/idps-escape.git\n    ```\n\n2. Unzip the archive, switch to the extracted directory (`cd foldername`) via a terminal running a shell (e.g., bash, zsh) and make the two shell scripts executable: `chmod +x script-name.sh`. Then, change working directory to the cloned folder containing all the files along with the Dockerfile and build the image by running our build script: `./build-adbox.sh`;\n\n3. Finally, launch ADBox by executing `./adbox.sh`, which runs the default mode if no arguments are provided to the command-line interface (CLI); running `./adbox.sh -h` displays the CLI help menu describing the available commands.\n\n![ADBox CLI](./docs/manual/_figures/adbox-cli.png)\n\n### Usage\n\nPlease note that you can set the parameters (IP, port, username and password) for connecting to Wazuh via the [Wazuh credentials JSON file](./siem_mtad_gat/assets/secrets/wazuh_credentials.json).\n\nThe ADBox driver/CLI currently provides four options:\n\n1. Running ADBox using the `-u` flag following by the ID of a use case YAML file (stored under `siem_mtad_gat/assets/drivers`), e.g., `./adbox.sh -u 2` to start a complete training and prediction pipeline determined by an AD [use case](./docs/manual/use_case.md) scenario, in this case `uc_2.yaml`.\n\n1. Running ADBox using the `-i` flag, i.e., `./adbox.sh -i` running the interactive console (**the console currently contains a known bug for prediction-only jobs (i.e., no training and using a trained model), please use option 1**).\n\n1. Running ADBox without any arguments: it runs a training and prediction pipeline using default configurations.\n\n1. Running ADBox using the `-c` flag, i.e., `./adbox.sh -c` to check your connection with Wazuh, which is recommended to ensure a successful channel can be established before executing AD workflows. Otherwise, in the absence of a functional connection, ADBox automatically falls back to local default configuration files and prepared sample training and prediction data.\n\n#### Verifying connection with Wazuh\n\nBefore running ADBox training and prediction scenarios, you can verify whether a connection between ADBox and an instance of Wazuh can be established successfully using the `-c` flag:\n\n```sh\n./adbox.sh -c\n```\n\nYou can set/modify the parameters (IP, port, username and password) for connecting to Wazuh via the [Wazuh credentials JSON file](./siem_mtad_gat/assets/secrets/wazuh_credentials.json).\n\n#### Executing a use case from a YAML file\n\nThe ADBox takes inputs from a YAML file stored in the `/siem_mtad_gat/assets/drivers/` folder. By default, the folder contains several YAML-encoded use cases, which can be used for training models and running predictions.\n\nA training and detection use case can be run by providing the `-u` flag along with an integer to the script.\n\n```sh\n./adbox.sh -u {number}\n```\n\nFor example, to run use case 1, execute the script as follows:\n\n```sh\n./adbox.sh -u 1\n```\n\nWith this input, the ADBox will take the inputs specified in the `uc_1.yaml` file.\n\nThe folder containing the YAML files also contains a `driver.yaml` file that provides a template for writing your own custom YAML files.\n\nFor example, one can run a new [use case](./docs/manual/use_case.md), by specifying different input parameters in a new YAML file, called `uc_6.yaml` and then run the adbox as follows:\n\n```sh\n./adbox.sh -u 6\n```\n\nAll the outputs produced as a result of running use cases are stored in `./siem_mtad_gat/assets/detector_models/{detector_id}/prediction_{current_date}/predict_output.json`.\n\n#### Interacting with a console (has bugs in alpha version):\n\n```sh\n./adbox.sh -i\n```\n\nRunning the script using the `-i` flag will open an interactive console which will ask for user inputs.\n\nThe output of the all the detections performed through the console are stored in `./siem_mtad_gat/assets/detector_models/{detector_id}/prediction_{current_date}/predict_output.json` file.\n\n\n#### Executing as default:\n\n```sh\n./adbox.sh\n```\n\nIn this mode, the ADBox will train a detector using the default arguments and then also perform detection based on default arguments, with the detector trained using the previously mentioned default arguments. To know more about the input arguments used in default mode, visit the [user manual](./docs/manual/README.md) page.\n\nThe output of the default detection is also stored in `./siem_mtad_gat/assets/detector_models/{detector_id}/prediction` folder.\n\n## Example of a use case scenario\n\nIn this section, we present an example illustrating the usage of ADBox, adopting the end user point of view.\n\n### My system\n\nI have deployed all the components as explained in the [guide for IDPS and SIEM integrated deployment](./deployment/guide_sids.md) and the\n[ADBox user manual installation page](./docs/manual/installation.md). Moreover, I have enabled [Linux resource monitoring](./docs/manual/linux_resource.md).\n\n### My use case\n\nI want a detector which correlates resource usage and rules statistics. Once created, I want this detector to keep running on new data.\n\nTherefore, I prepare a dedicated [use case file](./docs/manual/use_case.md).\n\nNamely,\n\n- I have to identify the features of the [multivariate time-series](./docs/manual/time_series.md) that I want to perform detection on. To do so, I analyze the event logs of my system.\n\n- For every feature, I choose a suitable aggregation method for the *granularity* I wish to use. \n\n- For example, I would like a new point every 30 seconds (**lowest advised granularity**). Every point has 3 dimensions representing the average CPU percent usage, average memory percent usage and the total `firedtimes` rule statistics parameter.\n\nI have to decide:\n\n- how I wish to handle missing values,\n- the detector name,\n- the data that is to be used to train my detector,\n- the detection interval (window size), and\n- the number of epochs for training.\n\nFor example, I want anomalies to be flagged over intervals of 3 minutes, so the window size should be 8. Then, I also want to use all my data of the current month that is already available to train the detector.\n\nFor the prediction, I want almost real-time results but I would like to fetch data in batches. For example, I want to get points every 6 minutes, then in batches of 12 points.\n\nI encode this in [`siem_mtad_gat/assets/drivers/uc-9.yaml`](../../siem_mtad_gat/assets/drivers/uc_9.yaml)\n\n```YAML\ntraining:\n  aggregation: true\n  aggregation_config:\n    features:\n      data.cpu_usage_%:\n      - average\n      data.memory_usage_%:\n      - average\n      rule.firedtimes:\n      - count\n    fill_na_method: Zero\n    granularity: 30s\n    padding_value: 0\n  categorical_features: false\n  columns:\n  - data.cpu_usage_%\n  - data.memory_usage_%\n  - rule.firedtimes\n  display_name: detector_example_3min\n  index_date: '2024-08-*'\n  train_config:\n    epochs: 8\n    window_size: 6\n\nprediction: \n    run_mode: BATCH\n    batch_size: 12\n```\n#### Running the pipeline\n\nI run ADBox\n\n```sh\n./adbox.sh -u 9\n```\n\nand stop it after a few hours.\n\nThis produces a detector with id `2d36a80a-c47a-4eb4-bb3e-5b2bfb90dc9` and the associated folder [`siem_mtad_gat/assets/detector_models/2d36a80a-c47a-4eb4-bb3e-5b2bfb90dc9`](./siem_mtad_gat/assets/detector_models/2d36a80a-c47a-4eb4-bb3e-5b2bfb90dc95/).\n\n```sh\n2d36a80a-c47a-4eb4-bb3e-5b2bfb90dc9\n├── input\n│   ├── detector_input_parameters.json\n│   └── training_config.json\n├── prediction\n│   ├── uc-9_predicted_anomalies_data-1_2024-08-30_10-24-15.json\n│   └── uc-9_predicted_data-1_2024-08-30_10-24-15.json\n└── training\n    ├── losses_train_data.json\n    ├── model.pt\n    ├── scaler.pkl\n    ├── spot\n    │   ├── spot_feature-0.pkl\n    │   ├── spot_feature-1.pkl\n    │   ├── spot_feature-2.pkl\n    │   └── spot_feature-global.pkl\n    ├── test_output.pkl\n    ├── train_losses.png\n    ├── train_output.pkl\n    └── validation_losses.png\n\n4 directories, 15 files\n```\n\n### Detection analysis\n\nUsing the [ADBox Result Visualizer Notebook](./siem_mtad_gat/frontend/viznotebook/result_visualizer.ipynb), I can plot the results and analyze them. Here, I collect a few observations.\n\n#### Training\n\nThe training losses are rather good for 8 epochs, while the same cannot be said about the validation losses. I could try the same setting, while training the detector for more epochs.\n\n![training losses](./siem_mtad_gat/assets/detector_models/2d36a80a-c47a-4eb4-bb3e-5b2bfb90dc95/training/train_losses.png)\n\n![validation losses](./siem_mtad_gat/assets/detector_models/2d36a80a-c47a-4eb4-bb3e-5b2bfb90dc95/training/validation_losses.png)\n\n#### Prediction\n\nI ran the batch prediction from `2024-08-30T10:18:04Z` to `2024-08-30T13:05:30Z` UTC time.\n\n**Global overview**\n\n![Batch](./docs/manual/_figures/example_global.png)\n\nDuring this time period, 5 anomalous windows were flagged, 4 consecutive and 1 alone. Let's call them A1 and A2, respectively.\n\n![a1](./docs/manual/_figures/example-a1.png)\n\n![a2](./docs/manual/_figures/example-a2.png)\n\n**Feature overview**\n\nLooking at the feature statistics, we can see these two anomalies expressing two different cases:\n\n- anomalies in A1 can be also considered anomalies at the feature level.\n- the anomaly in A2 is anomalous **only** at a global level. \n\n![f1](./docs/manual/_figures/f1.png)\n\n![f2](./docs/manual/_figures/f2.png)\n\n![t1](./docs/manual/_figures/true.png)\n\n**Wazuh Dashboard**\n\nLooking at the Wazuh Dashboard, we can observe a high number of events in proximity of A1:\n\n![w1](./docs/manual/_figures/example-w1.png)\n\n![w2](./docs/manual/_figures/example-w2.png)\n\n### Mapping anomalies to real events\n\nWe traced the two anomalies to two real events that had happened in the corresponding detections intervals:\n\n- A1 matches with the running of `apt update` and `apt upgrade` on the host machine.\n- A2 matches with a reboot.\n\n### Remarks\n\nIn both cases, the actions that (most probably) generated the anomalies had been carried out by a system administrator. In case not, while A1 would have been noticed by looking at single features and/or Wazuh; A2 would have not been as obvious to track.\n\n## Disclaimer: use of alpha/experimental software\n\nThis software is currently in its alpha or experimental phase and is provided for testing and evaluation purposes only. It may contain errors, bugs, or other issues that could result in security vulnerabilities, data loss, or other unpredictable outcomes. As such, **this software is not intended for use in production environments** or for handling sensitive, confidential, or critical information.\n\nIn particular, given the nature of security-related software, it is crucial to understand that the algorithms, protocols, and implementations within this software may not have undergone thorough security audits or peer review. **Do not rely on this software for critical system functions.**\n\nThe developers, contributors, and affiliated organizations **disclaim all warranties, express or implied,** including but not limited to the implied warranties of fitness for a particular purpose. **No guarantee is made regarding the correctness, completeness, or security** of the software, and you assume full responsibility for any risks associated with its use.\n\nBy using this software, you acknowledge that you understand the risks and agree to use it **at your own risk.** You are strongly encouraged to conduct your own security assessments and tests before deploying this software in any environment.\n\n### Usage recommendations and remarks\n\nWe advise the user __not__ to \n- set the detector time granularity parameter to a value lower than 30s,\n- run prediction-only use cases if there are no corresponding detectors available in the [detectors folder](./siem_mtad_gat/assets/detector_models/). \n\nFurthermore, we highlight the following points:\n\n- Anomalous timestamps should be considered more as period indicators rather than precise links to an event for two reasons: \n  - (i) data points from events are aggregated, to which rounding is applied during preprocessing; \n  - (ii) the anomaly is defined in terms of windows, hence consecutive sets of events.\n- Depending on the selected configurations, running the full stack of IDPS-ESCAPE may require up to 26 GB of persistent storage, while RAM usage for the default ADBox configuration remains close to 4 GB, the same as the recommended value for Wazuh, which is used as our source of data for training and prediction. Note that the various subsystems can be deployed on different nodes, e.g., the ADBox on one node and our customized Wazuh+Suricata setup on another, or all three on separate nodes (see the [integration](./deployment/README.md) and [remote monitoring](./deployment/remote_monitoring/remote_monitoring.md) pages).\n- Clearly, the MTAD-GAT hyperparameters (e.g., the number of GRU layers) require tuning when it comes to training machine learning models.\n\n## Roadmap\n\nSome of the currently planned items include:\n\n- automated unit/integration/system test suites providing coverage for critical parts of the software;\n- a web-based GUI frontend, either standalone or integrated into Wazuh or OpenSearch;\n- tailoring the underlying ADBox algorithms to specific SOC operations.\n\nFor details on our roadmap and features planned for future releases, please see the [Wiki](https://github.com/AbstractionsLab/idps-escape/wiki) section of this repository.\n\n## License\n\nCopyright (c) itrust Abstractions Lab and itrust consulting. All rights reserved.\n\nLicensed under the [GNU Affero General Public License (AGPL) v3.0](LICENSE) license.\n\n## Acknowledgment\n\nThe creation of the IDPS-ESCAPE software tools and its knowledge base is co-funded by the Ministry of the Economy of Luxembourg, in the context of the CyFORT project.\n\n## Contact\n\nIf you wish to learn more about the project, feel free to contact us at Abstractions Lab: info@abstractionslab.lu","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabstractionslab%2Fidps-escape","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabstractionslab%2Fidps-escape","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabstractionslab%2Fidps-escape/lists"}