{"id":43437904,"url":"https://github.com/idlab-discover/rustiflow","last_synced_at":"2026-02-02T21:02:09.783Z","repository":{"id":222807963,"uuid":"758410868","full_name":"idlab-discover/RustiFlow","owner":"idlab-discover","description":"Flow feature extraction tool built in Rust using eBPF","archived":false,"fork":false,"pushed_at":"2025-05-02T12:09:03.000Z","size":16082,"stargazers_count":18,"open_issues_count":4,"forks_count":1,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-05-02T13:23:30.695Z","etag":null,"topics":["data-science","dataset-generation","ebpf-programs","feature-extraction","machine-learning","network-analysis","network-monitoring","network-security","packet-analyser","packet-capture","pcap","rust","throughput-performance","traffic-analysis"],"latest_commit_sha":null,"homepage":"https://idlab-discover.github.io/RustiFlow","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/idlab-discover.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-02-16T09:06:20.000Z","updated_at":"2025-05-02T12:06:18.000Z","dependencies_parsed_at":"2024-03-02T15:23:20.879Z","dependency_job_id":"c7121f48-4fc1-4a19-aebd-43ad54cd7aab","html_url":"https://github.com/idlab-discover/RustiFlow","commit_stats":null,"previous_names":["matissecallewaert/nids-feature-extraction-tool","matissecallewaert/rustiflow","idlab-discover/rustiflow"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/idlab-discover/RustiFlow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idlab-discover%2FRustiFlow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idlab-discover%2FRustiFlow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idlab-discover%2FRustiFlow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idlab-discover%2FRustiFlow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/idlab-discover","download_url":"https://codeload.github.com/idlab-discover/RustiFlow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/idlab-discover%2FRustiFlow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29019545,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-02T18:51:31.335Z","status":"ssl_error","status_checked_at":"2026-02-02T18:49:20.777Z","response_time":58,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","dataset-generation","ebpf-programs","feature-extraction","machine-learning","network-analysis","network-monitoring","network-security","packet-analyser","packet-capture","pcap","rust","throughput-performance","traffic-analysis"],"created_at":"2026-02-02T21:02:08.104Z","updated_at":"2026-02-02T21:02:09.778Z","avatar_url":"https://github.com/idlab-discover.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"![banner](figures/banner.jpg)\n\n# A Network Traffic Feature Extraction Tool\n\n## \u003cimg src=\"figures/RustiFlow_nobg.png\" width=\"60px\"/\u003e Overview\n\nThis tool is engineered for robust and efficient feature extraction, particularly for applications such as network intrusion detection systems, among others. Leveraging Rust language and eBPF, it excels in processing high volumes of network traffic with remarkable speed and throughput. (When your traffic is already captured, don't worry! It also has a build in pcap reader which is also amazingly fast.) With various pre-defined feature sets and the ability to create custom feature sets, RustiFlow offers a versatile solution for network security applications.\n\n\u003ca href=\"https://github.com/idlab-discover/RustiFlow/actions\"\u003e![Badge displaying GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/idlab-discover/RustiFlow/rust.yml?logo=github)\u003c/a\u003e\n\u003ca href=\"https://idlab-discover.github.io/RustiFlow\"\u003e ![Badge linking to the project documentation website](https://img.shields.io/website?url=https%3A%2F%2Fidlab-discover.github.io%2FRustiFlow\u0026label=Documentation)\u003c/a\u003e \u003ca href=\"https://github.com/idlab-discover/RustiFlow/blob/main/LICENSE\"\u003e ![GitHub license](https://img.shields.io/github/license/idlab-discover/RustiFlow) \u003c/a\u003e\n\n![Ubuntu 24](https://img.shields.io/badge/Tested%20on%20ubuntu-purple?logo=ubuntu)\n\n![Animated image showing network flows](figures/flows.gif)\n\n## \u003cimg src=\"figures/RustiFlow_nobg.png\" width=\"60px\"/\u003e Key Features\n\n- **High Throughput:** Utilizes Rust and the [Aya](https://aya-rs.dev/) library for eBPF program compilation and execution, ensuring exceptional performance and resource efficiency.\n- **Versatile Feature Sets:** Offers a variety of pre-defined feature sets (flows) and the flexibility to create custom feature sets tailored to specific requirements. An example of the custom flow is shown [here](https://github.com/idlab-discover/RustiFlow/blob/main/rustiflow/src/flows/custom_flow.rs).\n- **Pcap File Support:** Facilitates packet analysis from pcap files, compatible with both Linux and Windows generated files.\n- **Diverse Output Options:** Features can be outputted to the console, a CSV file, or other formats with minimal effort.\n\n## Feature sets\n\nSee the [wiki](https://github.com/idlab-discover/RustiFlow/wiki) for the different feature sets available.\n\n## \u003cimg src=\"figures/RustiFlow_nobg.png\" width=\"60px\"/\u003e Architecture\n\n### Realtime processing\n\n![RustiFlow Architecture Realtime](figures/realtime.png)\n\n### Offline PCAP processing\n\n![RustiFlow Architecture Offline](figures/offline.png)\n\n## \u003cimg src=\"figures/RustiFlow_nobg.png\" width=\"60px\"/\u003e Using the release binary:\n\nCopy the rustiflow binary that you can find in this repo in releases to a location of your choice or to the `/usr/local/bin` folder.\nIf it does not have the right permissions, you can run the following command:\n\n```bash\nchmod +x /path/to/rustiflow\n```\n\n### Using commands:\n\nYou can then run the binary with the following commands displayed on the [help menu](#usage-instructions).\n\n### Using the tui interface:\n\nIf you want a more graphical interface, you can use the tui interface by just running `rustiflow` without any arguments. This will open a field where you can enter a configuration file you want to edit or you can choose to start new. After that, the following interface will show up:\n\n![The tui interface](figures/tui_rustiflow.GIF)\n\n\u003e **NOTE:** When using the save button, you will be prompted for a filename. You can reuse this file with following command:\n\n```bash\nrustiflow --config-file \u003cfilename\u003e realtime \u003cinterface\u003e [--only-ingress]\n```\n\n```bash\nrustiflow -c \u003cfilename\u003e pcap \u003cpath to pcap file\u003e\n```\n\n\u003e After saving the configuration file, you can safely reset without changing the configuration file.\n\n### Using the configuration file:\n\nThis is an example of a configuration file that you can use to run the tool with the `--config-file` option.\n\n```toml\n[config]\nfeatures = \"CIDDS\"\nactive_timeout = 522\nidle_timeout = 885855\nearly_export = 25\nexpiration_check_interval = 0\n\n[output]\noutput = \"Csv\"\nexport_path = \"path/to/output.csv\"\nheader = false\ndrop_contaminant_features = true\n```\n\nExample 2:\n\n```toml\n[config]\nfeatures = \"Nfstream\"\nactive_timeout = 3600\nidle_timeout = 120\nearly_export = 10\nexpiration_check_interval = 60\nthreads = 8\n\n[output]\noutput = \"Print\"\nheader = true\ndrop_contaminant_features = false\n```\n\n## \u003cimg src=\"figures/RustiFlow_nobg.png\" width=\"60px\"/\u003e Using the Container:\n\nMake sure that you don't use docker desktop and that you don't have it installed on your machine. If you have this setup, it will not work as intended as the `--network host` will not link the container to the host network, but to the network of a VM that docker desktop uses.\n\n- **Build the Container**:\n  ```bash\n  docker build -t rustiflow .\n  ```\n- **Run the Container**:\n  ```bash\n  docker run --network host -v /path/on/host:/app rustiflow [ARGS like you are used to]\n  ```\n  Run it with the --privileged flag if you want to capture traffic in real-time.\n- **Example**:\n  ```bash\n  docker run --network host -v /home/user/pcap:/app rustiflow pcap basic-flow 60 /app/pcap.pcap print\n  ```\n  ```bash\n  docker run --privileged --network host -v /home/matisse/Documents:/app rustiflow realtime enp5s0 cic-flow 60 csv /app/output.csv\n  ```\n\n## \u003cimg src=\"figures/RustiFlow_nobg.png\" width=\"60px\"/\u003e Installation Guide for development\n\n### Prerequisites:\n\n- **libpcap-dev**:\n  ```sh\n  sudo apt install libpcap-dev\n  ```\n- **Rust Installation**:\n  ```bash\n  curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\n  ```\n- **Nightly Rust Toolchain**:\n  ```bash\n  rustup install stable\n  rustup toolchain install nightly --component rust-src\n  ```\n\n### bpf Linker Installation:\n\n- **For Linux x86_64**:\n  ```bash\n  cargo install bpf-linker\n  ```\n- **For MacOS/Linux (Other Architectures)**:\n  ```bash\n  brew install llvm\n  cargo install --no-default-features bpf-linker\n  ```\n- **Ubuntu 20.04 LTS Specific**:\n  ```bash\n  sudo apt install linux-tools-5.8.0-63-generic\n  export PATH=/usr/lib/linux-tools/5.8.0-63-generic:$PATH\n  ```\n\n## \u003cimg src=\"figures/RustiFlow_nobg.png\" width=\"60px\"/\u003e Building the Project\n\n- **eBPF Programs**:\n  ```bash\n  cargo xtask ebpf-ipv4\n  cargo xtask ebpf-ipv6\n  ```\n- **User Space Programs**:\n  ```bash\n  cargo build\n  ```\n\n## Running the Project in dev mode\n\n```bash\ncargo xtask run -- [OPTIONS] \u003cCOMMAND\u003e\n```\n\n## \u003cimg src=\"figures/RustiFlow_nobg.png\" width=\"60px\"/\u003e Usage Instructions\n\n### Command Help:\n\n```bash\nrustiflow help\n```\n\n```bash\nUsage: rustiflow [OPTIONS] \u003cCOMMAND\u003e\n\nCommands:\n  realtime  Real-time feature extraction\n  pcap      Feature extraction from a pcap file\n  help      Print this message or the help of the given subcommand(s)\n\nOptions:\n  -c, --config-file \u003cCONFIG_FILE\u003e\n          Configuration file path\n\n  -f, --features \u003cFEATURES\u003e\n          The feature set to use (required if no config file is provided)\n\n          Possible values:\n          - basic:     A basic flow that stores the basic features of a flow\n          - cic:       Represents the CIC Flow, giving 83 features\n          - cidds:     Represents the CIDDS Flow, giving 10 features\n          - nfstream:  Represents a nfstream inspired flow, giving 69 features\n          - rustiflow: Represents the Rusti Flow, giving 120 features\n          - custom:    Represents a flow that you can implement yourself\n\n      --active-timeout \u003cACTIVE_TIMEOUT\u003e\n          The maximum time a flow is allowed to last in seconds (optional)\n\n          [default: 3600]\n\n      --idle-timeout \u003cIDLE_TIMEOUT\u003e\n          The maximum time with no packets for a flow in seconds (optional)\n\n          [default: 120]\n\n      --early-export \u003cEARLY_EXPORT\u003e\n          The print interval for open flows in seconds (optional)\n\n      --expiration-check-interval \u003cEXPIRATION_CHECK_INTERVAL\u003e\n          Interval (in seconds) for checking and expiring flows in the flowtable. This represents how often the flowtable should be scanned to remove inactive flows\n\n          [default: 60]\n\n      --threads \u003cTHREADS\u003e\n          The numbers of threads to use for processing packets (optional) (default: 5, maximum number of logical CPUs)\n\n      -o, --output \u003cOUTPUT\u003e\n              Output method (required if no config file is provided)\n\n              Possible values:\n              - print: The output will be printed to the console\n              - csv:   The output will be written to a CSV file\n\n          --export-path \u003cEXPORT_PATH\u003e\n              File path for output (used if method is Csv)\n\n          --header\n              Whether to export the feature header\n\n          --drop-contaminant-features\n              Whether to drop contaminant features\n\n      -h, --help\n              Print help (see a summary with '-h')\n\n      -V, --version\n              Print version\n\n```\n\n## Logging in both development or using the binary\n\n### Development\n\n```bash\nRUST_LOG=info cargo xtask run --\n```\n\n### Binary\n\n```bash\nsudo RUST_LOG=info rustiflow\n```\n\n**Note:** For specific logging levels, adjust `RUST_LOG` to `error` for error messages, and `debug` for debug messages. If you don't want any additional logs, just remove `RUST_LOG=info`.\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidlab-discover%2Frustiflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fidlab-discover%2Frustiflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fidlab-discover%2Frustiflow/lists"}