{"id":21541496,"url":"https://github.com/ate329/nsl-kdd-feature-extractor","last_synced_at":"2025-10-30T04:35:46.177Z","repository":{"id":263140017,"uuid":"889459012","full_name":"Ate329/NSL-KDD-feature-extractor","owner":"Ate329","description":"Python-based tool designed to process network traffic packets and extract features compliant with the NSL-KDD dataset format.","archived":false,"fork":false,"pushed_at":"2024-11-19T20:01:27.000Z","size":21,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-24T08:28:16.561Z","etag":null,"topics":["cyber-security","cybersecurity","data","data-science","extractor","feature-extraction","machine-learning","network-analysis","nsl-kdd","nsl-kdd-dataset"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ate329.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-16T12:47:31.000Z","updated_at":"2024-11-19T20:01:52.000Z","dependencies_parsed_at":"2025-01-24T08:36:04.717Z","dependency_job_id":null,"html_url":"https://github.com/Ate329/NSL-KDD-feature-extractor","commit_stats":null,"previous_names":["ate329/nsl-kdd-feature-extractor"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ate329%2FNSL-KDD-feature-extractor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ate329%2FNSL-KDD-feature-extractor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ate329%2FNSL-KDD-feature-extractor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ate329%2FNSL-KDD-feature-extractor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ate329","download_url":"https://codeload.github.com/Ate329/NSL-KDD-feature-extractor/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244122509,"owners_count":20401440,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cyber-security","cybersecurity","data","data-science","extractor","feature-extraction","machine-learning","network-analysis","nsl-kdd","nsl-kdd-dataset"],"created_at":"2024-11-24T05:04:54.822Z","updated_at":"2025-10-30T04:35:46.105Z","avatar_url":"https://github.com/Ate329.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NSL-KDD Feature Extractor\n\n## **Introduction**\n\nThe **NSL-KDD Feature Extractor** is a Python-based tool designed to process network traffic packets and extract features compliant with the NSL-KDD dataset format. It enables researchers and developers to analyze network traffic and apply machine learning models for intrusion detection, anomaly detection, or other cybersecurity applications.\n\nDataset used for testing: https://www.kaggle.com/datasets/hassan06/nslkdd/data\n\n## **Features**\n\n1. **Packet Analysis**\n   - Supports live packet capture using `scapy`.\n   - Processes TCP, UDP, ICMP, ARP, and DNS packets.\n\n2. **Feature Extraction**\n   - Generates NSL-KDD dataset-compatible features for machine learning.\n   - Includes connection-based and statistical features such as `same_srv_rate`, `srv_serror_rate`, and more.\n\n3. **Customizable and Scalable**\n   - Easily extendable for new protocols or custom features.\n   - Handles both live traffic and offline packet captures.\n\n4. **Internal Traffic Filtering**\n   - Option to exclude internal traffic during feature extraction.\n\n## **How It Works**\n\n### **Workflow Diagram**\n\n```\n                      +------------------+\n                      |   Network Traffic|\n                      +------------------+\n                               |\n                               v\n               +-------------------------------+\n               |    Packet Capturing           |\n               |  (Using Scapy Framework)      |\n               +-------------------------------+\n                               |\n                               v\n          +----------------------------------------+\n          |    NSL-KDD Feature Extraction          |\n          |  (network_feature_extractor.py)        |\n          +----------------------------------------+\n                               |\n                               v\n          +----------------------------------------+\n          |        Generated Feature Set           |\n          | - Duration, Protocol Type, Service     |\n          | - Flag, Src Bytes, Dst Bytes           |\n          | - Statistical Features (e.g.,          |\n          |   srv_serror_rate, same_srv_rate)      |\n          +----------------------------------------+\n```\n\n## **Setup**\n\n### **Prerequisites**\n\n- **Python 3.11** or later\n- **Scapy** for packet capture\n- **Pandas** for data manipulation\n\n### **Installation**\n\n1. Clone the repository:\n   ```\n   git clone https://github.com/Ate329/NSL-KDD-feature-extractor.git\n   cd nsl-kdd-feature-extractor\n   ```\n\n2. Install required dependencies:\n   ```\n   pip install -r requirements.txt\n   ```\n\n## **Usage**\n\n### **1. Extracting Features**\n\n```python\nfrom network_feature_extractor import NetworkFeatureExtractor\n\n# Initialize the extractor\nextractor = NetworkFeatureExtractor(interface=\"eth0\", timeout=60)\n\n# Capture live traffic and extract features\ndef process_packet(packet):\n    features = extractor.extract_features(packet)\n    if features:\n        print(features)\n\nextractor.start_capture(callback=process_packet)\n```\n\n### **2. Example Output**\n\nExtracted features will include:\n\n```json\n{\n  \"duration\": 1.23,\n  \"protocol_type\": \"tcp\",\n  \"service\": \"http\",\n  \"flag\": \"SF\",\n  \"src_bytes\": 345,\n  \"dst_bytes\": 512,\n  \"same_srv_rate\": 0.75,\n  \"srv_serror_rate\": 0.0,\n  ...\n}\n```\n\n## **Customization**\n\n1. **Add New Features**:\n   - Extend the `extract_features()` method to compute additional metrics.\n\n2. **Handle Custom Protocols**:\n   - Add specific processing for protocols like DNS or HTTP in `_extract_ip_features()` or `_extract_arp_features()`.\n\n3. **Exclude Internal Traffic**:\n   - Enable internal traffic detection using the `detect_internal=True` parameter.\n\n## **Development Notes**\n\n- This feature extractor aligns with the NSL-KDD dataset specification, enabling seamless integration with machine learning models trained on similar datasets.\n- The modular structure makes it adaptable for other datasets or real-world scenarios.\n\n## **Contributing**\n\nWe welcome contributions! If you’d like to extend the functionality or report a bug, feel free to submit a pull request or open an issue.\n\n## **License**\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for more details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fate329%2Fnsl-kdd-feature-extractor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fate329%2Fnsl-kdd-feature-extractor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fate329%2Fnsl-kdd-feature-extractor/lists"}