{"id":18426688,"url":"https://github.com/marferdom/angi","last_synced_at":"2025-04-13T18:55:11.924Z","repository":{"id":208632650,"uuid":"722066506","full_name":"MarFerDom/angi","owner":"MarFerDom","description":"Tests on generalization of ML-based NIDS from different datasets","archived":false,"fork":false,"pushed_at":"2023-11-22T13:18:11.000Z","size":2326,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-16T08:27:21.219Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MarFerDom.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-22T11:12:29.000Z","updated_at":"2024-02-28T10:53:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"eb1515b6-08cd-446d-a6ce-d493994cda68","html_url":"https://github.com/MarFerDom/angi","commit_stats":null,"previous_names":["marferdom/angi"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarFerDom%2Fangi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarFerDom%2Fangi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarFerDom%2Fangi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MarFerDom%2Fangi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MarFerDom","download_url":"https://codeload.github.com/MarFerDom/angi/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248766518,"owners_count":21158301,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T05:08:33.895Z","updated_at":"2025-04-13T18:55:11.917Z","avatar_url":"https://github.com/MarFerDom.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Analysis of NIDS Generalization\n\n\tThe last step in an analysis of NIDS generalization between datasets of different contexts.\n\tCode is maintained for reproducibility of the source material and will not be improved upon nor will it be updated in any way.\n\n## Fist step\n\n- Clone https://github.com/c2dc/aBFF-sbseg2022 and run prior steps.\n\n- Include the notebook files on this repository.\n\n## MAIN FILES\n\n\u003e - `analyse_all_results_graphs.ipynb`\n\u003e - `analyse_all_results.ipynb`\n\u003e - `explain_selected_pairs_reduced.ipynb`\n\n\n### analyse_all_results_graphs.ipynb\n\n\u003e Loads all results from tests with and without sampling strategies.\n\u003e Makes the nomenclature consistent.\n\u003e Calculates metrics for each result (Recall, Precision, TNR, F1-score, and MCC).\n\u003e Makes TPR and TNR, F1-score, and MCC graphs.\n\u003e Builds latex for graphs.\n\u003e Makes DT importance graphs.\n\n### analyse_all_results.ipynb\n\n\u003e Similar to previous file.\n\u003e Only calculates MCC.\n\u003e Builds tables for cross-testing MCC results per model type and per source dataset.\n\u003e Builds latex for each table.\n\n### explain_selected_pairs_reduced.ipynb\n\n\u003e Builds explanation models for the selected models and targets.\n\u003e Creates importance bar plots.\n\u003e Retrain models on selected features.\n\u003e Test new models.\n\u003e Make LaTeX table of results for source dataset and FS's target dataset.\n\n\n# ABOUT FEATURES\n## FEATURES EXTRACTION\n- NB-15 extracts features from Argus and Zeek programs. featuresNB15.py joins with label file provided by the pcap source and calculates 12 extra features as described in (Moustafa et al 2015).\n- CICFlowMeter extracts CIC features. featuresCIC.py joins with label file provided by the pcap source and adds Flow ID.\n\n# SOURCE PCAP\n\nAB-TRAP (bonafide) is extracted from MAWILab by filering with labeled file provided by the pcap source.\nbonafide is split for processing in Argus/Zeek. Files must be joined before calculating group features.\nattack is generated by simulation and each ip represents an attack category.\nattack and bonafide datasets must be joined at the end.\nattacks and traffic occur separatedly in time.\n\nUNSW-NB15 label file NUSW-NB15_GT.csv is provided by the pcap source.\nCIC-IDS2017 label files in GeneratedLabelledFlows.zip are provided by the pcap source.\n\n# PCAPs\n Some packet capture files used/generated during aBFF development\n\n* Original MAWILab pcap files extracted from: http://www.fukuda-lab.org/mawilab/\n\nR. Fontugne, P. Borgnat, P. Abry, K. Fukuda. \"MAWILab: Combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking\". ACM CoNEXT 2010. Philadelphia, PA. December 2010.\n\n* CIC-BoT-IoT and CIC-ToN-IoT extracted from https://staff.itee.uq.edu.au/marius/NIDS_datasets/\n\nM. Sarhan, S. Layeghy, and M. Portmann, An explainable machine learning-based network intrusion detection system for enabling generalisability in securing iot networks, 2021. arXiv:2104.07183 [cs.CR]\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarferdom%2Fangi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarferdom%2Fangi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarferdom%2Fangi/lists"}