{"id":17967244,"url":"https://github.com/mohammedsaim-quadri/intrusion_detection-system","last_synced_at":"2025-07-09T05:04:07.971Z","repository":{"id":259299456,"uuid":"877238287","full_name":"MohammedSaim-Quadri/Intrusion_Detection-System","owner":"MohammedSaim-Quadri","description":"This project is an Intrusion Detection System (IDS) using machine learning (ML) and deep learning (DL) to detect network intrusions. It leverages the CICIDS2018 dataset to classify traffic as normal or malicious. Key features include data preprocessing, model training, hyperparameter tuning, and Docker containerization for scalable deployment.","archived":false,"fork":false,"pushed_at":"2025-07-04T22:25:26.000Z","size":9028,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-04T23:26:49.756Z","etag":null,"topics":["bayesian-optimization","cicids2018","cybersecurity","datapreprocessing","deep-learning","docker","hyperparameter-tuning","intrusion-detection","machinelearning","neural-networks"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MohammedSaim-Quadri.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-10-23T10:21:42.000Z","updated_at":"2025-07-04T22:25:29.000Z","dependencies_parsed_at":"2024-10-24T07:10:14.712Z","dependency_job_id":"7948e7db-0b28-43f3-822e-ea5b7d12dbfe","html_url":"https://github.com/MohammedSaim-Quadri/Intrusion_Detection-System","commit_stats":{"total_commits":17,"total_committers":2,"mean_commits":8.5,"dds":0.4117647058823529,"last_synced_commit":"b80e5a6f1c0239464707a1e92af88d211f3fb18b"},"previous_names":["mohammedsaim-quadri/intrusion_detection-system"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/MohammedSaim-Quadri/Intrusion_Detection-System","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedSaim-Quadri%2FIntrusion_Detection-System","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedSaim-Quadri%2FIntrusion_Detection-System/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedSaim-Quadri%2FIntrusion_Detection-System/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedSaim-Quadri%2FIntrusion_Detection-System/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MohammedSaim-Quadri","download_url":"https://codeload.github.com/MohammedSaim-Quadri/Intrusion_Detection-System/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MohammedSaim-Quadri%2FIntrusion_Detection-System/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264396622,"owners_count":23601541,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-optimization","cicids2018","cybersecurity","datapreprocessing","deep-learning","docker","hyperparameter-tuning","intrusion-detection","machinelearning","neural-networks"],"created_at":"2024-10-29T14:04:48.301Z","updated_at":"2025-07-09T05:04:07.965Z","avatar_url":"https://github.com/MohammedSaim-Quadri.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Intrusion Detection System (IDS)\n![Python](https://img.shields.io/badge/python-3.12%2B-blue)  \n![TensorFlow](https://img.shields.io/badge/TensorFlow-2.16.1-orange)  \n![Docker](https://img.shields.io/badge/docker-available-brightgreen)  \n![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg) \n\n---\n\n## 🚀 UPDATE! [July 2025]\n\n📹 **Demo Video: (click the below image)**\n\n[![Watch the Demo](https://img.youtube.com/vi/PJu7FfHhPmQ/0.jpg)](https://youtu.be/PJu7FfHhPmQ)\n\n\n🌐 **Live App:**\n[Try the deployed IDS app here](https://ids-api-frontend.onrender.com)\n\n\u003e We’ve launched the first version of the web-based IDS system using the XGBoost model from Part 1!  \n\u003e This includes fast inference on network flow data, batch upload support, and a clean UI for predictions and visualizations.\n\n\u003e Stay tuned!\n\n---\n\n## Table of Contents\n1. [Overview](#overview)\n2. [Project Structure](#project-structure)\n3. [Features](#key-features)\n4. [Performace](#performance-metrics)\n5. [Installation](#installation)\n6. [Usage](#usage)\n7. [Hyperparameter Tuning](#hyperparameter-tuning)\n8. [Model Evaluation](#model-evaluation)\n9. [Docker Setup](#docker-setup)\n10. [Future Work](#future-work)\n11. [Contributing](#contributing)\n\n\n## Overview\n\nThis project involves developing an Intrusion Detection System (IDS) using machine learning techniques to identify and prevent network intrusions. The final model is an XGBoost classifier trained on the CICIDS2018 dataset. The project incorporates full data pipeline automation — from ingestion to deployment — and is production-ready with Docker support.\n![Architecture Diagram](diagrams/intrusion_detection_system_architecture.png)\n\n## Project Structure\n``` bash\n├── .gitattributes\n├── .gitignore\n├── Dockerfile                 # Docker configuration for containerization\n├── README.md\n├── requirements.txt           # Project dependencies\n├── setup.py                   # Package setup script\n\n├── artifacts/                 # Folder for artifacts like trained models and preprocessed data\n│   ├── IDS_data.csv           # Original dataset\n│   ├── model_trained.keras      # Trained model\n│   ├── model_trained.pkl      # Trained model\n│   ├── preprocessor.pkl       # Data preprocessor\n│   ├── test.csv               # Test dataset\n│   └── train.csv              # Training dataset\n\n├── dataset/                   # Folder containing dataset files\n│   └── train_data.csv         # Raw training data\n\n├── logs/                      # Log files for tracking execution\n\n├── src/                       # Source code for the project\n│   ├── exception.py           # Custom exception handling\n│   ├── logger.py              # Logging module\n│   ├── utils.py               # Utility functions\n\n│   ├── components/            # Folder containing main components\n│   │   ├── data_ingestion.py  # Data ingestion logic\n│   │   ├── data_transformation.py  # Data preprocessing and feature engineering\n│   │   ├── model_trainer.py # Model training and evaluation\n│   │   ├── bayesian_tuner.py  # Bayesian hyperparameter tuning\n│   │   ├── optuna_tuner.py    # Optuna hyperparameter tuning\n│   │   └── __init__.py\n│   └── __init__.py\n```\n\n\n\n## Key Features\n\n- **Data Preprocessing**: \n  - Data ingestion and transformation processes to clean and prepare the CICIDS2018 dataset.\n  - Handling missing values, encoding categorical features, and scaling numerical data.\n  \n- **Model Training**:\n  - Includes neural networks (commented), with final model selected as XGBoost based on performance.\n  - Training and evaluation of the model with performance metrics.\n\n- **Hyperparameter Tuning**:\n  - Utilization of Optuna for optimizing hyperparameters to enhance model performance.\n\n- **Model Evaluation**:\n  - Metrics used include accuracy, precision, recall, F1 score, and ROC AUC.\n\n- **Docker Containerization**:\n  - The project includes a Dockerfile to simplify the deployment of the IDS. This allows the application to run consistently across various environments.\n\n## Performance Metrics\n\nFinal model: XGBoost Classifier\n- Testing Accuracy Score: 89.75%\n- Training Accuracy Score: 89.87%\n- Testing F1 Score: 88.27%\n- Training F1 Score: 88.40%\n- Testing Recall Score: 89.75%\n- Training Recall Score: 89.87%\n- Testing Precision Score: 89.08%\n- Training Precision Score: 89.31%\n- Balanced Accuracy Score: 86.55%\n- ROC AUC (Testing): 99.17%\n- ROC AUC (Training): 99.21%\n\nThese results indicate a well-performing model that generalizes effectively to unseen data, achieving high accuracy and a strong balance between precision and recall.\n\n\n## Installation\n\n1. Clone the repository:\n   ```bash\n   git clone https://github.com/username/IDS.git\n   ```\n\n2. Navigate to the project directory:\n   ```bash\n   cd IDS\n   ```\n\n3. Create and activate a virtual environment (optional but recommended):\n   ```bash\n   python -m venv venv\n   source venv/bin/activate   # For Windows: venv\\Scripts\\activate\n   ```\n\n4. Install required dependencies:\n   ```bash\n   pip install -r requirements.txt\n   ```\n\n## Usage\n\n1. Run the data ingestion pipeline:\n   ```bash\n   python src/components/data_ingestion.py\n   ```\n\n2. (Optional) Perform hyperparameter tuning:\n   ```bash\n   python src/components/bayesian_tuner.py\n   python src/components/optuna_tuner.py\n   ```\n\n3. View logs for detailed execution info:\n   ```bash\n   tail -f logs/*.log\n   ```\n\n## Hyperparameter Tuning\n\nThis project includes two methods for tuning model hyperparameters:\n\n1. **Bayesian Optimization:** This uses probabilistic models to explore the hyperparameter space. Run it using:\n   ```bash\n   python src/components/bayesian_tuner.py\n   ```\n\n2. **Optuna:** A popular framework for efficient hyperparameter optimization. To use Optuna, run:\n   ```bash\n   python src/components/optuna_tuner.py\n   ```\n\nBoth methods aim to improve the model’s accuracy while reducing training time.\n\n## Model Evaluation\n\nAfter training the model, it's evaluated using the following metrics:\n\n- **Accuracy:** Measures the percentage of correct predictions.\n- **Precision \u0026 Recall:** Useful for understanding the trade-off between false positives and false negatives.\n- **F1-score:** A balanced measure between precision and recall.\n\n## Docker Setup\n\nThe Dockerfile provided sets up the environment with TensorFlow and Python 3 support, installs the necessary dependencies, and exposes the required port for monitoring.\n\n#### Option 1: Build the Docker Image Locally\n\n1. **Build the Docker image**:\n\n   ```bash\n   docker build -t ids-system .\n   ```\n2. **Run the Docker container**:\n\n```bash\ndocker run -p 6006:6006 ids-system\n```\nThis exposes port 6006 for TensorBoard or other monitoring tools.\n\n3. **Default Command: The default command in the container is to run the data ingestion script**:\n\n```bash\nCMD [\"python\", \"src/components/data_ingestion.py\"]\n```\nYou can modify the command to run other scripts as needed.\n\n4. **Access Monitoring Tools**: Access TensorBoard or any other monitoring tools at http://localhost:6006.\n\n### Option 2: Pull the Docker Image from Docker Hub\nIf you prefer not to build the image locally, you can directly pull the pre-built Docker image from Docker Hub:\n\n1. **Pull the Docker image**:\n\n```bash\ndocker pull saimquadri/ids-project\n```\n\n2. **Run the Docker container**:\n\n```bash\ndocker run -p 6006:6006 saimquadri/ids-project\n```\nThis will expose port 6006 for monitoring tools like TensorBoard.\n\n## Future Work\n\n- Add support for additional machine learning algorithms.\n- Implement real-time intrusion detection using streaming data.\n- Improve model accuracy with advanced feature engineering techniques.\n- Expand Docker support to Kubernetes for large-scale deployments.\n- Feature selection and ensemble model stacking.\n- Integration with cloud-based dashboards for alerts\n\n## Contributing\n\nWe welcome contributions from the community! Please feel free to fork the repository and submit a pull request with your improvements. For major changes, please open an issue first to discuss what you would like to change.\nTo contribute:\n\n1. Fork the repository.\n2. Create a new branch for your feature:\n   ```bash\n   git checkout -b feature/your-feature\n   ```\n\n3. Make your changes and push to your branch:\n   ```bash\n   git push origin feature/your-feature\n   ```\n\n4. Create a pull request\n\n## License\nThis project is licensed under the MIT License(Modified for Attribution and Non-Commercial Use). See the [LICENSE](./LICENSE) file for details.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmohammedsaim-quadri%2Fintrusion_detection-system","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmohammedsaim-quadri%2Fintrusion_detection-system","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmohammedsaim-quadri%2Fintrusion_detection-system/lists"}