https://github.com/pranaypkadu/networksecurity

End To End MLOPS Project With ETL Pipelines- Building Network Security System
https://github.com/pranaypkadu/networksecurity

aws-ec2 aws-ecr aws-s3 dagshub docker etl-pipelines fastapi github-actions mlflow mlops mongodb-atlas network-security numpy pandas python pytorch scikit-learn tensorflow vscode

Last synced: over 1 year ago
JSON representation

End To End MLOPS Project With ETL Pipelines- Building Network Security System

• Host: GitHub
• URL: https://github.com/pranaypkadu/networksecurity
• Owner: pranaypkadu
• Created: 2024-10-20T10:57:42.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2025-01-13T16:08:46.000Z (over 1 year ago)
• Last Synced: 2025-01-25T01:26:49.195Z (over 1 year ago)
• Topics: aws-ec2, aws-ecr, aws-s3, dagshub, docker, etl-pipelines, fastapi, github-actions, mlflow, mlops, mongodb-atlas, network-security, numpy, pandas, python, pytorch, scikit-learn, tensorflow, vscode
• Language: Python
• Homepage:
• Size: 18.6 MB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
**Explore the Readme Folder for an in-depth overview.**

**End-to-End MLOps Project with ETL Pipelines - Building a Network Security System**

**Technology Stack Utilized for the MLOps Network Security System Project**

**Development Environment:**

- IDE: Visual Studio Code

- Version Control: GitHub

- Packaging: Python setup.py

**Backend Technologies:**

- Programming Language: Python

- Database: MongoDB Atlas

- Cloud Platform: AWS (EC2, S3, ECR)

**MLOps and Machine Learning Stack:**

- ML Framework: TensorFlow or PyTorch (used for model training)

- Experiment Tracking: MLflow

- Remote Experiment Repository: DagsHub

- Hyperparameter Tuning: Scikit-learn or Optuna

**Data Engineering:**

- ETL Pipeline: Python-based data processing

- Data Validation: Custom validation components

- Data Transformation: Pandas and NumPy

**DevOps and Deployment:**

- Containerization: Docker

- CI/CD: GitHub Actions

- Deployment: AWS EC2 instance

- Container Registry: AWS ECR

**Monitoring and Logging:**

- Logging: Custom logging implementation

- Exception Handling: Custom error management

**Key Project Components:**

- Network security system

- Machine learning model training

- Batch prediction pipeline

- Model artifact management

3.Reference : Krish Naik https://www.udemy.com/course/complete-mlops-bootcamp-with-10-end-to-end-ml-projects/