{"id":19277470,"url":"https://github.com/karimosman89/iot-predictive-maintenance","last_synced_at":"2026-04-29T09:05:42.205Z","repository":{"id":261756008,"uuid":"885240416","full_name":"karimosman89/iot-predictive-maintenance","owner":"karimosman89","description":"This repository will simulate an IoT-based predictive maintenance system designed to monitor industrial equipment through sensors. It will include data ingestion, processing, and machine learning components to predict potential failures, optimizing maintenance schedules and reducing downtime.","archived":false,"fork":false,"pushed_at":"2024-11-08T09:52:58.000Z","size":39,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-17T10:10:03.008Z","etag":null,"topics":["api","cloud-platform","dashboard","data-collection","data-processing","deployment","iot-platform","predictive-analytics","pressure-sensor","real-time-sensor","sensors","spark","temperature-sensor","vibration"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/karimosman89.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-08T08:18:45.000Z","updated_at":"2025-06-07T15:19:39.000Z","dependencies_parsed_at":"2024-11-08T09:25:24.562Z","dependency_job_id":"1ff55102-5175-4485-a60d-3ce37207596a","html_url":"https://github.com/karimosman89/iot-predictive-maintenance","commit_stats":null,"previous_names":["karimosman89/iot-predictive-maintenance"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/karimosman89/iot-predictive-maintenance","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karimosman89%2Fiot-predictive-maintenance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karimosman89%2Fiot-predictive-maintenance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karimosman89%2Fiot-predictive-maintenance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karimosman89%2Fiot-predictive-maintenance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/karimosman89","download_url":"https://codeload.github.com/karimosman89/iot-predictive-maintenance/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karimosman89%2Fiot-predictive-maintenance/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260336343,"owners_count":22993740,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","cloud-platform","dashboard","data-collection","data-processing","deployment","iot-platform","predictive-analytics","pressure-sensor","real-time-sensor","sensors","spark","temperature-sensor","vibration"],"created_at":"2024-11-09T21:05:47.825Z","updated_at":"2026-04-29T09:05:42.143Z","avatar_url":"https://github.com/karimosman89.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IoT Predictive Maintenance System\n\nThis repository implements a comprehensive IoT Predictive Maintenance System using Kafka for real-time data streaming, Spark for data processing, and AWS IoT for cloud connectivity. The project includes machine learning models for failure prediction, using MLflow to track experiments and manage model versions.\n\n## Table of Contents\n- [Overview](#overview)\n- [Key Components](#key-components)\n  - [AWS IoT Integration](#aws-iot-integration)\n  - [Kafka Streaming](#kafka-streaming)\n  - [Data Preprocessing](#data-preprocessing)\n  - [Machine Learning with MLflow](#machine-learning-with-mlflow)\n- [Setup Instructions](#setup-instructions)\n  - [Requirements](#requirements)\n  - [Configuration](#configuration)\n- [Usage](#usage)\n- [Contributing](#contributing)\n- [License](#license)\n\n---\n\n## Overview\n\nThis IoT Predictive Maintenance System is designed to monitor sensor data from connected devices, predict potential failures, and alert users before a breakdown occurs. The pipeline:\n1. Simulates sensor data and streams it to Kafka.\n2. Processes data in real-time with Spark.\n3. Connects to AWS IoT Core for device management.\n4. Uses machine learning models for predictive analytics, tracking experiments and versions with MLflow.\n\n\n## Key Components\n\n### AWS IoT Integration\n\nThe AWS IoT Core configuration (aws_iot_config.json) enables cloud connectivity for managing IoT devices and sending/receiving data securely. This part supports real-time data management and monitoring through AWS's secure infrastructure.\n\n**Kafka Streaming**\nThe kafka_producer.py script generates and streams simulated sensor data (e.g., temperature, vibration, pressure) to a Kafka topic. Kafka serves as the data backbone for processing and analytics, enabling scalable and real-time data handling.\n\n### Data Preprocessing\n\nThe preprocess_data.py script cleans and prepares raw data, making it suitable for model training. Preprocessing steps include:\n\n**Missing value handling:** Removes or imputes missing data.\n**Scaling:** Standardizes data for consistent model performance.\n**Feature engineering:** Adds or transforms features based on analysis requirements.\n\n### Machine Learning with MLflow\nThe mlflow_experiment.py script handles model training, tracking, and version control with MLflow. Key features include:\n\n**Experiment logging:** Tracks model parameters, metrics, and training details.\n**Model registry:** Saves models in MLflow for easy retrieval and deployment.\n**Hyperparameter management:** Uses configurations from model_config.yaml for model consistency and reproducibility.\n\n## Setup Instructions\n\n### Requirements\nEnsure you have the following installed:\n\n**Python 3.8+**\n**Kafka (locally or a Kafka cluster)**\n**Spark for distributed processing**\n**MLflow for experiment tracking**\n\n### Configuration\n\n1. Clone the Repository:\n\n\n           git clone https://github.com/karimosman89/iot-predictive-maintenance.git\n   \n           cd iot-predictive-maintenance\n\n3. Install Dependencies:\n\n\n             pip install -r requirements.txt\n\n4. Configure Services:\n\nUpdate the aws_iot_config.json file with AWS IoT Core details.\nModify the configs/kafka_config.yaml and configs/spark_config.yaml to match your Kafka and Spark environments.\nSet model parameters in configs/model_config.yaml for training and evaluation.\n\n\n## Usage\n1. Start Kafka Producer: Run the kafka_producer.py script to start streaming sensor data to Kafka.\n\n\n             python scripts/kafka_producer.py\n\n2. Preprocess Data: After gathering sufficient data, preprocess it for model training.\n\n\n            python scripts/preprocess_data.py\n\n3. Run MLflow Experiment: Use the mlflow_experiment.py script to train and log the model.\n\n\n            python scripts/mlflow_experiment.py\n\nCheck your MLflow UI at http://localhost:5000 (or configured port) to view experiment results.\n\n## Contributing\n\nWe welcome contributions to enhance the IoT Predictive Maintenance System. Please fork the repository and create a pull request with any additions or improvements.\n\n\n\n## Acknowledgements\n\nThis project utilizes open-source libraries and tools, including Kafka, Spark, and MLflow. We appreciate the contributions of the open-source community in making such tools accessible.\n\n\n\n\n---\n\n### **Explanation of Each Section**\n\n- **Overview**: Briefly explains the purpose and functionality of the project.\n- **Project Structure**: Outlines the files and folders, helping users navigate and understand the code organization.\n- **Key Components**: Describes each module, such as AWS IoT, Kafka, data preprocessing, and MLflow integration.\n- **Setup Instructions**: Guides users through cloning the repository, installing dependencies, and configuring each component.\n- **Usage**: Provides a step-by-step walkthrough on running each part of the project.\n- **Contributing**: Invites others to contribute, making it clear how they can get involved.\n- **License**: States the license under which the code is shared.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarimosman89%2Fiot-predictive-maintenance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkarimosman89%2Fiot-predictive-maintenance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarimosman89%2Fiot-predictive-maintenance/lists"}