https://github.com/samuelpillai/iot-airquality-pipeline

IoT-based air quality prediction pipeline using Docker, MQTT, RabbitMQ, and ML. Built and deployed across Azure Edge/Cloud VMs using real-time PM2.5 data from Newcastle Urban Observatory.
https://github.com/samuelpillai/iot-airquality-pipeline

air-quality-data azure cloud-computing data-ingestion docker edge-computing emqx forecasting iot matplotlib mqtt-client pandas-dataframe pm25 prophet python rabbitmq-consumer sensor-data time-series-analysis

Last synced: 3 months ago
JSON representation

IoT-based air quality prediction pipeline using Docker, MQTT, RabbitMQ, and ML. Built and deployed across Azure Edge/Cloud VMs using real-time PM2.5 data from Newcastle Urban Observatory.

• Host: GitHub
• URL: https://github.com/samuelpillai/iot-airquality-pipeline
• Owner: Samuelpillai
• License: other
• Created: 2025-06-17T17:37:46.000Z (4 months ago)
• Default Branch: main
• Last Pushed: 2025-06-28T18:35:52.000Z (3 months ago)
• Last Synced: 2025-06-28T19:42:45.818Z (3 months ago)
• Topics: air-quality-data, azure, cloud-computing, data-ingestion, docker, edge-computing, emqx, forecasting, iot, matplotlib, mqtt-client, pandas-dataframe, pm25, prophet, python, rabbitmq-consumer, sensor-data, time-series-analysis
• Language: Python
• Homepage:
• Size: 22.4 MB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# IoT Air Quality Prediction Pipeline

This project demonstrates the design and deployment of a full IoT data processing pipeline across **Azure-based Edge and Cloud VMs** using **Docker, MQTT (EMQX), RabbitMQ**, and a time-series **machine learning predictor** for PM2.5 air quality data.

---

## Features

-  Docker-based MQTT Broker (EMQX)

-  IoT data ingestion from Newcastle Urban Observatory

-  Python-based data preprocessing operator

-  RabbitMQ-based message passing to ML engine

-  Daily averaging & outlier filtering

-  PM2.5 trend forecasting for 15 days using Prophet

-  Matplotlib-based visualizations

---

## Tech Stack

| Component        | Tech Used                            |

|------------------|--------------------------------------|

| Data Source       | Newcastle Urban Observatory API      |

| Data Broker (Edge)| EMQX (Docker)                        |

| Preprocessing     | Python (Paho-MQTT, Pika)             |

| Messaging (Cloud) | RabbitMQ + Docker Compose            |

| ML Engine         | Prophet + Pandas + Matplotlib        |

| Deployment        | Azure Lab (Edge + Cloud Ubuntu VMs)  |

---

## Project Structure

- `docs/`: Screenshots of setup, logs, and results

- `CSC8112-report-samuel-pillai.pdf`: Docs report

- `README.md`: This file

---

## Screenshots

### 🔹 Data Injector Output

![Data Injector](docs/TaskOneSS.png)

### 🔹 Data Injector Confirmation

![Data Injector Log](docs/TaskOne2.png)

### 🔹 Preprocessing Output

![Preprocessing](docs/TaskTwo.png)

### 🔹 RabbitMQ Consumer Waiting

![RabbitMQ Consumer](docs/TaskTwo2.png)

### 🔹 PM2.5 Daily Average Plot

![Averaged PM2.5](docs/TaskThree.png)

### 🔹 PM2.5 Prediction Plot

![Predicted PM2.5](docs/TaskThreePredicted.png)

---

## License

This project is published under the MIT License.

---

## Author

**Samuel Sathiyamoorthy**  

MSc Cloud Computing – Newcastle University  

ssamuelpillai@gmail.com