https://github.com/alexisbalayre/iot-environmental-monitoring
Cloud Computing and IoT for Environmental Monitoring Project
https://github.com/alexisbalayre/iot-environmental-monitoring
apache-spark aws cloud-computing grafana iot timestream
Last synced: 8 months ago
JSON representation
Cloud Computing and IoT for Environmental Monitoring Project
- Host: GitHub
- URL: https://github.com/alexisbalayre/iot-environmental-monitoring
- Owner: AlexisBalayre
- Created: 2023-11-28T10:29:49.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-01-12T10:06:23.000Z (almost 2 years ago)
- Last Synced: 2025-01-23T09:35:37.283Z (9 months ago)
- Topics: apache-spark, aws, cloud-computing, grafana, iot, timestream
- Language: Jupyter Notebook
- Homepage:
- Size: 66.3 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Cloud Computing and IoT for Environmental Monitoring
## Overview 🌍
This project integrates Cloud Computing with the Internet of Things (IoT) for comprehensive environmental monitoring, with a focus on air quality analysis. Utilising Apache Spark and Amazon Timestream, it manages large volumes of data from IoT sensors to calculate the Air Quality Index (AQI) accurately, offering insights into environmental conditions. The file `report.pdf` provides a detailed overview of the project.
## Key Features 🌟
- **Real-time Data Collection**: IoT sensors gather environmental data continuously.
- **Efficient Data Processing**: Leverages Apache Spark for effective data handling.
- **Robust Data Storage**: Uses Amazon Timestream for optimised time-series data management.
- **Dynamic Data Visualisation**: Features a Grafana dashboard for interactive and real-time data insights.
## Pipeline Architecture 🏗️
## Grafana Dashboard 📊
## Getting Started 🚀
### Installation
1. **Clone the repository**:
```bash
git clone git@github.com:AlexisBalayre/environmental-monitoring-project.git
```
2. **Navigate to the project directory**:
```bash
cd environmental-monitoring-project
```
3. **Set up a virtual environment**:
```bash
python3 -m venv venv
```
4. **Activate the virtual environment**:
- For Windows:
```bash
.\venv\Scripts\activate
```
- For Unix or MacOS:
```bash
source venv/bin/activate
```
5. **Install dependencies**:
```bash
pip install -r requirements.txt
```
### Usage
- **Start the system**: Run `main.py` to begin data collection and processing.
```bash
python main.py
```
- **Visualise the data**: Access the Grafana dashboard for real-time data analysis and visualisations.
## Project Structure 📂
- `lib/`: Core library modules for data collection, processing, and storage.
- `scripts/`: Scripts for IAM credentials retrieval and Spark job initiation.
- `services/`: Service configurations for IAM and Spark.
- `test/`: Testing scripts and visualisation tools.
- `main.py`: Main executable script.
- `requirements.txt`: Project dependencies.
## Testing 🧪
The project includes comprehensive testing:
- Load testing configurations and results.
- Unit testing for data collection, processing, and storage.
- Visualisation tools for data analysis.
## Dependencies 🛠️
- Apache Spark
- Python 3.x
- Amazon Timestream
- Grafana