Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ajyounguk/bme280-sensor-logger

A python app, that can read BME280 sensor data attached to a device, and post the data to MongoDB and/or MQTT for Home Assistant Integration. The app can also collect environmental data from the MetOffice API.
https://github.com/ajyounguk/bme280-sensor-logger

bme280 mongo mongodb mqtt python

Last synced: 17 days ago
JSON representation

Host: GitHub
URL: https://github.com/ajyounguk/bme280-sensor-logger
Owner: ajyounguk
Created: 2024-10-18T17:06:39.000Z (21 days ago)
Default Branch: main
Last Pushed: 2024-10-20T15:42:36.000Z (19 days ago)
Last Synced: 2024-10-20T18:00:44.436Z (19 days ago)
Topics: bme280, mongo, mongodb, mqtt, python
Language: Python
Homepage:
Size: 324 KB
Stars: 0
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

# BME280 Temperature Sensor Data Logger (Python)

This Python-based application monitors environmental data using a BME280 sensor and the Met Office API. Temperature, pressure and humdity readings can then be persisted in a Mongo database, or sent to a MQTT for integration with Home Assistant .

## Features

Data collection:
- BM280 Sensor - Captures temperature, humidity, and pressure data from the BME280 sensor. The application captures data from the BME280 sensor at regular intervals defined in the config.
- Met Office API: Fetches external / outdoor weather data to enhance your sensor's readings.

Data persistency:
- MQTT: Publishes data to an MQTT broker for real-time monitoring (e.g., Home Assistant).
- MongoDB: Logs sensor and Met Office data to MongoDB for historical records.

Configuration File:
Configuration parameters in .json file for reading frequency, sensor, MetOffice, Mongo and MQQT parameters.

_{example Chart generated from BME280 data posted to Mongo Atlas and charted with Mongo Charts (all in free tier)_{![MongoChart](/screenshots/mongoChart.png?raw=true)}}

## Todo
- [ ] Post MQTT data in the format for Home Assistant autodiscovery
- [ ] Change MetOffice API from [DataPoint](https://www.metoffice.gov.uk/services/data/datapoint) to [DataHub](https://www.metoffice.gov.uk/services/data/met-office-weather-datahub) as DataPoint is being retired in 2025

## High level code logic:
1. load configuration parameters from /config/config.json
2. main loop
3. get sensor readings
4. post sensor data to MQTT
5. post sensor data to Mongo
6. get MetOffice readings
7. post MetOffice data to MQTT
8. post MetOffice data to Mongo
9. sleep for configurable amount of seconds & loop to 2
>Mongo, MQTT and MetOffice integrations can be enabled/disabled in config

## Application structure

- Main
/main.py = main application

- Python Modules
/src/sensor.py = BME280 integration code
/src/metoffice.py = MetOffice API integration code
/src/mongo.py = Mongo database integration code
/src/mqtt_handler = MQTT handler code (can be used for integration with Home Assistant)

***

## Installation

Ensure that I2C interface is enabled on the Raspberry Pi. Check `raspi-config` (under `Interface Options` > `I2C` = enabled):
```bash
sudo raspi-config
```
- Update OS packages
```bash
sudo apt-get update
```

- Install git
```bash
sudo apt-get install git
```

- Clone the repository
```bash
git clone https://github.com/ajyounguk/bme280-sensor-logger
```

- Install Python
```bash
sudo apt install python3-pip
```

- move to bme280-sensor-logger directory
```bash
cd bme280-sensor-logger
```

- Activate a Python Virtual Environment
install venv if not available
```bash
sudo apt install python3-venv
```

```bash
python3 -m venv venv
```
```bash
source venv/bin/activate
```

- Install Dependencies
```bash
pip install -r requirements.txt
```

***

## Configuration

- Copy the example config file to config.json
```bash
cd config
```
```bash
cp example-config.json config.json
```
- Customise the configuration filewith your MongoDB, MQTT, and Met Office credentials as per below:

Application:
- `readingIntervalSeconds` - Application interval between sensor and MetOffice readings
- `deviceName` - Name for this device, used to tag Mongo and MQTT records

Sensor:
- `i2c_port`: Port number (typically 1).
- `bme280_address`: The default I2C address for the BME280 sensor (varies between 0x76 and 0x77).

MetOffice:
- `enabled`: Enable or disable MetOffice data integration
- `locationID`: Numeric value for the MetOffice location where data is collected (see notes below).
- `APIKey`: Your MetOffice API key (can be obtained [here](https://www.metoffice.gov.uk/services/data/datapoint/api)).

> The `locationID` denotes your weather data location. You may find success using 4-digit location IDs, often representing larger regions or Counties in the UK. See [this list](https://gist.github.com/ajyounguk/e05db10df74e0b86c7e6a0a39a95f1f4) for more.

Mongo:
- `enabled`: Enable or disable MongoDB data integration
- `uri`: Connection string for your Mongo database
- `dbName`: Mongo database name
- `collection`: collection name for storing readings.
- `sensorSource`: source name for the bme280 documents stored in mongo
- `metofficeSource`: source name for the MetOffice documents stored in mongo

MQTT:
- `enabled`: Enable or disable MQTT data integration
- `broker`: IP address or FQDN for the MQTT Broker (For Home Assistant this is the address of your Home Assistant host)
- `port`: 1833 is the default port
- `mqttUsername`: MQTT username (for Home Assistant, use a dedicated user created for MQTT)
- `mqttPassword`: The corresponding MQTT password.
- `sensorTopic`: MQTT topic name to receive the bme280 sensor data
- `metofficeTopic`: MQTT topic name to receive the MetOffice data

Example configuration.json:
```json
{
"app": {
"readingIntervalSeconds": 600,
"deviceName": "lounge_rpi"
},
"sensor": {
"i2c_port": 1,
"bme280_address": "0x76"
},
"metoffice": {
"enabled": true,
"locationID": "3414",
"APIKey": "YOUR_METOFFICE_API_KEY"
},
"mongo": {
"enabled" : true,
"uri": "mongodb+srv://MONGO_USERNAME:[email protected]/DATABASE?retryWrites=true&w=majority",
"dbName": "YOUR_DB_NAME",
"collection": "house_temperature_readings",
"sensorSource": "lounge",
"metofficeSource": "shropshire_outside"
},
"mqtt": {
"enabled": true,
"broker": "192.168.1.250",
"port": 1883,
"mqttUsername": "YOUR_MQTT_USERNAME",
"mqttPassword": "YOUR_MQTT_PASSWORD",
"sensorTopic": "homeassistant/bme280_lounge",
"metofficeTopic": "homeassistant/metoffice"
}
}
```

***
## Run the Application:

```bash
python main.py
```

## Sample Output
```
[INFO 2024-10-19 16:29:47] - Connected to MQTT broker at 192.168.1.250:1883
[INFO 2024-10-19 16:29:47] - I2C Port: 1, BME280 Address: 118
[INFO 2024-10-19 16:29:47] - Reading Interval: 30 seconds
[INFO 2024-10-19 16:29:47] - MongoDB Enabled: True
[INFO 2024-10-19 16:29:47] - Met Office Enabled: True
[INFO 2024-10-19 16:29:47] - MQTT Enabled: True
[INFO 2024-10-19 16:29:47] - BME280 - Temperature: 23.44 °C, Pressure: 1001.85 hPa, Humidity: 47.63 %
[INFO 2024-10-19 16:29:48] - MetOffice - Temperature: 15.7 °C, Humidity: 58.3 %, Pressure: 1011.0 hPa, Wind: 6.0 m/s
[INFO 2024-10-19 16:30:19] - BME280 - Temperature: 23.44 °C, Pressure: 1001.87 hPa, Humidity: 47.45 %
[INFO 2024-10-19 16:30:20] - MetOffice - Temperature: 15.7 °C, Humidity: 58.3 %, Pressure: 1011.0 hPa, Wind: 6.0 m/s
```

***
# Additional Information

#### MongoDB

MongoDB can be hosted anywhere. The app works well with a free-tier Mongo Atlas cloud database. See [MongoDB Atlas](https://cloud.mongodb.com).

- The `sensorSource` configuration is used to populate the `source` field for sensor reading data in Mongo, allowing multiple devices to store readings in the same collection.
- The `metofficeSource` configuration is used to populate the `source` field for MetOffice data documents.
- The MongoDB document structure looks like this:
```json
{
"source": "lounge",
"temperature": 22.5,
"humidity": 55.2,
"pressure": 1012.8,
"wind": 5.6, // Only for Met Office data documents, Null for BME280 readings
"timestamp": "2024-10-19T15:11:21.031+00:00",
}
```
>You can visualize data from different devices using MongoDB Charts. See [MongoDB Charts](https://www.mongodb.com/products/charts).

---
#### MQTT

For debugging purposes in Home Assitant you can use the inbuilt MQTT listener to monitor for messages from the clients by setting it to the relevant MQTT topic name:

> In Home Assistant go to Settings | Devices and Services | MQTT | Configure

![MQTT](/screenshots/MQTT.png?raw=true)

---
#### MetOffice

- MetOffice readings typically only updated once per hour.
- In oder to avoid redundant MetOffice readings, the code checks if a MetOffice document already exists in the Mongo database, if so it will skip creating another.
- THis also ensures that if you are running multiple sensors, only a single MetOffice record is created in each hourly interval.

---
#### BME280 Sensor

- The sensor typically operates on an I2C bus. To ensure proper communication, verify the bus number and address configuration. You can use `i2cdetect` to verify if your sensor is detected correctly.

To detect the port number for the BME280 sensor, use this command (note: the bus port may vary depending on the Pi model):

```bash
# install i2cdetect
sudo apt-get update
sudo apt-get install i2c-tools
```

```bash
i2cdetect -y 1 # for newwer models
i2cdetect -y 0 # for older 256MB Raspberry Pi
```
- In case of errors, double-check the wiring and bus settings. [There is a guide here](https://www.hackster.io/Shilleh/beginner-tutorial-how-to-connect-raspberry-pi-and-bme280-4fdbd5)

---
#### pm2

pm2 is a process manager for keeping applications running in the background (and autostart on boot).
although it is mainly a NodeJS process manager, it works great with Python applications too. If you want to install it try:

- nodejs installation
```bash
sudo apt update
```
```bash
sudo apt install -y nodejs npm
```

- verify nodejs and npm installation
```bash
node -v
```
```bash
npm -v
```

- install pm2 using npm
```bash
sudo npm install -g pm2
```

- verify pm2 nstallation
```bash
pm2 -v
```

- set pm2 to start on boot
```bash
pm2 startup
```

- it will generate a command specific to your system, which might look like this
> sudo env PATH=$PATH:/usr/bin pm2 startup systemd -u pi --hp /home/pi
- run the generated command to enable pm2 to start on boot

- change to application directory

e.g.
```bash
cd bme280-sensor-logger
```
- start the the application (main.py) with pm2
```bash
pm2 start main.py
```

- save it into the pm2 process list
```bash
pm2 save
```

- other pm2 commands (see `pm2 --help`) :

```bash
pm2 status # will display process number for the application if runnign
pm2 logs # view logs
pm2 start 0 # or your process number from status
pm2 stop 0 # or your process number from status
pm2 restart 0 # or your process number from status
```

---
#### Home Assistant Integration

For those using Home Assistant, you can use the MQTT integration to display temperature, pressure, and humidity data from your sensors.

Home assistant integration is beyond the scope of this README, there are many better guides on the web for this.
I recommend starting with the MQTT mosquito add-on for Home Assistant. Then configure the MQTT device integration. A separate credential (Home Assistant person) is recommended to provide access controls for the MQTT integration. Use the same credentials in the MQTT configuration for this application (`mqttUsername` and `mqttPassword`).

Finally you will need to configure your Home Assistant `configuration.yaml` file to listen to the relevant MQTT topics and define the devices/entities for the sensor readings in line with the example below:

```yaml
mqtt:
sensor:
- name: "BME280 Temperature"
state_topic: "homeassistant/bme280_lounge"
unit_of_measurement: "°C"
value_template: "{{ value_json.temperature }}"

- name: "BME280 Pressure"
state_topic: "homeassistant/bme280_lounge"
unit_of_measurement: "hPa"
value_template: "{{ value_json.pressure }}"

- name: "BME280 Humidity"
state_topic: "homeassistant/bme280_lounge"
unit_of_measurement: "%"
value_template: "{{ value_json.humidity }}"

- name: "BME280 Wind Speed"
state_topic: "homeassistant/bme280_lounge"
unit_of_measurement: "m/s"
value_template: "{{ value_json.wind }}"

- name: "Met Office Temperature"
state_topic: "homeassistant/metoffice"
unit_of_measurement: "°C"
value_template: "{{ value_json.temperature }}"

- name: "Met Office Pressure"
state_topic: "homeassistant/metoffice"
unit_of_measurement: "hPa"
value_template: "{{ value_json.pressure }}"

- name: "Met Office Humidity"
state_topic: "homeassistant/metoffice"
unit_of_measurement: "%"
value_template: "{{ value_json.humidity }}"

- name: "Met Office Wind Speed"
state_topic: "homeassistant/metoffice"
unit_of_measurement: "m/s"
value_template: "{{ value_json.wind }}"
```
Home Assistant Screenshots

![HA](/screenshots/HA.png?raw=true)

![HA2](/screenshots/HA2.png?raw=true)