{"id":22691532,"url":"https://github.com/rpj/rpi","last_synced_at":"2025-04-12T23:10:21.262Z","repository":{"id":43250760,"uuid":"147305598","full_name":"rpj/rpi","owner":"rpj","description":"RPJiOS: RPJ's RPi OS, a sensor data platform for the Raspberry Pi built with python2.7 and redis.","archived":false,"fork":false,"pushed_at":"2023-03-31T14:31:25.000Z","size":73,"stargazers_count":25,"open_issues_count":16,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-12T23:10:06.109Z","etag":null,"topics":["data-pipeline","data-platform","data-processing","data-stream","garden-bots","python","raspberry-pi","redis","rpi","sensor","sensors"],"latest_commit_sha":null,"homepage":"https://rpjios.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rpj.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-04T07:27:18.000Z","updated_at":"2023-04-01T13:57:23.000Z","dependencies_parsed_at":"2024-12-10T01:11:32.697Z","dependency_job_id":"ec409968-96e7-4279-8c0b-e1258cb51af4","html_url":"https://github.com/rpj/rpi","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rpj%2Frpi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rpj%2Frpi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rpj%2Frpi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rpj%2Frpi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rpj","download_url":"https://codeload.github.com/rpj/rpi/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248643004,"owners_count":21138355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-pipeline","data-platform","data-processing","data-stream","garden-bots","python","raspberry-pi","redis","rpi","sensor","sensors"],"created_at":"2024-12-10T01:11:29.729Z","updated_at":"2025-04-12T23:10:21.243Z","avatar_url":"https://github.com/rpj.png","language":"Python","readme":"# RPJiOS\n\n[![Raspberry Pi Zero W](https://www.vectorlogo.zone/logos/raspberrypi/raspberrypi-ar21.svg)](https://www.raspberrypi.org/)\n[![Python](https://www.vectorlogo.zone/logos/python/python-ar21.svg)](https://www.python.org/)\n[![Redis](https://www.vectorlogo.zone/logos/redis/redis-ar21.svg)](https://redis.io/)\n[![Debian](https://www.vectorlogo.zone/logos/debian/debian-ar21.svg)](https://www.raspbian.org/)\n\nA [pub/sub](https://en.wikipedia.org/wiki/Publish–subscribe_pattern)-based implementation of a \nRaspberry Pi data pipeline built on [redis](https://redis.io) and centered around sensors.\n\nThe general philosophy is that a \"sensor\" is any entity (physical or not) that operates primarily\nin an output-only mode (configuration doesn't necessarily count as an \"input\" so is allowable).\n\nThese outputs are treated ephemerally, in a \"fire and forget\" manner, creating a data stream.\nThe intent is for interested entities to subscribe to the data stream and transform, interpret \nand/or persist it according to their requirements.\n\n## Caveat emptor\n\nThis is still very much an active work-in-progress! However, as it is functional and actively deployed I figured it was worth making public in the event it might help others in their projects.\n\nThose current deployments consist of my [atmospheric particulate matter sensor](https://www.hackster.io/rpj/atmospheric-particulate-matter-environmental-sensing-fb31a1) and my garden monitoring bots, with the entire system handling (on average) about a half-million units of sensor data per day.\n\n## Requirements\n\n* Hardware:\n\t* a Raspberry Pi running a recent Raspbian build (for sensor nodes)\n\t* any Debian-like system running apt (for managment/non-sensor nodes)\n* some [sensors](#sensors), configured appropriately (most easily done with `raspi-config`):\n\t* depending on your chosen sensors: I2C enabled, SPI enabled, 1-wire enabled\n\t* other sensors (LM335, Soil, TEPT5700) require an [external ADC](#devices) which itself will require SPI\n\n## Setup\n\n* Clone the repo\n* `cd` into repo dir\n* `./setup.sh` (you might need to enter your `sudo` password to install requirements)\n* `source env/bin/activate` and go!\n\n## Tools\n\n* [sensors-src](bin/sensors-src): source daemon that manages all specified sensors and publishes their data as configured\n* [downsample](bin/downsample): a very flexible data stream downsampler/forwarder/transformer. example uses:\n\t* an at-frequency forwarder (set `-r 1`)\n\t* a loopback downsampler (set `-o` to the same as `-i`)\n\t* a many-to-one reducer (set `-t` to `key`)\n\t* a one-to-many exploder (set `-m` to `flatten:[options]`)\n\t* a bounded ephemeral cache (set `-t` to `list:[options]`, where the `limit=X` option sets the bound)\n\t* ...\n\t* profit!\n* [sqlite-sink](bin/sqlite-sink): an [SQLite](https://www.sqlite.org) sink for data streams. examples:\n\t* sink to an SQLite database on a different host a downsampled data stream:\n\t\t1. on the source device:\n\t\t\t* `downsample -i redis://localhost -o redis://sql-db-host -r ... -p ...`\n\t\t2. on the sink host \"`sql-db-host`\":\n\t\t\t* `sqlite-sink path-to-db.sqlite3`\n\t\t\t\t* (lots of \"TODOs\" here, obviously) \n* [oled-display](bin/oled-display): an [OLED display](https://www.adafruit.com/product/661) driver for consuming \u0026 display some sensor data, among other things\n* [thingspeak](bin/thingspeak): a simple example of a [ThingSpeak](http://thingspeak.com) data forwarder for the SPS30 particulate matter sensor data. [Example resulting data set](https://thingspeak.com/channels/655525).\n\n## Library\n\n### Sensors\n\nThe following are currently supported (with the required drivers / interfaces setup of course):\n\n* [SPS30](https://www.sensirion.com/en/environmental-sensors/particulate-matter-sensors-pm25/) I2C particulate matter air quality sensor. [Source.](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/SPS30.py)\n* [BME680](https://cdn-shop.adafruit.com/product-files/3660/BME680.pdf) temperature / humidity / barometeric pressure / volatile organic compound I2C sensor. My setup uses [AdaFruit's awesome breakout board](https://www.adafruit.com/product/3660) for ease-of-integration. [Source.](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/BME680.py)\n* [BME280](https://www.bosch-sensortec.com/bst/products/all_products/bme280) temperature / humidity / barometeric pressure I2C sensor. [Source.](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/BME280.py)\n* [DHTXX](https://www.mouser.com/ds/2/737/dht-932870.pdf)(DHT11/DHT22) temperature / humidity sensors. I use [DFRobot's](https://www.dfrobot.com/product-1102.html) [breakouts](https://www.dfrobot.com/product-174.html) but you can find similar breakout's from many (re)sellers online. [Source.](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/DHTXX.py)\n* [DS18S20](https://datasheets.maximintegrated.com/en/ds/DS18S20.pdf) high-precision 1-wire temperature sensor. [Source.](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/DS18S20.py)\n* [LM335](http://www.ti.com/lit/ds/symlink/lm335.pdf) precision analog temperature sensor. [Source.](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/LM335.py)\n* Capacative soil moisture sensors such as the DFRobot [SEN0114](https://www.dfrobot.com/product-599.html) or any [simple-to-build capactive analog moisture sensor](http://gardenbot.org/howTo/soilMoisture/). [Source.](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/Soil.py)\n* [TEPT5700](https://www.vishay.com/docs/81321/tept5700.pdf) ambient light sensor. [Source.](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/TEPT5700.py)\n* \"Virtual\" sensors such as [SysInfo](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/SysInfo.py) and [NetInfo](https://github.com/rpj/rpi/blob/master/lib/rpjios/sensors/NetInfo.py) that do not require any additional hardware.\n\nVariants of these sensors could likely be made to work with this system via simple modifications if any are required at all.\n\n### Devices\n\n* A simple Python wrapper driver for the SPS30 sensor is included [here](https://github.com/rpj/rpi/blob/master/lib/rpjios/devices/SPS30.py), wrapping the included [embedded-sps](https://github.com/rpj/embedded-sps/tree/1aabaead20059262d66e113d511157c6fda4133a) I2C driver from Sensiron implemented in C (forked to include a shared object build step for Python consumption). \n* A simple Python driver for the [74HC595](http://www.ti.com/lit/ds/symlink/sn74hc595.pdf) 8-bit shift register is included [here](https://github.com/rpj/rpi/blob/master/lib/rpjios/devices/74HC595.py).\n* The venerable [MCP3008](http://ww1.microchip.com/downloads/en/devicedoc/21295c.pdf) 10-bit 8-channel analog-to-digital converter is directly supported by the [base analog sensor implementation](https://github.com/rpj/rpi/blob/master/lib/rpjios/AnalogBase.py). Other ADCs would be quite simple to adapt (even more so if I implemented a HAL... #TODO).\n\t* channel (zero-indexed) is specified in [`config.json`](https://github.com/rpj/rpi/blob/master/config.json#L62)\n\n## Related\n\n* [The old now-archived repository](https://github.com/rpj/rpi.archive) from whence this all came\n\t* might still contain some useful stuff: D3-based \"live\" analog data plotting code in `rpjctrl`, and I'm still using `ledCount.py` on some of my units because I'm too lazy to re-write it properly\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frpj%2Frpi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frpj%2Frpi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frpj%2Frpi/lists"}