{"id":17967671,"url":"https://github.com/simonprickett/redis-pi-camera","last_synced_at":"2025-03-25T08:31:24.234Z","repository":{"id":157433311,"uuid":"629485276","full_name":"simonprickett/redis-pi-camera","owner":"simonprickett","description":"Experiments with the Raspberry Pi Camera module, Redis and Flask","archived":false,"fork":false,"pushed_at":"2023-07-04T16:23:13.000Z","size":1398,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-19T23:52:46.856Z","etag":null,"topics":["flask","picamera2","python","raspberry-pi","redis","redis-py"],"latest_commit_sha":null,"homepage":"https://simonprickett.dev/taking-pictures-with-raspberry-pi-and-redis/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simonprickett.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-18T12:11:44.000Z","updated_at":"2024-05-22T04:20:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"5b2e26f1-0cfc-4e93-afb0-43b516ac74c5","html_url":"https://github.com/simonprickett/redis-pi-camera","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonprickett%2Fredis-pi-camera","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonprickett%2Fredis-pi-camera/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonprickett%2Fredis-pi-camera/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonprickett%2Fredis-pi-camera/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simonprickett","download_url":"https://codeload.github.com/simonprickett/redis-pi-camera/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245426263,"owners_count":20613321,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flask","picamera2","python","raspberry-pi","redis","redis-py"],"created_at":"2024-10-29T14:09:15.530Z","updated_at":"2025-03-25T08:31:24.228Z","avatar_url":"https://github.com/simonprickett.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Experiments with Redis and the Raspberry Pi Camera Module\n\n## Overview \n\nThis repository contains an example application that demonstrates how to capture images from a Raspberry Pi using the camera module, store them in [Redis Hashes](https://redis.io/docs/data-types/hashes/) and render them in a web application.  \n\nThe project started by using an older Raspberry Pi camera module and the Redis `SCAN` command to retrieve all of the data and present it in the front end.  [Release v0.0.1](https://github.com/simonprickett/redis-pi-camera/releases/tag/v0.0.1) has that code.\n\nI enhanced the code to use the newer Raspberry Pi camera module with autofocus and also added configurable image expiry time in Redis and capture of Lux values from the camera.  Check out [release v0.0.2](https://github.com/simonprickett/redis-pi-camera/releases/tag/v0.0.2) for that.\n\nIn my second live stream for this project, I replaced the `SCAN` command with the [Search capability of Redis Stack](https://redis.io/docs/stack/search/).  [Release 0.0.3](https://github.com/simonprickett/redis-pi-camera/releases/tag/v0.0.3) contains the code from that stream.\n\nHere's what the front end looks like when a few images have been captured by the Raspberry Pi...\n\n![Front end showing captured images](server_component_running.png)\n\nAnd here's a Raspberry Pi with a camera attached:\n\n![Raspberry Pi 3 with Camera Module attached](raspberry_pi_3_with_camera_module.jpg)\n\n## Watch the Videos\n\nWatch the first video for a full walkthrough of this project.  [Watch on YouTube](https://www.youtube.com/watch?v=OTDZIK55DX0).\n\nIn the second episode I started to use Redis Stack's Search capability.  [Watch on YouTube](https://www.youtube.com/watch?v=mcGL6Lk2IXU).\n\nNot sure what I'll do yet, but I've scheduled a third episode for June 1st.  Maybe add a trigger sensor and some more search functionality in the front end?  [Watch on YouTube](https://www.youtube.com/watch?v=qci-pz8e5nk).\n\n\n## Components of the Demo\n\nThis repository contains two components:\n\n* **[Image capture component](pi/):** This is written in Python and runs on a Raspberry Pi with a camera module attached.  It takes a new picture every so often and stores it, plus some basic metadata, in Redis.  Python was the obvious implementation choice here as Raspberry Pi provides an excellent Python library for their camera modules. See the `pi` folder for full details.\n* **[Front end component](server/):** This comprises a server written in Python using the Flask framework.  It reads data from Redis and uses a HTML front end to display it in a grid view.  It uses HTML, JavaScript and the Bulma framework.  These were all arbitrary choices - this component could equally be built with Node.js and Express for example.  See the `server` folder for full details.\n\nDetails of how each component works including how to configure and run it can be found in the README files in each of the above folders.\n\n## Redis\n\nBoth components need to be connected to the same Redis Stack instance to talk to each other. \n\nIf you want to use Docker, start Redis Stack like this:\n\n```\ndocker-compose up -d\n```\n\nWhen you're done with the Docker container, stop it like this:\n\n```\ndocker-compose down\n```\n\nYour data is saved in a [Redis Append Only File](https://redis.io/docs/management/persistence/) in the `redisdata` folder.  Redis Stack will reload the dataset from this file when you restart the container.\n\nWith the container running, you can access the [Redis CLI](https://redis.io/docs/ui/cli/) using this command:\n\n```\ndocker exec -it redispiimages redis-cli\n```\n\nSee the RedisInsight section of this document if you're interested in a graphical alternative to the Redis CLI interface.\n\nYou'll need to make sure that both components of the application can connect to your Redis instance.  See details in each component's README.\n\nThis project will also work with a free Redis Stack cloud instance from Redis (the company).  To use this, [sign up here](https://redis.com/try-free/) and make a note of your Redis host, port and password.  You'll need those to configure each component.  When using the free cloud instance, note that you get 30Mb space which will fill up with images quickly.  You can manage this by setting a shorter image expiry time in the capture component's configuration.\n\n## The Redis Data Model, Key Naming Strategy and Indexing\n\n### Data Type and Key Naming Strategy\n\nThe application stores each image plus associated metadata in its own [Redis Hash](https://redis.io/docs/data-types/hashes/).  A Hash in Redis can be thought of as a flat map of name/value properties.  Each Hash is stored in its own key in Redis.\n\nThe key naming strategy is as follows.  Each Hash's key has a fixed prefix `image:` followed by the [UNIX timestamp](https://en.wikipedia.org/wiki/Unix_time) representing the time that the image was captured.\n\nFor example the key `image:1681844748` would contain a Hash with data for an image taken on Tuesday 18th April 2023 at 7:05:48pm GMT.\n\nEach key contains a hash with the following name/value pairs:\n\n* `mime_type`: The [MIME type](https://en.wikipedia.org/wiki/Media_type) for the captured image data.  This will always be `image/jpeg` unless you change it and the image capture format in the `capture.py` script.\n* `timestamp`: The [UNIX timestamp](https://en.wikipedia.org/wiki/Unix_time) that the image was captured at, as recoded from the Raspberry Pi's clock.  This will be the same value as the timestamp in the key name.\n* `lux`: The [lux](https://en.wikipedia.org/wiki/Lux) value captured by the camera when the image was taken.\n* `image_data`: A binary representation of the bytes for the image captured by the camera.  This will be a [JPEG image](https://en.wikipedia.org/wiki/JPEG) unless you change the capture format in `capture.py`.\n\nHere's a complete example, with the image data truncated for brevity:\n\n```\n\u003e HGETALL image:1681843615\n1) \"mime_type\"\n2) \"image/jpeg\"\n3) \"timestamp\"\n4) \"lux\"\n5) \"268\"\n6) \"1681843615\"\n7) \"image_data\"\n8) \"\\xff\\xd8\\xff\\xe0\\x00\\x10JFIF\\x00\\x01\\x01\\x00\\x00...\n```\nWith the camera that I used ([Raspberry Pi Camera Module 3](https://www.raspberrypi.com/products/camera-module-3/) capturing at 4608x2592 pixels - configurable in `capture.py`) you can expect each Hash to require around 1Mb of RAM in Redis.\n\n### Indexing and Querying\n\nAs we saw in the initial live stream video, using `SCAN` allows us to retrieve all of the keys containing image data for display in the front end.  This has a couple of problems:\n\n* We can't do any meaningful filtering or searching on the server side in Redis.\n* `SCAN` is effectively a O(n) time complexity command, so the bigger the dataset the longer it's going to take / the more load it will put on the Redis server.\n\nThe Search capability of Redis Stack addresses both of these and gives us a flexible way to index and query our data.  To use it, we first have to create and index with the [`FT.CREATE` command](https://redis.io/commands/ft.create/).  Once created, Redis Stack will monitor changes to keys that match the index criteria and update the index automatically.  We can then write rich queries using the `FT.SEARCH` command.\n\nHere's our index creation command - you'll need to paste this into Redis CLI or RedisInsight and execute it before running the front end:\n\n```\nFT.CREATE idx:images ON HASH PREFIX 1 image: SCHEMA mime_type TAG lux NUMERIC SORTABLE timestamp NUMERIC SORTABLE\n```\n\nThis command creates an index on keys in Redis whose key names begin `image:` and which are of type `HASH`.  Where found, the values of fields named `timestamp` and `lux` are indexed as numeric values and the values of fields named `mime_type` are indexed as tags (exact match string values).\n\nHere are some example queries that we can run against this index.\n\nFind the 9 most recent images (most recent first), returning their timestamp, MIME type and lux values:\n\n```\nFT.SEARCH idx:images \"*\" RETURN 3 timestamp mime_type lux SORTBY timestamp DESC LIMIT 0 9\n```\n\nTruncated example response:\n\n```\n1) \"12\"\n2) \"image:1684427475\"\n3) 1) \"timestamp\"\n   2) \"1684427475\"\n   3) \"mime_type\"\n   4) \"image/jpeg\"\n   5) \"lux\"\n   6) \"85\"\n4) \"image:1684427251\"\n5) 1) \"timestamp\"\n   2) \"1684427251\"\n   3) \"mime_type\"\n   4) \"image/jpeg\"\n   5) \"lux\"\n   6) \"104\"\n6) \"image:1684427190\"\n7) 1) \"timestamp\"\n   2) \"1684427190\"\n   3) \"mime_type\"\n   4) \"image/jpeg\"\n   5) \"lux\"\n   6) \"109\"\n8) \"image:1684427130\"\n9) 1) \"timestamp\"\n   2) \"1684427130\"\n   3) \"mime_type\"\n   4) \"image/jpeg\"\n   5) \"lux\"\n   6) \"93\"\n...\n```\n\nFind the 9 most recent images with lux value between 100 and 120, again with the most recent image first:\n\n```\nFT.SEARCH idx:images \"@lux:[100 120]\" RETURN 3 timestamp mime_type lux SORTBY timestamp DESC LIMIT 0 9\n```\n\nFor more information on how to write search queries, check out:\n\n* [Search query syntax documentation](https://redis.io/docs/stack/search/reference/query_syntax/).\n* [RU203 - Querying, Indexing and Full-Text Search](https://university.redis.com/courses/ru203/): A free course at Redis University.\n\n## (Optional, but Recommended): RedisInsight\n\nRedisInsight is a free graphical management and database browsing tool for Redis. You don't need it to look at how the application stores data in Redis (you can use redis-cli if you prefer) but I'd recommend it as it's much easier to get an overall picture of the state of the database with a graphical tool.  RedisInsight runs as a desktop application on your Mac, Windows or Linux machine.\n\n[Download RedisInsight here](https://redis.io/docs/ui/insight/).\n\nIf you're using the Docker compose file provided with this project to run Redis Stack, you can also access a web based version of RedisInsight with no additional software to install.  With the Docker container running, navigate to `http://localhost:8001` to use the web version.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonprickett%2Fredis-pi-camera","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonprickett%2Fredis-pi-camera","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonprickett%2Fredis-pi-camera/lists"}