{"id":24429219,"url":"https://github.com/peterk/redis-queue","last_synced_at":"2025-08-01T14:12:35.487Z","repository":{"id":66470431,"uuid":"581897196","full_name":"peterk/redis-queue","owner":"peterk","description":"Minimal set up of a queue using Docker compose / Redis / Python","archived":false,"fork":false,"pushed_at":"2022-12-25T08:18:34.000Z","size":7,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-20T13:35:34.206Z","etag":null,"topics":["docker","python","queue","queue-workers","redis"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/peterk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-24T19:17:55.000Z","updated_at":"2022-12-24T19:27:00.000Z","dependencies_parsed_at":"2023-02-25T02:00:32.939Z","dependency_job_id":null,"html_url":"https://github.com/peterk/redis-queue","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterk%2Fredis-queue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterk%2Fredis-queue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterk%2Fredis-queue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/peterk%2Fredis-queue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/peterk","download_url":"https://codeload.github.com/peterk/redis-queue/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243456567,"owners_count":20293905,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","python","queue","queue-workers","redis"],"created_at":"2025-01-20T13:33:52.599Z","updated_at":"2025-03-13T18:16:22.076Z","avatar_url":"https://github.com/peterk.png","language":"Python","readme":"# Minimal set up of a queue using Docker compose / Redis / Python\n\nThis is an example setup of a docker compose solution where a frontend (web) adds jobs to a queue and multiple workers (worker) pick jobs and executes them. I could not find a similar solution that did not rely on a specific library. This solution only uses the standard redis library and msgpack (for passing job data to workers).\n\nThis solution may be useful if you have long running jobs and want to use Redis as a simple queue. If the [Redis](https://redis.io) connection is lost the worker will try to reconnect. \n\nIf you have any suggestions for improvement, please add an issue.\n\n## Run this example\n\nFirst, clone this repository:\n\n```bash\ngit clone --depth=1 https://github.com/peterk/redis-queue\n```\n\nEnter the folder and create a .env file by copying the example file:\n```bash\ncd redis-queue\ncp .env_example .env\n```\n\nBuild and start the system with two workers:\n```bash\ndocker compose build\ndocker compose up --scale worker=2\n```\n\nOpen http://127.0.0.1:8080 to add jobs. Look at the log to see what happens.\n\nAfter adding a couple of long running jobs (increase the number of seconds), try stopping the Redis server from a different terminal:\n```bash\ndocker compose stop redis\n```\n\nThe workers should start trying to reconnect. Start the Redis server to see jobs getting worked on again.\n```bash\ndocker compose start redis\n```\n\n## Notes\n\n1. Please note the default information in the .env file. Not suitable for production.\n2. Update versions of all libraries/images in requirements.txt, Dockerfile and docker-compose.yml if you start from this solution in your own project.\n3. A failed job execution does not reschedule the job.\n4. No proper logging","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterk%2Fredis-queue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpeterk%2Fredis-queue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpeterk%2Fredis-queue/lists"}