{"id":13827543,"url":"https://github.com/adjust/redismq","last_synced_at":"2026-01-12T03:07:10.271Z","repository":{"id":566406,"uuid":"11223149","full_name":"adjust/redismq","owner":"adjust","description":"a durable message queue system for go based on redis, see also https://github.com/adjust/rmq","archived":false,"fork":false,"pushed_at":"2022-04-20T07:22:40.000Z","size":117,"stargazers_count":507,"open_issues_count":4,"forks_count":73,"subscribers_count":88,"default_branch":"master","last_synced_at":"2025-08-14T10:35:26.997Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adjust.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-07-06T19:19:22.000Z","updated_at":"2025-06-26T01:30:22.000Z","dependencies_parsed_at":"2022-08-03T06:30:29.124Z","dependency_job_id":null,"html_url":"https://github.com/adjust/redismq","commit_stats":null,"previous_names":["adeven/redismq","adeven/rqueue"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/adjust/redismq","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adjust%2Fredismq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adjust%2Fredismq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adjust%2Fredismq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adjust%2Fredismq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adjust","download_url":"https://codeload.github.com/adjust/redismq/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adjust%2Fredismq/sbom","scorecard":{"id":166780,"data":{"date":"2025-08-11","repo":{"name":"github.com/adjust/redismq","commit":"21a19167d34630246e4a2ce134551b0158106291"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.3,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":2,"reason":"Found 6/24 approved changesets -- score normalized to 2","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 14 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-16T15:09:00.606Z","repository_id":566406,"created_at":"2025-08-16T15:09:00.606Z","updated_at":"2025-08-16T15:09:00.606Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279005461,"owners_count":26083902,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T09:02:01.143Z","updated_at":"2025-10-10T22:47:14.518Z","avatar_url":"https://github.com/adjust.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"Note: This project is no longer actively maintained. Please refer to its spiritual successor [rmq].\n\n[rmq]: https://github.com/adjust/rmq\n\n--\n\n# redismq\n\n[![Build Status](https://secure.travis-ci.org/adjust/redismq.png)](http://travis-ci.org/adjust/redismq) [![godoc](http://img.shields.io/badge/godoc-reference-blue.svg?style=flat)](https://godoc.org/github.com/adjust/redismq)\n\n## What is this\n\nThis is a fast, persistent, atomic message queue implementation that uses redis as its storage engine written in go.\nIt uses atomic list commands to ensure that messages are delivered only once in the right order without being lost by crashing consumers.\n\nDetails can be found in the blog post about its initial design:\n[http://big-elephants.com/2013-09/building-a-message-queue-using-redis-in-go/](http://big-elephants.com/2013-09/building-a-message-queue-using-redis-in-go/)\n\nA second article desribes the performance improvements of the current version:\n[http://big-elephants.com/2013-10/tuning-redismq-how-to-use-redis-in-go/](http://big-elephants.com/2013-10/tuning-redismq-how-to-use-redis-in-go/)\n\n## What it's not\n\nIt's not a standalone server that you can use as a message queue, at least not for now. The implementation is done purely client side. All message queue commands are \"translated\" into redis commands and then executed via a redis client.\n\nIf you want to use this with any other language than go you have to translate all of the commands into your language of choice.\n\n## How to use it\n\nAll most all use cases are either covered in the [examples](https://github.com/adjust/redismq/tree/master/example)\nor in the [tests](https://github.com/adjust/redismq/tree/master/test).\n\nSo the best idea is just to read those and figure it from there. But in any case:\n\n### Basics\n\nTo get started you need a running redis server. Since the tests run `FlushDB()` an otherwise unused database is highly recommended\nThe first step is to create a new queue:\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"github.com/adjust/redismq\"\n)\n\nfunc main() {\n\ttestQueue := redismq.CreateQueue(\"localhost\", \"6379\", \"\", 9, \"clicks\")\n\t...\n}\n```\nTo write into the queue you simply use `Put()`:\n```go\n\t...\n\ttestQueue := redismq.CreateQueue(\"localhost\", \"6379\", \"\", 9, \"clicks\")\n\ttestQueue.Put(\"testpayload\")\n\t...\n}\n```\nThe payload can be any kind of string, yes even a [10MB one](https://github.com/adjust/redismq/blob/master/test/integration_test.go#L217).\n\nTo get messages out of the queue you need a consumer:\n```go\n\t...\n\tconsumer, err := testQueue.AddConsumer(\"testconsumer\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tpackage, err := consumer.Get()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tfmt.Println(package.Payload)\n\t...\n}\n```\n`Payload` will hold the original string, while `package` will have some additional header information.\n\nTo remove a package from the queue you have to `Ack()` it:\n```go\n\t...\n\tpackage, err := consumer.Get()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\terr = package.Ack()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\t...\n}\n```\n\n### Buffered Queues\n\nWhen input speed is of the essence `BufferedQueues` will scratch that itch.\nThey pipeline multiple puts into one fast operation. The only issue is that upon crashing or restart the packages in\nthe buffer that haven't been written yet will be lost. So it's advised to wait one second before terminating your program to flush the buffer.\n\nThe usage is as easy as it gets:\n```go\n\t...\n\tbufferSize := 100\n\ttestQueue := redismq.CreateBufferedQueue(\"localhost\", \"6379\", \"\", 9, \"clicks\", bufferSize)\n\ttestQueue.Start()\n\t...\n}\n```\n`Put()` and `Get()` stay exactly the same.\nI have found anything over 200 as `bufferSize` not to increase performance any further.\n\nTo ensure that no packages are left in the buffer when you shut down your program you need to call\n`FlushBuffer()` which will tell the queue to flush the buffer and wait till it's empty.\n```go\n\ttestQueue.FlushBuffer()\n```\n\n### Multi Get\n\nLike `BufferedQueues` for `Get()` `MultiGet()` speeds up the fetching of messages. The good news it comes without the buffer loss issues.\n\nUsage is pretty straight forward with the only difference being the `MultiAck()`:\n```go\n\t...\n\tpackages, err := consumer.MultiGet(100)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tfor i := range packages {\n\t\tfmt.Println(p[i].Payload)\n\t}\n\tpackages[len(p)-1].MultiAck()\n\t...\n}\n```\n`MultiAck()` can be called on any package in the array with all the prior packages being \"acked\". This way you can `Fail()` single packages.\n\n### Reject and Failed Queues\n\nSimilar to AMQP redismq supports `Failed Queues` meaning that packages that are rejected by a consumer will be stored in separate queue for further inspection. Alternatively a consumer can also `Requeue()` a package and put it back into the queue:\n```go\n\t...\n\tpackage, err := consumer.Get()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\terr = package.Requeue()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\t...\n}\n```\n\nTo push the message into the `Failed Queue` of this consumer simply use `Fail()`:\n```go\n\t...\n\tpackage, err := consumer.Get()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\terr = package.Fail()\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tpackage, err = suite.consumer.GetUnacked()\n\t...\n}\n```\nAs you can see there is also a command to get messages from the `Failed Queue`.\n\n## How fast is it\n\nEven though the original implementation wasn't aiming for high speeds the addition of `BufferedQueues` and `MultiGet`\nmake it go something like [this](https://youtu.be/g8nOci4Xhns?t=5).\n\nAll of the following benchmarks were conducted on a MacBook Retina with a 2.4 GHz i7.\nThe InputRate is the number of messages per second that get inserted, WorkRate the messages per second consumed.\n\nSingle Publisher, Two Consumers only atomic `Get` and `Put`\n```\nInputRate:\t12183\nWorkRate:\t12397\n```\n\nSingle Publisher, Two Consumers using `BufferedQueues` and `MultiGet`\n```\nInputRate:\t46994\nWorkRate:\t25000\n```\n\nAnd yes that is a persistent message queue that can move over 70k messages per second.\n\nIf you want to find out for yourself checkout the `example` folder. The `load.go` or `buffered_queue.go`\nwill start a web server that will display performance stats under `http://localhost:9999/stats`.\n\n## How persistent is it\n\nAs redis is the underlying storage engine you can set your desired persistence somewhere between YOLO and fsync().\nWith somewhat sane settings you should see no significant performance decrease.\n\n## Copyright\nredismq is Copyright © 2014 adjust GmbH.\n\nIt is free software, and may be redistributed under the terms\nspecified in the LICENSE file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadjust%2Fredismq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadjust%2Fredismq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadjust%2Fredismq/lists"}