{"id":13413423,"url":"https://github.com/robinjoseph08/redisqueue","last_synced_at":"2025-04-06T13:12:17.722Z","repository":{"id":40413533,"uuid":"195610200","full_name":"robinjoseph08/redisqueue","owner":"robinjoseph08","description":"redisqueue provides a producer and consumer of a queue that uses Redis streams","archived":false,"fork":false,"pushed_at":"2024-03-08T06:24:44.000Z","size":33,"stargazers_count":132,"open_issues_count":7,"forks_count":57,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-30T12:07:23.540Z","etag":null,"topics":["consumer","go","golang","producer","producer-consumer","redis","streams"],"latest_commit_sha":null,"homepage":"https://godoc.org/github.com/robinjoseph08/redisqueue","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robinjoseph08.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-07T04:36:54.000Z","updated_at":"2025-03-15T23:20:52.000Z","dependencies_parsed_at":"2024-06-18T13:45:34.498Z","dependency_job_id":null,"html_url":"https://github.com/robinjoseph08/redisqueue","commit_stats":{"total_commits":14,"total_committers":4,"mean_commits":3.5,"dds":0.5,"last_synced_commit":"195b427f6d5d99c0986d9b219811155ecfc01ee8"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robinjoseph08%2Fredisqueue","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robinjoseph08%2Fredisqueue/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robinjoseph08%2Fredisqueue/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robinjoseph08%2Fredisqueue/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robinjoseph08","download_url":"https://codeload.github.com/robinjoseph08/redisqueue/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247485290,"owners_count":20946398,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consumer","go","golang","producer","producer-consumer","redis","streams"],"created_at":"2024-07-30T20:01:40.012Z","updated_at":"2025-04-06T13:12:17.695Z","avatar_url":"https://github.com/robinjoseph08.png","language":"Go","readme":"# redisqueue\n\n![Version](https://img.shields.io/badge/version-v2.1.0-green.svg)\n[![GoDoc](https://godoc.org/github.com/robinjoseph08/redisqueue?status.svg)](https://pkg.go.dev/github.com/robinjoseph08/redisqueue/v2?tab=doc)\n[![Build Status](https://travis-ci.org/robinjoseph08/redisqueue.svg?branch=master)](https://travis-ci.org/robinjoseph08/redisqueue)\n[![Coverage Status](https://coveralls.io/repos/github/robinjoseph08/redisqueue/badge.svg?branch=master)](https://coveralls.io/github/robinjoseph08/redisqueue?branch=master)\n[![Go Report Card](https://goreportcard.com/badge/github.com/robinjoseph08/redisqueue)](https://goreportcard.com/report/github.com/robinjoseph08/redisqueue)\n![License](https://img.shields.io/github/license/robinjoseph08/redisqueue.svg)\n\n`redisqueue` provides a producer and consumer of a queue that uses [Redis\nstreams](https://redis.io/topics/streams-intro).\n\n## Features\n\n- A `Producer` struct to make enqueuing messages easy.\n- A `Consumer` struct to make processing messages concurrenly.\n- Claiming and acknowledging messages if there's no error, so that if a consumer\n  dies while processing, the message it was working on isn't lost. This\n  guarantees at least once delivery.\n- A \"visibility timeout\" so that if a message isn't processed in a designated\n  time frame, it will be be processed by another consumer.\n- A max length on the stream so that it doesn't store the messages indefinitely\n  and run out of memory.\n- Graceful handling of Unix signals (`SIGINT` and `SIGTERM`) to let in-flight\n  messages complete.\n- A channel that will surface any errors so you can handle them centrally.\n- Graceful handling of panics to avoid crashing the whole process.\n- A concurrency setting to control how many goroutines are spawned to process\n  messages.\n- A batch size setting to limit the total messages in flight.\n- Support for multiple streams.\n\n## Installation\n\n`redisqueue` requires a Go version with Modules support and uses import\nversioning. So please make sure to initialize a Go module before installing\n`redisqueue`:\n\n```sh\ngo mod init github.com/my/repo\ngo get github.com/robinjoseph08/redisqueue/v2\n```\n\nImport:\n\n```go\nimport \"github.com/robinjoseph08/redisqueue/v2\"\n```\n\n## Example\n\nHere's an example of a producer that inserts 1000 messages into a queue:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/robinjoseph08/redisqueue/v2\"\n)\n\nfunc main() {\n\tp, err := redisqueue.NewProducerWithOptions(\u0026redisqueue.ProducerOptions{\n\t\tStreamMaxLength:      10000,\n\t\tApproximateMaxLength: true,\n\t})\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tfor i := 0; i \u003c 1000; i++ {\n\t\terr := p.Enqueue(\u0026redisqueue.Message{\n\t\t\tStream: \"redisqueue:test\",\n\t\t\tValues: map[string]interface{}{\n\t\t\t\t\"index\": i,\n\t\t\t},\n\t\t})\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\n\t\tif i%100 == 0 {\n\t\t\tfmt.Printf(\"enqueued %d\\n\", i)\n\t\t}\n\t}\n}\n```\n\nAnd here's an example of a consumer that reads the messages off of that queue:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/robinjoseph08/redisqueue/v2\"\n)\n\nfunc main() {\n\tc, err := redisqueue.NewConsumerWithOptions(\u0026redisqueue.ConsumerOptions{\n\t\tVisibilityTimeout: 60 * time.Second,\n\t\tBlockingTimeout:   5 * time.Second,\n\t\tReclaimInterval:   1 * time.Second,\n\t\tBufferSize:        100,\n\t\tConcurrency:       10,\n\t})\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tc.Register(\"redisqueue:test\", process)\n\n\tgo func() {\n\t\tfor err := range c.Errors {\n\t\t\t// handle errors accordingly\n\t\t\tfmt.Printf(\"err: %+v\\n\", err)\n\t\t}\n\t}()\n\n\tfmt.Println(\"starting\")\n\tc.Run()\n\tfmt.Println(\"stopped\")\n}\n\nfunc process(msg *redisqueue.Message) error {\n\tfmt.Printf(\"processing message: %v\\n\", msg.Values[\"index\"])\n\treturn nil\n}\n```\n","funding_links":[],"categories":["Messaging","Go","Relational Databases","消息","Repositories","消息系统","机器学习"],"sub_categories":["Search and Analytic Databases","Advanced Console UIs","检索及分析资料库","SQL 查询语句构建库"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinjoseph08%2Fredisqueue","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobinjoseph08%2Fredisqueue","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobinjoseph08%2Fredisqueue/lists"}