{"id":40996573,"url":"https://github.com/covrom/redispubsub","last_synced_at":"2026-01-22T08:13:27.447Z","repository":{"id":63288922,"uuid":"564634338","full_name":"covrom/redispubsub","owner":"covrom","description":"Redis Streams queue driver for https://godoc.org/gocloud.dev/pubsub package","archived":false,"fork":false,"pushed_at":"2022-11-28T16:47:31.000Z","size":271,"stargazers_count":13,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-06-21T11:17:20.996Z","etag":null,"topics":["go","golang","kafka","redis","streams"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/covrom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-11T06:12:28.000Z","updated_at":"2023-11-30T09:22:12.000Z","dependencies_parsed_at":"2022-11-16T10:52:27.044Z","dependency_job_id":null,"html_url":"https://github.com/covrom/redispubsub","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/covrom/redispubsub","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/covrom%2Fredispubsub","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/covrom%2Fredispubsub/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/covrom%2Fredispubsub/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/covrom%2Fredispubsub/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/covrom","download_url":"https://codeload.github.com/covrom/redispubsub/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/covrom%2Fredispubsub/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28659439,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T01:17:37.254Z","status":"online","status_checked_at":"2026-01-22T02:00:07.137Z","response_time":144,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","kafka","redis","streams"],"created_at":"2026-01-22T08:13:26.799Z","updated_at":"2026-01-22T08:13:27.440Z","avatar_url":"https://github.com/covrom.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# redispubsub\nRedis driver for https://godoc.org/gocloud.dev/pubsub package.\n\nA great alternative to using Kafka, with the ability to quickly switch to it. You can use this driver for MVP, local, small or medium projects. When your project grows you can simply switch to another driver from https://pkg.go.dev/gocloud.dev/pubsub#section-directories.\n\nUsing Redis Streams, this driver supports `at-least-once` delivery.\n\nThe driver uses these Redis commands:\n- XADD\n- XGROUP CREATE\n- XREADGROUP (with pending and then new messages - only this library actually supports it)\n- XACK\n- XAUTOCLAIM\n\nMany other queuing implementations with Redis Streams contain a big bug. They incorrectly support reconnecting a consumer to a topic if a message has been received but not acknowledged. They use \"\u003e\" streaming strategy, which does not deliver unacknowledged messages more than once. And you miss messages when microservices are restarted.\nThis library does not have this disadvantage.\n\n## Connection to Redis\nThe connection string must be defined in the `REDIS_URL` environment value.\n\n## Warning about creating a topic consumer group for the first time\nAll consumers already have a group, even if there is only one consumer in the group.\n\nConsumer groups receive the same messages from the topic, and consumers within the group receive these messages exclusively.\n\n![Messages flow](flow.png)\n\nThis driver supports new consumers joining with a new group name after the publisher has sent multiple messages to a topic before the group was created. These consumers will receive all previous non-ACK-ed messages from the beginning of the topic.\n\n## How to open topic and send message\n```go\nimport (\n    _ \"github.com/covrom/redispubsub\"\n    \"gocloud.dev/pubsub\"\n)\n\nctx := context.Background()\ntopic, err := pubsub.OpenTopic(ctx, \"redis://topics/1\")\nif err != nil {\n    return fmt.Errorf(\"could not open topic: %v\", err)\n}\ndefer topic.Shutdown(ctx)\n\nm := \u0026pubsub.Message{\n    Body: []byte(\"Hello, World!\\n\"),\n    // Metadata is optional and can be nil.\n    Metadata: map[string]string{\n        // These are examples of metadata.\n        // There is nothing special about the key names.\n        \"language\":   \"en\",\n        \"importance\": \"high\",\n    },\n}\n\nerr = topic.Send(ctx, m)\nif err != nil {\n    return err\n}\n```\n\nOpenTopic connection string host/path is required and must contain the topic name.\n\nConnection string query parameters:\n- `maxlen` is MAXLEN parameter for XADD (limit queue length), unlimited if not set.\n\n## How to subscribe on topic\n```go\nimport (\n    _ \"github.com/covrom/redispubsub\"\n    \"gocloud.dev/pubsub\"\n)\n\nsubs, err := pubsub.OpenSubscription(ctx, \"redis://group1?consumer=cons1\u0026topic=topics/1\")\nif err != nil {\n    return err\n}\ndefer subs.Shutdown(ctx)\n\nmsg, err := subs.Receive(ctx)\nif err != nil {\n    // Errors from Receive indicate that Receive will no longer succeed.\n    return fmt.Errorf(\"Receiving message: %v\", err)\n}\n// Do work based on the message, for example:\nfmt.Printf(\"Got message: %q\\n\", msg.Body)\n// Messages must always be acknowledged with Ack.\nmsg.Ack()\n```\n\nOpenSubscription connection string host(path) is required and must contain the consumer group name.\n\nConnection string query parameters:\n- `topic` is topic name, required\n- `consumer` is unique consumer name, required\n- `from` is FROM option for creating a consumer group (if not exists) with XGROUP CREATE, default is '0'\n- `autoclaim` is *min-idle-time* option for XAUTOCLAIM, 30 min by default\n- `noack` is NOACK option for XREADGROUP, not used by default\n\nSee [basic_test.go](basic_test.go) for full usage example.\n\n## Monitoring with Prometheus \u0026 Grafana\nUse [redis-exporter](https://github.com/oliver006/redis_exporter) prometheus exporter with `check-streams` option.\n\nSee [streams.go](https://github.com/oliver006/redis_exporter/blob/master/exporter/streams.go) for details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcovrom%2Fredispubsub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcovrom%2Fredispubsub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcovrom%2Fredispubsub/lists"}