{"id":15687316,"url":"https://github.com/calebboyd/redis-x-stream","last_synced_at":"2026-02-21T16:11:22.763Z","repository":{"id":43489761,"uuid":"301262269","full_name":"calebboyd/redis-x-stream","owner":"calebboyd","description":"An AsyncIterable interface for reading redis streams","archived":false,"fork":false,"pushed_at":"2023-08-06T13:09:34.000Z","size":1753,"stargazers_count":9,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-11-02T09:51:35.855Z","etag":null,"topics":["async","consumer","iterable","javascript","job-queue","nodejs","nodejs-library","queue","redis","redis-queue","streams","task-queue","typescript","typescript-library","worker-queue"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/calebboyd.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-10-05T01:14:23.000Z","updated_at":"2024-05-26T05:48:43.000Z","dependencies_parsed_at":"2024-10-23T20:36:59.240Z","dependency_job_id":"f967548e-bbba-40cf-931d-4c7d8c68fd4d","html_url":"https://github.com/calebboyd/redis-x-stream","commit_stats":{"total_commits":83,"total_committers":2,"mean_commits":41.5,"dds":"0.32530120481927716","last_synced_commit":"d752ff02a7d9471c7fb82593373e186462e38ca3"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/calebboyd%2Fredis-x-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/calebboyd%2Fredis-x-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/calebboyd%2Fredis-x-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/calebboyd%2Fredis-x-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/calebboyd","download_url":"https://codeload.github.com/calebboyd/redis-x-stream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223811811,"owners_count":17206933,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","consumer","iterable","javascript","job-queue","nodejs","nodejs-library","queue","redis","redis-queue","streams","task-queue","typescript","typescript-library","worker-queue"],"created_at":"2024-10-03T17:47:12.863Z","updated_at":"2026-02-21T16:11:22.757Z","avatar_url":"https://github.com/calebboyd.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# redis-x-stream\n\nAsync iterables over Redis streams. Requires Redis 5+ (6.2+ for `claimIdleTime`).\n\n[![release](https://badgen.net/github/release/calebboyd/redis-x-stream)](https://www.npmjs.com/package/redis-x-stream)\n[![license](https://badgen.net/badge/license/MIT/blue)](https://github.com/calebboyd/redis-x-stream/blob/main/LICENSE)\n[![test](https://github.com/calebboyd/redis-x-stream/actions/workflows/test.yml/badge.svg)](https://github.com/calebboyd/redis-x-stream/actions)\n\n## Install\n\n```bash\nnpm install redis-x-stream ioredis\n```\n\n## Basic Usage\n\n```typescript\nimport { RedisStream } from 'redis-x-stream'\n\nfor await (const [stream, [id, keyvals]] of new RedisStream('my-stream')) {\n  console.log(stream, id, keyvals) // 'my-stream', '1234-0', ['key', 'value']\n}\n```\n\n## Consumer Groups\n\n```typescript\nconst stream = new RedisStream({\n  streams: ['my-stream'],\n  group: 'my-group',\n  consumer: 'worker-1',\n  block: Infinity,\n  count: 10,\n  ackOnIterate: true,\n  deleteOnAck: true,\n})\n\nfor await (const [name, [id, keyvals]] of stream) {\n  await process(keyvals)\n}\n```\n\nGroups and consumers are created automatically. On startup, pending entries (PEL) are\nre-delivered before new entries.\n\n## Typed Parsing\n\nPass a `parse` callback to transform the raw key-value array. The return type flows\nthrough the generic to the iterator.\n\n```typescript\ninterface Order {\n  product: string\n  qty: number\n}\n\nconst stream = new RedisStream\u003cOrder\u003e({\n  stream: 'orders',\n  group: 'workers',\n  ackOnIterate: true,\n  parse: (id, kv) =\u003e ({ product: kv[1], qty: Number(kv[3]) }),\n})\n\nfor await (const [name, [id, order]] of stream) {\n  order.product // string\n  order.qty     // number\n}\n```\n\n## Dead Consumer Recovery\n\n`claimIdleTime` uses `XAUTOCLAIM` (Redis 6.2+) to automatically claim entries\nfrom consumers that have been idle too long. Claimed entries are yielded alongside\nnew entries.\n\n```typescript\nconst stream = new RedisStream({\n  streams: ['tasks'],\n  group: 'workers',\n  consumer: 'worker-1',\n  block: Infinity,\n  claimIdleTime: 30_000, // claim entries idle \u003e 30s\n  ackOnIterate: true,\n})\n```\n\n## Dynamic Streams\n\nAdd streams at runtime, even while blocked:\n\n```typescript\nconst stream = new RedisStream({\n  streams: ['stream-a'],\n  block: Infinity,\n})\n\n// later, from another context:\nstream.addStream('stream-b')\n```\n\n## Lifecycle\n\n```typescript\n// Graceful shutdown: finish PEL, flush acks, stop\nawait stream.drain()\n\n// Immediate stop: flush acks, close connections\nawait stream.quit()\n\n// break also cleans up (via try/finally)\nfor await (const entry of stream) {\n  if (done) break // connections are closed automatically\n}\n```\n\n## Flush Timer\n\nWhen the consumer is slow or the reader is blocked, acks can pile up.\n`flushPendingAckInterval` adds a watchdog timer that flushes pending acks\nif no new acks are queued within the interval.\n\n```typescript\nconst stream = new RedisStream({\n  streams: ['tasks'],\n  group: 'workers',\n  ackOnIterate: true,\n  flushPendingAckInterval: 5000, // flush every 5s of inactivity\n})\n```\n\n## Observability\n\nQuery stream and consumer group state without dropping to raw ioredis:\n\n```typescript\nconst info = await stream.info()         // XINFO STREAM (per stream)\nconst groups = await stream.groups()     // XINFO GROUPS\nconst consumers = await stream.consumers() // XINFO CONSUMERS\nconst pending = await stream.pending()   // XPENDING summary\n```\n\nAll return typed objects (`StreamInfo`, `GroupInfo`, `ConsumerInfo`, `PendingSummary`).\n\n## Events\n\nConnection lifecycle events are forwarded from ioredis:\n\n```typescript\nstream.on('error', (err) =\u003e console.error(err))\nstream.on('ready', () =\u003e console.log('connected'))\nstream.on('close', () =\u003e console.log('disconnected'))\nstream.on('reconnecting', () =\u003e console.log('reconnecting'))\n```\n\n## Options\n\n| Option | Type | Default | Description |\n|--------|------|---------|-------------|\n| `streams` / `stream` | `string \\| string[] \\| Record\u003cstring, string\u003e` | | Stream keys to read |\n| `group` | `string` | | Consumer group name |\n| `consumer` | `string` | auto | Consumer name |\n| `redis` | `Redis \\| string \\| RedisOptions` | `new Redis()` | Reader connection |\n| `redisControl` | `Redis \\| string \\| RedisOptions` | auto | Control connection (blocking mode) |\n| `block` | `number` | | Block timeout in ms (`Infinity` for indefinite) |\n| `count` | `number` | `100` | Max entries per read |\n| `ackOnIterate` | `boolean` | `false` | Auto-ack previous entry on each iteration |\n| `deleteOnAck` | `boolean` | `false` | XDEL after XACK |\n| `noack` | `boolean` | `false` | Bypass PEL (NOACK flag) |\n| `claimIdleTime` | `number` | | Min idle ms for XAUTOCLAIM (Redis 6.2+) |\n| `flushPendingAckInterval` | `number \\| null` | `null` | Ack flush watchdog timer in ms |\n| `parse` | `(id, kv, stream) =\u003e T` | | Entry transform callback |\n| `buffers` | `boolean` | `false` | Return Buffers instead of strings |\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalebboyd%2Fredis-x-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcalebboyd%2Fredis-x-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcalebboyd%2Fredis-x-stream/lists"}