{"id":38358855,"url":"https://github.com/ebogdum/steadykey","last_synced_at":"2026-01-17T03:18:21.444Z","repository":{"id":323392949,"uuid":"1093071468","full_name":"ebogdum/steadykey","owner":"ebogdum","description":"Deterministic idempotency keys and storage adapters for taming duplicate requests across Redis, SQL, MongoDB, and in-memory backends.","archived":false,"fork":false,"pushed_at":"2025-11-09T21:24:38.000Z","size":149,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-09T23:18:40.759Z","etag":null,"topics":["idempotence-middleware","idempotency","idempotency-key","idempotency-support","idempotent","idempotent-kafka","idempotent-requests","typescript","typescript-idempotency","typescript-idempotent"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ebogdum.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-09T21:04:18.000Z","updated_at":"2025-11-09T21:33:37.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ebogdum/steadykey","commit_stats":null,"previous_names":["ebogdum/steadykey"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/ebogdum/steadykey","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebogdum%2Fsteadykey","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebogdum%2Fsteadykey/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebogdum%2Fsteadykey/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebogdum%2Fsteadykey/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ebogdum","download_url":"https://codeload.github.com/ebogdum/steadykey/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebogdum%2Fsteadykey/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28492636,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T02:39:23.645Z","status":"ssl_error","status_checked_at":"2026-01-17T02:34:19.649Z","response_time":85,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["idempotence-middleware","idempotency","idempotency-key","idempotency-support","idempotent","idempotent-kafka","idempotent-requests","typescript","typescript-idempotency","typescript-idempotent"],"created_at":"2026-01-17T03:18:21.223Z","updated_at":"2026-01-17T03:18:21.433Z","avatar_url":"https://github.com/ebogdum.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# steadykey\n\nDeterministic idempotency manager for JSON payloads with pluggable persistence. Generate stable idempotency keys, prevent duplicate work, and keep canonical payloads for auditing when you need them.\n\n## Contents\n\n- Getting Started\n- Installation\n- Quick Tour\n- How It Works\n- API Reference\n- Storage Adapters\n- Utilities\n- Error Reference\n- Developing and Testing\n- Need Help?\n\n## Getting Started\n\nUse `IdempotencyManager` to protect any workflow where repeated payloads should only be processed once. The manager stores a marker the first time it sees a payload, then lets you decide what to do when the payload returns.\n\n```ts\nimport { createClient } from \"redis\";\nimport { IdempotencyManager, RedisIdempotencyStore } from \"steadykey\";\n\nconst redis = createClient({ url: process.env.REDIS_URL });\nawait redis.connect();\n\nconst store = new RedisIdempotencyStore(redis);\nconst manager = new IdempotencyManager(store, {\n  keyPrefix: \"checkout\",\n  defaultTtlSeconds: 3600,\n  storeCanonicalPayload: true,\n});\n\nconst payload = { orderId: \"order-123\", total: 42.5 };\n\nconst registration = await manager.register(payload, {\n  metadata: { workflow: \"checkout\" },\n});\n\nif (registration.stored) {\n  // First encounter: perform the expensive work and persist your result.\n  // Later you can call manager.clear(id) or manager.updateTtl(id, ttl) when done.\n} else {\n  // Duplicate payload: skip the work and reuse the prior result.\n}\n```\n\nFor quick checks, call `steadyKey(payload)` to get a deterministic hash without creating a manager.\n\n## Installation\n\nInstall the core package plus the adapter dependencies your project uses.\n\n```sh\nnpm install steadykey\n\n# Optional adapter helpers\nnpm install redis             # RedisIdempotencyStore\nnpm install memcached         # MemcachedIdempotencyStore\nnpm install pg                # PostgresIdempotencyStore\nnpm install mysql2            # MySqlIdempotencyStore\nnpm install mongodb           # MongoIdempotencyStore\nnpm install better-sqlite3    # SqliteIdempotencyStore\nnpm install @aws-sdk/client-dynamodb @aws-sdk/lib-dynamodb  # DynamoDbIdempotencyStore\n```\n\n## Quick Tour\n\n- `IdempotencyManager` orchestrates key generation, storage, TTL management, and collision detection.\n- Storage adapters implement the lightweight `IdempotencyStore` interface so you can bring your own persistence layer.\n- Utility helpers (`steadyKey`, `canonicalize`, `hashCanonicalValue`) let you generate and inspect deterministic payload hashes outside of a full manager.\n- Typed results explain whether the current call was stored (`stored: true`) or matches an existing record (`stored: false`).\n\n## How It Works\n\n1. Payloads are canonicalized before hashing. Object keys are sorted, `undefined` values are dropped, Maps/Sets/BigInts/Buffers are normalized, and Dates become ISO strings. Identical logical payloads always hash to the same value.\n2. The chosen hash algorithm (`sha256` by default) creates an idempotency identifier.\n3. `IdempotencyManager` prefixes the identifier (default `idempotency:`) to build the storage key.\n4. The storage adapter stores the record if the key is not already present. When the key exists, the stored payload hash is compared to guard against silent collisions.\n5. TTLs come either from the manager constructor (`defaultTtlSeconds`), from each registration call, or can be removed entirely by passing `null` or `0`.\n\n## API Reference\n\n### `steadyKey(payload, options?)`\n\n- Returns a deterministic string hash for any JSON-like payload.\n- `options.hashAlgorithm` accepts \"sha256\" (default) or \"sha512\".\n\n```ts\nimport { steadyKey } from \"steadykey\";\n\nconst key = steadyKey({ customerId: 123, items: [\"A\", \"B\"] });\n// same key every call, regardless of object key order\n```\n\n### `class IdempotencyManager`\n\n```ts\nconst manager = new IdempotencyManager(store, options?);\n```\n\n- `store` must satisfy the `IdempotencyStore` interface (see adapters below).\n- `options.keyPrefix` (string) defaults to \"idempotency\". Trailing colons are trimmed automatically.\n- `options.defaultTtlSeconds` (positive integer | `null` | `undefined`) sets the fallback TTL. `null` or `undefined` means no expiration.\n- `options.hashAlgorithm` overrides the hashing algorithm used for the manager (\"sha256\" or \"sha512\").\n- `options.storeCanonicalPayload` stores the canonical JSON alongside the record to help with auditing or debugging.\n\n#### `generateId(payload)`\n\nReturns the deterministic hash for a payload using the manager's algorithm. Useful if you want to build keys or pre-compute lookups.\n\n#### `buildKey(id)`\n\nCombines `keyPrefix` and an id into the stored key. The legacy alias `buildRedisKey` is still available but deprecated.\n\n#### `register(payload, options?)`\n\nStores a record the first time the payload is seen.\n\n```ts\nconst result = await manager.register(payload, {\n  ttlSeconds: 900,\n  metadata: { workflow: \"checkout\" },\n  storeCanonicalPayload: false,\n});\n\nif (result.stored) {\n  // process payload\n}\n```\n\n- `options.ttlSeconds` overrides the manager default for this call.\n- `options.metadata` accepts any JSON-serializable value (objects, strings, numbers, etc.) and is stored with the record.\n- `options.storeCanonicalPayload` toggles payload storage per call.\n- Result shape: `{ id, key, stored, record }` where `record` reflects the stored data (including metadata and canonical payload when present).\n\n#### `lookupByPayload(payload)` / `lookupById(id)`\n\nFetch existing records without registering anything. Returns `null` when no record is found.\n\n```ts\nconst previous = await manager.lookupByPayload(payload);\nif (previous) {\n  console.log(previous.record.metadata);\n}\n```\n\n- Lookup results include `{ id, key, record, ttlSeconds }` where `ttlSeconds` comes from the backing store when available.\n\n#### `updateTtl(id, ttlSeconds)`\n\nRefreshes, sets, or removes the TTL for an existing record. Pass `null` or `0` to make the record persistent. Throws when the key does not exist.\n\n#### `clear(id)`\n\nDeletes the stored record. Returns `true` when a record was removed.\n\n### Records and Data Shapes\n\n- `IdempotencyRecord`: `{ id, payloadHash, createdAt, metadata?, canonicalPayload?, ttlSeconds? }`\n- `IdempotencyRegistrationResult`: `{ id, key, stored, record }`\n- `IdempotencyLookupResult`: `{ id, key, record, ttlSeconds? }`\n- `HashAlgorithm`: union of \"sha256\" | \"sha512\"\n\nThese types are exported from `steadykey` so you can annotate your code when TypeScript type safety matters.\n\n### Creating Custom Stores\n\nImplement the `IdempotencyStore` interface if you need a bespoke persistence layer.\n\n```ts\ninterface IdempotencyStore {\n  setIfAbsent(key: string, value: string, ttlSeconds: number | null): Promise\u003cboolean\u003e;\n  get(key: string): Promise\u003c{ value: string; ttlSeconds?: number | null } | null\u003e;\n  update(key: string, value: string, ttlSeconds: number | null): Promise\u003cvoid\u003e;\n  delete(key: string): Promise\u003cboolean\u003e;\n}\n```\n\n- `setIfAbsent` must behave atomically: only return `true` when the key did not exist.\n- `get` should ignore expired entries and return their TTL when known.\n- `update` must throw when the key is missing to avoid silently masking data issues.\n- `delete` should return whether the key was removed.\n\n## Storage Adapters\n\n### InMemoryIdempotencyStore\n\n- Lightweight Map-based implementation ideal for tests.\n- Constructor accepts `{ now?: () =\u003e number }` for deterministic time sources.\n- Exposes `advanceTime(milliseconds)` to fast-forward expirations in tests.\n\n```ts\nimport { InMemoryIdempotencyStore } from \"steadykey\";\n\nconst store = new InMemoryIdempotencyStore();\nconst manager = new IdempotencyManager(store);\n\nstore.advanceTime(5_000); // simulate clock jumps in unit tests\n```\n\n### RedisIdempotencyStore\n\n- Wraps a `redis` client with `set`, `get`, `ttl`, `persist`, and `del` methods.\n- Pass TTLs via `EX` so expirations are handled server-side.\n- `update` removes TTLs when `ttlSeconds` is `null`.\n\n```ts\nconst redisStore = new RedisIdempotencyStore(redisClient);\n```\n\n### MemcachedIdempotencyStore\n\n- Works with clients compatible with the `memcached` npm package.\n- TTL reporting is not available, so lookups return `ttlSeconds: undefined`.\n- Uses `add` for atomic set-if-absent operations.\n\n```ts\nconst memcachedStore = new MemcachedIdempotencyStore(memcachedClient);\n```\n\n### PostgresIdempotencyStore\n\n- Requires any client exposing a `query(sql, params)` method (e.g., `pg.Pool`).\n- Options: `{ tableName?: string, ensureTable?: boolean }`.\n- Defaults to creating `steadykey_entries` with an `expires_at` index. Disable auto-DDL with `ensureTable: false`.\n\n```ts\nconst pgStore = new PostgresIdempotencyStore(pool, {\n  tableName: \"public.steadykey_entries\",\n});\n```\n\n### MySqlIdempotencyStore\n\n- Works with `mysql2/promise` connections.\n- Options: `{ tableName?: string, ensureTable?: boolean, keyLength?: number }`.\n- Auto-DDL creates an indexed table with configurable primary key length.\n\n```ts\nconst mysqlStore = new MySqlIdempotencyStore(connection, {\n  tableName: \"steadykey_entries\",\n  keyLength: 128,\n});\n```\n\n### MongoIdempotencyStore\n\n- Accepts a MongoDB collection implementing `insertOne`, `findOne`, `updateOne`, `deleteOne`, and `createIndex`.\n- Options: `{ ensureIndexes?: boolean }`. Defaults to building a TTL index on `expiresAt`.\n\n```ts\nconst mongoStore = new MongoIdempotencyStore(collection, {\n  ensureIndexes: true,\n});\n```\n\n### DynamoDbIdempotencyStore\n\n- Works with AWS SDK v3 `DynamoDBDocumentClient` or compatible clients exposing `put`, `get`, `update`, and `delete`.\n- Options: `{ tableName: string, partitionKey?: string, valueAttribute?: string, ttlAttribute?: string, consistentRead?: boolean }`.\n- Uses conditional writes for atomic inserts and stores TTL as epoch seconds when provided.\n\n```ts\nimport { DynamoDBClient } from \"@aws-sdk/client-dynamodb\";\nimport { DynamoDBDocumentClient } from \"@aws-sdk/lib-dynamodb\";\nimport { DynamoDbIdempotencyStore } from \"steadykey\";\n\nconst client = new DynamoDBClient({});\nconst documentClient = DynamoDBDocumentClient.from(client);\n\nconst dynamoStore = new DynamoDbIdempotencyStore(documentClient, {\n  tableName: \"steadykey_entries\",\n});\n```\n\n### SqliteIdempotencyStore\n\n- Compatible with synchronous libraries such as `better-sqlite3` or async wrappers that match the minimal interface.\n- Options: `{ tableName?: string, ensureTable?: boolean }`.\n- Automatically creates a table keyed by `key` with an index on `expires_at` (epoch seconds).\n\n```ts\nconst sqliteStore = new SqliteIdempotencyStore(sqliteDb, {\n  tableName: \"steadykey_entries\",\n});\n```\n\n## Utilities\n\n- `canonicalize(value)` returns the deterministic JSON string used for hashing. Useful for debugging when combined with `storeCanonicalPayload`.\n- `hashCanonicalValue(canonicalValue, algorithm)` hashes previously canonicalized JSON. This is exported for advanced integrations or to align custom tooling with Steadykey.\n\n```ts\nimport { canonicalize, hashCanonicalValue } from \"steadykey\";\n\nconst canonical = canonicalize(payload);\nconst id = hashCanonicalValue(canonical, \"sha512\");\n```\n\n## Error Reference\n\n- `IdempotencyError`: thrown for invalid input or misconfigured stores.\n- `IdempotencyCollisionError`: thrown when two different payloads attempt to reuse the same key.\n- `IdempotencySerializationError`: wraps canonicalization or JSON serialization issues. Inspect the message for the underlying cause.\n\nAlways surface collisions and serialization errors in logs or metrics—they indicate data drift or payloads the hashing strategy cannot support yet.\n\n## Developing and Testing\n\n- Run unit tests with `npm test` (Vitest).\n- Build distributable bundles with `npm run build` (outputs ESM, CJS, and type declarations under `dist/`).\n- Build once before running the Node examples under `examples/` (they import from `dist/index.js`).\n- When adding new storage backends, implement the `IdempotencyStore` contract and add adapter-specific tests under `tests/`.\n\n## Need Help?\n\n- Open an issue or discussion in the repository with payload samples and adapter details.\n- Pull requests are welcome—please include tests and update this README when the API surface changes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febogdum%2Fsteadykey","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Febogdum%2Fsteadykey","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febogdum%2Fsteadykey/lists"}