{"id":50968957,"url":"https://github.com/albe/node-event-storage-http","last_synced_at":"2026-06-18T23:32:43.854Z","repository":{"id":362998982,"uuid":"1250603434","full_name":"albe/node-event-storage-http","owner":"albe","description":"HTTP API layer on top of node-event-storage","archived":false,"fork":false,"pushed_at":"2026-06-06T23:06:38.000Z","size":86,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-07T00:12:34.804Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/albe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-26T19:49:21.000Z","updated_at":"2026-06-06T23:06:41.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/albe/node-event-storage-http","commit_stats":null,"previous_names":["albe/node-event-storage-http"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/albe/node-event-storage-http","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albe%2Fnode-event-storage-http","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albe%2Fnode-event-storage-http/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albe%2Fnode-event-storage-http/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albe%2Fnode-event-storage-http/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/albe","download_url":"https://codeload.github.com/albe/node-event-storage-http/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/albe%2Fnode-event-storage-http/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34511617,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-18T02:00:06.871Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-18T23:32:43.480Z","updated_at":"2026-06-18T23:32:43.822Z","avatar_url":"https://github.com/albe.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# event-storage-http\n\nHTTP API layer for [`event-storage`](https://www.npmjs.com/package/event-storage) — exposes an EventStore instance as a set of REST endpoints over NDJSON streaming.\n\nRequires `event-storage \u003e= 1.3.0`.\n\n## What and why\n\n`event-storage-http` bridges a Node.js `EventStore` instance and any HTTP client. It is designed for setups where the event store lives in a **dedicated backend service** that multiple consumers — frontends, microservices, serverless functions, or other runtimes — need to reach over the network without a direct Node.js dependency.\n\n**Use this package when:**\n- you run the EventStore as a standalone service and trusted backend clients (Python workers, Go services, other Node.js services, …) need to connect remotely\n- you need a simple, language-agnostic integration point between internal services without embedding the storage library\n\n**Do not use this package when:**\n- your consumers run in the same Node.js process — use `event-storage` directly for best performance and type safety\n- you want to expose data to browsers or the public internet — this layer provides no authentication, authorization, or input sanitization and must only be reachable by trusted clients\n- you need fine-grained access control, authentication, or schema validation — add those layers yourself before reaching for this package\n- low-latency write paths are critical — the HTTP round-trip adds overhead that in-process access avoids\n\n## Usage\n\nMinimal setup — create an EventStore, wrap it in the HTTP API, and start listening:\n\n```js\nimport EventStore from 'event-storage';\nimport EventStoreHttpApi from 'event-storage-http';\n\nconst eventStore = new EventStore({\n    storageDirectory: './data',\n    typeAccessor: 'type'\n});\n\nconst api = new EventStoreHttpApi(eventStore);\napi.listen(3000, () =\u003e console.log('Event store listening on port 3000'));\n```\n\n### With options\n\nThe second argument to `EventStoreHttpApi` accepts configuration for consumer behaviour:\n\n```js\nimport EventStore from 'event-storage';\nimport EventStoreHttpApi from 'event-storage-http';\n\nconst eventStore = new EventStore({\n    storageDirectory: './data',\n    typeAccessor: 'type'\n});\n\nconst api = new EventStoreHttpApi(eventStore, {\n    // Re-open and register all persisted consumers at boot time so they begin\n    // processing immediately without a client issuing a PUT first.\n    autoStartConsumers: true,\n\n    // How long (ms) GET /consumers/:id/after/:version waits before responding\n    // with 408 Request Timeout.  Defaults to 10 000.\n    consumerPollTimeoutMs: 30_000,\n\n    // How long (ms) stream/join/category reads wait for the next event when\n    // long-polling is active before timing out. Defaults to 10 000.\n    streamPollTimeoutMs: 30_000\n});\n\napi.listen(3000, () =\u003e console.log('Event store listening on port 3000'));\n```\n\n## Endpoints\n\n- `POST /streams/{stream}/commit`\n- `PUT /streams/{stream}`\n- `GET /streams`\n- `GET /streams/{stream}[/from/{from}][/until/{until}][/forwards/{amount}][/backwards/{amount}]`\n- `GET /streams/{stream}/version`\n- `GET /streams/join[/from/{from}][/until/{until}][/forwards/{amount}][/backwards/{amount}]?streams=...`\n- `GET /streams/category/{category}[/from/{from}][/until/{until}][/forwards/{amount}][/backwards/{amount}]`\n- `GET /query[/from/{revision}]?types=...`\n- `PUT /consumers/{identifier}/stream/{stream}[/from/{revision}]`\n- `GET /consumers/{identifier}`\n- `GET /consumers/{identifier}/after/{minVersion}`\n- `GET /consumers`\n- `GET /health`\n\nStream, join, category, and query reads return `application/x-ndjson`. These endpoints use the core EventStore raw mode, so event documents are streamed as newline-delimited JSON buffers directly to the HTTP response.\n\nQuery responses also expose a serialized optimistic-concurrency condition in the `x-event-store-query-condition` response header so clients can pass it back to `POST /streams/{stream}/commit`.\n\n`start` and `end` are accepted wherever a revision boundary is expected. Matchers are JSON object matchers using the same shape as the core storage matchers (`{ stream, payload, metadata }`). The HTTP layer reuses the canonical matcher implementation from `event-storage`.\n\n`GET /health` returns a small JSON snapshot for liveness and basic diagnostics. It includes whether the store appears open, whether it is currently writable, store length, stream/consumer counts, `event-storage` version (best effort), and selected runtime information from the Express/Node process.\n\n### Stream endpoints\n\n`POST /streams/{stream}/commit` appends events to a stream.\n\n- Path params:\n  - `stream`: target stream name.\n- Body (JSON object):\n  - `events` (required): non-empty array of event payload objects.\n  - `expectedVersion` (optional): integer, `\"any\"`, or `\"empty\"`.\n  - `condition` (optional): serialized DCB commit condition (`{ types, noneMatchAfter, matcher? }`).\n  - `metadata` (optional): object merged into commit metadata for all events.\n\n```json\n{\n  \"events\": [\n    { \"type\": \"OrderPlaced\", \"orderId\": \"1\" }\n  ],\n  \"expectedVersion\": \"any\",\n  \"metadata\": { \"requestId\": \"req-1\" }\n}\n```\n\n`PUT /streams/{stream}` creates a stream index (matcher stream).\n\n- Path params:\n  - `stream`: name of the stream to create.\n- Body (JSON object): matcher definition, either directly as the request body or wrapped in `matcher`.\n\n```json\n{\n  \"stream\": [\"orders-1\", \"orders-2\"],\n  \"payload\": { \"type\": \"OrderPlaced\" }\n}\n```\n\nor\n\n```json\n{\n  \"matcher\": {\n    \"stream\": [\"orders-1\", \"orders-2\"],\n    \"payload\": { \"type\": \"OrderPlaced\" }\n  }\n}\n```\n\n`GET /streams` returns all known streams from the in-memory stream registry, including `stream`, `closed`, `version`, and `metadata`.\n\n`GET /streams/{stream}[/from/{from}][/until/{until}][/forwards/{amount}][/backwards/{amount}]` returns NDJSON events for one stream.\n\n- Path params:\n  - `stream`: stream name.\n  - `from` / `until` (optional): revision boundary (`start`, `end`, or integer).\n  - `forwards` / `backwards` (optional): max number of events in selected direction.\n- Query params:\n  - `filter` (optional): matcher JSON object (URL-encoded when passed as string).\n\n**Long-polling:** When `until` is greater than the current visible version (or `from` exceeds it), stream/join/category reads enter long-poll mode. The API streams everything it can see immediately and only waits while the next in-range event is missing. If at least one event is emitted, the response status is `200` and the response closes on timeout when no further event arrives. If no event could be emitted at all before timeout, the server responds with HTTP `408 Request Timeout`.\n\n`GET /streams/{stream}/version` returns `{ stream, version }`.\n\n`GET /streams/join[/from/{from}][/until/{until}][/forwards/{amount}][/backwards/{amount}]?streams=...` returns one merged NDJSON stream over all listed streams.\n\n- Query params:\n  - `streams` (required): comma-separated stream names.\n  - `filter` (optional): matcher JSON object.\n\n\n`GET /streams/category/{category}[/from/{from}][/until/{until}][/forwards/{amount}][/backwards/{amount}]` returns NDJSON events for all streams in a category (`{category}-...` or `{category}/...`).\n\n\n### Consumer endpoints\n\n`PUT /consumers/{identifier}/stream/{stream}[/from/{revision}]` starts a durable consumer that is kept running in memory and registered in the EventStore's internal `consumers` map (keyed by `identifier`). Re-issuing the PUT replaces the existing consumer and restarts from the new handler and state.\n\n`GET /consumers/{identifier}` returns the live position and state of the named consumer from the in-memory registry. Returns `404` if the consumer is not registered.\n\n`GET /consumers/{identifier}/after/{minVersion}` is a long-poll endpoint that blocks until the named consumer's position reaches `minVersion` or later, then responds with the consumer's current position and state. The returned position is therefore guaranteed to be at least the requested version and may be higher. If the consumer does not advance to `minVersion` within the configured timeout (default 10 s, configurable via `options.consumerPollTimeoutMs`), the server responds with HTTP `408 Request Timeout`. The consumer must be registered in the event store's consumer registry (via `PUT` or by the startup scan) before calling this endpoint.\n\n```http\nGET /consumers/orders-reader/after/5\n```\n\n```json\n{\n  \"identifier\": \"orders-reader\",\n  \"stream\": \"orders\",\n  \"position\": 5,\n  \"state\": { \"count\": 5 }\n}\n```\n\n`GET /consumers` lists all consumers currently registered in memory. It also fires an asynchronous filesystem scan to keep the registry eventually consistent with consumers created outside of this process.\n\nOn startup, `EventStoreHttpApi` calls `eventStore.scanConsumers()` once to pre-populate the consumer registry. Pass `options.autoStartConsumers: true` to open and register all existing consumers on disk at boot time.\n\nRaw-mode matcher notes:\n\n- Object matchers are evaluated using the same core matcher semantics as `event-storage`, including nested equality, array OR values, and scalar operators (`$gt`, `$gte`, `$lt`, `$lte`, `$eq`, `$ne`).\n- Object matchers are evaluated against compact JSON bytes (no parsing in the HTTP layer).\n- Function matchers in raw mode receive a raw document `Buffer`.\n- Raw object matchers require the default compact JSON serializer format.\n\n## Benchmark snapshot (local loopback)\n\nThe benchmark script in `bench/bench-http-layer.js` measures client-observed throughput including HTTP round-trips over `127.0.0.1`.\nThese numbers are a coarse upper bound for the HTTP layer on this machine and can be compared directionally against the in-process benchmark.\n\nRun used for the table below:\n\n- Date: 2026-05-26\n- Command: `npm run bench:http`\n- Read fixture: 10,000 events\n- Requests per lane: write `400`, read `4`\n\n### Write performance (events/s, 1 event/commit)\n\n| Scenario | 1 | 2 | 4 | 8 | 16 |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| single-event commit (sharded streams) | 809 | 1,422 | 1,731 | 2,109 | 2,389 |\n| single-event commit (single stream) | 1,051 | 1,494 | 1,882 | 2,142 | 2,401 |\n\n### Read performance (events/s)\n\n| Scenario | 1 | 2 | 4 | 8 | 16 |\n| --- | ---: | ---: | ---: | ---: | ---: |\n| 1 - forward full scan | 107,829 | 122,049 | 133,120 | 139,627 | 138,417 |\n| 2 - backwards full scan | 113,233 | 131,419 | 140,503 | 138,803 | 139,791 |\n| 3 - join stream | 127,713 | 132,546 | 142,758 | 145,804 | 143,378 |\n| 4 - range scan | 97,060 | 117,719 | 131,956 | 138,267 | 137,104 |\n\n## Further reading\n\nFull documentation for the underlying `event-storage` library, including the EventStore API, streams, consumers, DCB concurrency, and performance notes, is available at:\n\n👉 **https://node-event-storage.readthedocs.io/en/latest/**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falbe%2Fnode-event-storage-http","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falbe%2Fnode-event-storage-http","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falbe%2Fnode-event-storage-http/lists"}