{"id":40471162,"url":"https://github.com/polad/kylfil","last_synced_at":"2026-01-20T18:17:38.818Z","repository":{"id":319479114,"uuid":"1073246387","full_name":"polad/kylfil","owner":"polad","description":"Database-Agnostic Event Store","archived":false,"fork":false,"pushed_at":"2025-11-05T23:34:27.000Z","size":24587,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-11-06T01:19:03.898Z","etag":null,"topics":["database","event-sourcing","event-store","eventsourcing","eventstore","javascript","node","nodejs"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/polad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-09T20:31:33.000Z","updated_at":"2025-11-05T23:41:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"f6848174-7053-49f8-bc38-e54cf29c8fe9","html_url":"https://github.com/polad/kylfil","commit_stats":null,"previous_names":["polad/kylfil"],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/polad/kylfil","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polad%2Fkylfil","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polad%2Fkylfil/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polad%2Fkylfil/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polad%2Fkylfil/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/polad","download_url":"https://codeload.github.com/polad/kylfil/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polad%2Fkylfil/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28608672,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T16:10:39.856Z","status":"ssl_error","status_checked_at":"2026-01-20T16:10:39.493Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","event-sourcing","event-store","eventsourcing","eventstore","javascript","node","nodejs"],"created_at":"2026-01-20T18:17:38.700Z","updated_at":"2026-01-20T18:17:38.769Z","avatar_url":"https://github.com/polad.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# KylFil\n\nAll killer. No filler.\\\nA tiny database-agnostic Event Store\u0026mdash;clever by design, minimal by choice.\n\n- Optimistic Concurrency Control (OCC): no resource locks required.\n- Simple Functional API: for great composition and expressive code.\n- Atomic Guarantees: append multiple events in a single-transaction.\n- Storage Agnostic: supports virtually any Database or KV Store via a simple `StoreProvider` interface requiring only two functions: `append()` and `read()`.\n- Reference Implementations: get started right away with Postgres, SQLite, MySQL (more coming soon)\u0026mdash;or use them as a blueprint for integrating with your favourite store.\n- Lazy DB Connection \u0026 Deferred Query Execution: improves performance by avoiding premature resource allocation.\n- Flexible Querying: read events in any direction or from a specific version.\n- Immutable Operations: all functions return new objects without modifying original objects.\n- Flexible Event Payload: it's your event, put anything you want inside.\n\n### Contents\n- [How to use](#how-to-use)\n- [Simple API](#simple-api)\n  - [stream ( )](#stream--streamparams---storeprovider-)\n  - [createEvent ( )](#createevent--eventparams---eventdata-)\n  - [append ( )](#append--events---stream-)\n  - [read ( )](#read--readparams---stream-)\n- [Optimistic Concurrency Control (OCC)](#optimistic-concurrency-control-occ)\n- [Concurrency with append() and read()](#concurrency-with-append-and-read)\n- [Event Sourcing helpers](#event-sourcing-helpers)\n  - [aggregate ()](#aggregate-aggregateparams-stream)\n  - [evolveWith ()](#evolvewith-evolvers-state-event)\n- [Snapshots](#snapshots)\n- [StoreProvider Interface](#storeprovider-interface)\n  - [append ( )](#append-streamparams-events)\n  - [read ( )](#read-streamparams-readparams)\n\n## How to use:\n\n```js\nconst { append, createEvent, read, stream } = require(\"kylfil\");\nconst postgres = require(\"kylfil/postgres\")(dbConnection);\n\nconst myStream = stream (\"be2ed21cad4b412f69c558510112262f\") (postgres);\n\nconst sale = {\n  airline: \"Air Canada\",\n  tkt: \"7712935657218\",\n  pax: \"MRS HAZEL NUTT\",\n  saleDate: \"2025-08-17\",\n};\n\nconst newEvent = createEvent (\"AirTicketSold\") (sale);\n  \n(async () =\u003e {\n  const [ storedEvent ] = await append (newEvent) (myStream);\n\n  const events = await read ({ maxCount: 5 }) (myStream);\n})()\n```\nThe `append()` function above will return an array with the newly appended event that now has a `streamId`, `version` at `0`, since it's the 1st event in the stream and a sequence number `seq` indicating _(Nth)_ position in the global event log:\n```js\n[{\n  seq: 15,\n  id: \"e2befa9a-141d-46fa-8f70-8fc48a7a7bbc\",\n  type: \"AirTicketSold\",\n  streamId: \"be2ed21cad4b412f69c558510112262f\",\n  version: 0,\n  data: {\n    airline: \"AC\",\n    tkt: \"7712935657218\",\n    pax: \"MRS HAZEL NUTT\",\n    saleDate: \"2025-08-17\"\n  }\n}]\n```\n\n## Simple API:\n\n### stream ( streamParams ) ( storeProvider )\n`stream :: String | StreamParams -\u003e StoreProvider -\u003e EventStream`\n\nUse this function to get a hold of a specific `EventStream` which can be supplied to other functions like `append()` or `read()` to work with events in a given stream.\n\n#### Arguments:\n\n- **streamParams** argument can be either a `String` indicating a `streamId` or a `StreamParams` object with the following properties:\n\n  | Name | Type | Default | Description |\n  | ---- |:----:|:-------:| ----------- |\n  | `storageName` | String | \"events\" | Indicates where events are stored. For RDBMS can be used as a table name. For Key-Value stores can be used as a key prefix. |\n  | `streamId` | String | N/A | Unique stream ID |\n\n- **storeProvider** is an implementation of the `StoreProvider` interface for working with the underlying persistence storage engine.\n\n#### Returns:\nAn instance of the `EventStream` function which accepts a single callback and invokes it with the following arguments:\n```js\n(cb) =\u003e cb (StreamParams) (StoreProvider)\n```\n\n### Examples:\n1\\. Return a stream by a given ID:\n```js\nstream(\"be2ed21cad4b412f69c558510112262f\")\n```\n2\\. For streams that are stored in dedicated DB tables for example `order_events` and `sales_events` you can do the following:\n```js\nconst orderStream = stream({ \n  storageName: \"order_events\",\n  streamId: \"766b8aa93d71e3f460b0f2524e1d271c\"\n}) \n\nconst saleStream = stream({ \n  storageName: \"sale_events\",\n  streamId: \"d1c61832169d7e053a96969e90f6b54b\"\n})\n```\n\n### createEvent ( eventParams ) ( eventData )\n`createEvent :: String | EventParams -\u003e a -\u003e Event`\n\nThis function creates events with a proper structure, suitable for appending to any stream in the event store.\n\n#### Arguments:\n\n- **eventParams** can be a `String` indicating the `eventType` or an `EventParams` object with the following properties:\n  \n  | Name | Type | Default | Description |\n  | ---- |:----:|:-------:| ----------- |\n  | `idGenerator` | eventData -\u003e String | randomUUID() | Function to generate event ID that receives `eventData` is an argument |\n  | `type` | String | N/A | Event type |\n  | `version` | Integer | `0` | Stream version |\n\n- **eventData** payload for the event data can be of any type.\n\n#### Returns:\nAn `Event` object.\n\n#### Examples:\n1\\. Create an event representing airline ticket sale:\n```js\nconst event = createEvent (\"AirTicketSold\")({\n  airline: \"AC\",\n  tkt: \"7712935657218\",\n  pax: \"MRS HAZEL NUTT\",\n  saleDate: \"2025-08-17\"\n})\n\n// event will be:\n{\n  id: \"6675e754-6dd9-4ac8-bfef-56532bfc9505\",\n  type: \"AirTicketSold\",\n  version: 0,\n  data: {\n    airline: \"AC\",\n    tkt: \"7712935657218\",\n    pax: \"MRS HAZEL NUTT\",\n    saleDate: \"2025-08-17\"\n  }\n}\n```\n2\\. Create an event at version 3 using a custom ID generator:\n```js\nconst event = createEvent ({\n  idGenerator: ({ airline, tkt }) =\u003e `${airline}-${tkt}`,\n  type: \"AirTicketSold\",\n  version: 3\n})({\n  airline: \"AC\",\n  tkt: \"7712935657218\",\n  pax: \"MRS HAZEL NUTT\",\n  saleDate: \"2025-08-17\"\n})\n\n// event will be:\n{\n  id: \"AC-7712935657218\",\n  type: \"AirTicketSold\",\n  version: 3,\n  data: {\n    airline: \"AC\",\n    tkt: \"7712935657218\",\n    pax: \"MRS HAZEL NUTT\",\n    saleDate: \"2025-08-17\"\n  }\n}\n```\n3\\. Generate sequence of events for one sale:\n```js\n[\n  { type: \"ProductSold\", version: 0 },\n  { type: \"ReceiptPrinted\", version: 1 },\n  { type: \"ProductShipped\", version: 2 }\n].map(params =\u003e createEvent(params)(sale))\n\n```\n4\\. Generate a list of a certain event type for a list of entities:\n```js\nsales.map(createEvent(\"ProductSold\"))\n```\n\n### append ( events ) ( stream )\n`append :: Event | Array Event -\u003e EventStream -\u003e Promise Array StoredEvent`\n\nUse this function to append events to a given stream.\n\n#### Arguments:\n\n- **events** can be either a single `Event` object or an `Array` of `Event` objects to append to a stream.\n- **stream** is an `EventStream` to append events to.\n\n\u003e **\u0026#128712; NOTE:** This function will store the events using the ID of the **stream** regardless of the `streamId` of the events. The original event objects won't be changed because the library performs all operations immutably.\n\n#### Returns:\nA `Promise` containing an array of `StoredEvent` objects with the following properties.\n  \n  | Name | Type | Description |\n  | ---- |:----:| ----------- |\n  | `seq` | Integer | Event's position in the global event log |\n  | `id` | String | Event ID |\n  | `type` | String  | Event type |\n  | `streamId` | String | Stream ID |\n  | `version` | Integer | Stream version |\n  | `data` | Any | Event payload |\n\nThrows an `OccError` that includes the current `streamVersion` if it fails due to an OCC conflict.\n\n#### Examples:\n1\\. Append two events to the stream:\n```js\nconst [storedEvent1, storedEvent2] = await append ([ event1, event2 ]) (myStream)\n```\n2\\. Append same events to multiple streams concurrently:\n```js\nconst results = await Promise.allSettled(\n  [myStream, andYourStream].map( append ([ event1, event2 ]) )\n)\n```\n\n### read ( readParams ) ( stream )\n`read :: ReadParams -\u003e EventStream -\u003e Promise Array StoredEvent`\n\nUse this function to read events from a stream in any direction and/or from a specific version. You can also set the max number of events to return.\n\n#### Arguments:\n\n- **readParams** is of type `ReadParams` with the following properties:\n\n  | Name | Type | Default | Description |\n  | ---- |:----:|:-------:| ----------- |\n  | `direction` | ReadDirection | FORWARD | Indicates the direction of the read operation. Can be `FORWARD` or `BACKWARD` |\n  | `fromVersion` | Integer \u003e= `0` | `0` | Indicates an inclusive version number of the events to read from. |\n  | `maxCount` | Integer \u003e `0` | N/A | Indicates the max number of events to return from stream |\n\n- **stream** is an `EventStream` to read events from.\n\n#### Returns:\nA `Promise` containing an array of `StoredEvent` objects with the following properties.\n  \n  | Name | Type | Description |\n  | ---- |:----:| ----------- |\n  | `seq` | Integer | Event's position in the global event log |\n  | `id` | String | Event ID |\n  | `type` | String  | Event type |\n  | `streamId` | String | Stream ID |\n  | `version` | Integer | Stream version |\n  | `data` | Any | Event payload |\n\n\u003e **\u0026#x26A1; TIP:** Partial application of `read()` function creates reusable readers that can be applied to many streams\n\n#### Examples:\n1\\. Read all events from a stream:\n```js\nconst results = await read () (myStream)\n// results will be:\n[\n  {\n    seq: 15,\n    id: \"e2befa9a-141d-46fa-8f70-8fc48a7a7bbc\",\n    type: \"AirTicketSold\",\n    streamId: \"be2ed21cad4b412f69c558510112262f\",\n    version: 0,\n    data: {\n      airline: \"AC\",\n      tkt: \"7712935657218\",\n      pax: \"MRS HAZEL NUTT\",\n      saleDate: \"2025-08-17\"\n    }\n  }\n  ... // other events in this stream\n]\n```\n2\\. Read events after version 12 _(inclusive)_:\n```js\nawait read ({ fromVersion: 12 }) (myStream)\n```\n3\\. Read latest 5 events from the stream:\n```js\nawait read ({ direction: ReadDirection.BACKWARD, maxCount: 5 }) (myStream)\n```\n4\\. Create reusable readers via partial application to read events from different streams concurrently:\n```js\nconst readLast5 = read ({\n  direction: ReadDirection.BACKWARD,\n  maxCount: 5\n})\n\nconst results = await Promise.allSettled(\n  [myStream, andYourStream].map(readLast5)\n)\n```\n\n### Optimistic Concurrency Control (OCC)\n\nOptimistic Concurrency Control prevents appending events with conflicting `id` or a composite key of `(streamId + version)`. A conflict throws an `OccError` that includes the stream's current `streamVersion`. Because reference implementations for storage engines provided with this library use \"multi-value\" inserts, the entire append process is an atomic transaction: if one event fails to append, none will be stored. Following example shows the difference:\n```js\n// no events will be stored due to OccError\nawait append ([ sameEvent, sameEvent ]) (myStream)\n\n// here the first append will succeed \nawait append (sameEvent) (myStream)\n// the second will fail due to OccError\nawait append (sameEvent) (myStream)\n```\nIn the following simulation of a racing condition where different events have the same version `3` only one will succeed and the other will throw `OccError`:\n```js\ncons results = await Promise.allSettled([\n  eventVersion3,\n  anotherEventVersion3\n].map(event =\u003e append(event)(myStream)))\n\nresults\n  .map((res, index) =\u003e {\n    if (res.status === 'rejected') {\n      const err = res.reason\n      console.error(\n        \"Failed appending event #\", index+1,\n        \"at streamVersion: \", err.streamVersion,\n        \"due to:\", err.name\n      )\n      // will print:\n      // \"Failed appending event #X at streamVersion: 3 due to: OccError\"\n    }\n  })\n```\n\n## Concurrency with append() and read()\n\n- Calling `append()` or `read()` concurrently can consume multiple DB connections, depending on how you provide these connections to the `StoreProvider`.\n- If you're using a `StoreProvider` implementation that supports a connection `Pool` make sure you have set a conection limit.\n- When the connection pool limit is reached, concurrent calls to `append()` or `read()` are processed sequentially.\n\n## Event Sourcing helpers\n\n### aggregate (aggregateParams) (stream)\n`aggregate :: AggregateParams -\u003e EventStream -\u003e Promise (Integer, State)`\n\nUse this function to get the current state of the entity represented by the stream. By default it reads all events in the stream to rebuild the entity state. However, if you provide a `version` in `AggregateParams` it will only read events that have occured since that version _(i.e. version + 1)_. You can pass an `evolve` function to `AggregateParams` which reduces a list of events into a resulting entity state. By default it will use a `NoFold` evolve which appends each event to the state where the state is a list of events.\n\n### Arguments:\n- **aggregateParams** is of type `AggregateParams` with the following properties:\n\n  | Name | Type | Default | Description |\n  | ---- |:----:|:-------:| ----------- |\n  | `evolve` | State -\u003e Event -\u003e State | NoFold | Reducer function to build the stream state |\n  | `initialState` | Any | [ ] | Initial state of the entity |\n  | `version` | Integer \u003e= `0` | N/A | Current version of the entity |\n\n- **stream** is an `EventStream` to read events from.\n\n### Returns:\nA `Promise` containing a Pair `(Integer, State)` _(i.e. array)_ where the 1st element is an entity version and 2nd is an entity state.\n\n#### Examples:\n1\\. Aggregate events to rebuild the state of an order:\n```js\nconst evolve = order =\u003e event =\u003e\n  event.type === \"OrderItemAdded\"\n    ? { ...order, items: [...order.items, event.data] }\n    : event.type === \"OrderItemRemoved\"\n      ? removeItemById(event.data.id)(order)\n      : order;\n\nconst initialState = {\n  id: \"my-order\",\n  createdDate: \"2025-04-01\",\n  items: []\n}\n\nconst [version, order] = await aggregate({\n  evolve,\n  initialState,\n  version: 0\n})(stream)\n\n// assuming events were:\n// 1) OrderItemAdded: \"First Item\"\n// 2) OrderItemAdded: \"Second Item\"\n// 3) OrderItemRemoved: \"First Item\"\n// version will be 3 and order will be:\n{\n  id: \"my-order\",\n  createdDate: \"2025-04-01\",\n  items: [\n    { id: \"item-2-id\", name: \"Second Item\" },\n  ]\n}\n```\n\n### evolveWith (evolvers) (state) (event)\n`evolveWith :: StrMap (State -\u003e Event -\u003e State) -\u003e State -\u003e Event -\u003e State`\n\nThis is a convenience function that helps to quickly setup an `evolve` function in an easy and declarative way. It allows mapping specific event types to their respective handlers using the `evolvers` argument. The resulting `evolve` function can be passed into an aggregate for handling events.\n\n### Arguments:\n- **evolvers** is an object that maps event types to their handlers with a signature `State -\u003e Event -\u003e State`.\n- **state** is a current state of the aggregate\n- **event** is an event to be processed\n\n### Returns:\nA new `evolve` function with the following signature `State -\u003e Evolve -\u003e State` which produces next state by applying the event to the current state.\n\n#### Examples:\n1\\. Create evolve function for order event handling:\n```js\n// create evolve function\nconst evolveOrder = evolveWith({\n  OrderCreated: () =\u003e (event) =\u003e Order(event.data),\n  OrderItemAdded: (order) =\u003e (event) =\u003e ({\n    ...order,\n    items: [...order.items, event.data],\n  }),\n  OrderItemRemoved: (order) =\u003e (event) =\u003e ({\n    ...order,\n    items: order.items.filter(({ id }) =\u003e id !== event.data.id),\n  }),\n});\n\nconst order = { id: \"my-order\", items: [] }\nconst itemAddedEvent = createEvent(\"OrderItemAdded\")({\n  id: \"some-item\",\n  name: \"Some Item\"\n})\n\nconst updatedOrder = evolveOrder(order)(itemAddedEvent);\n// updatedOrder will be:\n{\n  id: \"my-order\",\n  items: [\n    { id: \"some-item\", name: \"Some Item\" }\n  ]\n}\n\n// can be passed into aggregate:\nawait aggregate({ evolve: evolveOrder, initialState: order })\n```\n\n## Snapshots\nThe `aggregate` function can be used for **Snapshotting** as follows:\n- Load the last known entity state from your \"Read View\".\n- Pass it as the `initialState` along with the `version` in the `AggregateParams`.\n- `aggregate` will load all the events from the stream since that version, and replay them using the `evolve` function producing the updated snapshot of the entity.\n- Save the new snapshot along with its version back to your \"Read View\".\n\nAssuming the aggregate example above we can do the following:\n```js\nconst snapshot = await findSnapshotById(orderId)\nconst [version, order] = await aggregate({\n  evolve,\n  initialState: snapshot.doc,\n  version: snapshot.version\n})(stream)\nawait saveSnapshot({ id: orderId, version, doc: order })\n```\n\u003csup\u003e*\u003c/sup\u003eSince the implementation of snapshot loading/saving is highly dependent on the user's choices and the storage engine used, these details have been omitted.\n\n## StoreProvider Interface\n\n### append (streamParams) (events)\n`append :: StreamParams -\u003e Array Event -\u003e Promise Array StoredEvent`\n\nThis function stores events in the underlying storage engine. It's partially applied by the library, deferring the database connection and query execution until the main `append()` function of the library is invoked. This helps to avoid premature resource allocation.\n\n#### Arguments:\n- **streamParams** is of type `StreamParams` and supplied by the `stream()` function of the library.\n- **events** is an `Array` of events and provided by the `append()` function of the library.\n\n#### Returns:\nA `Promise` containing an array of `StoredEvent` objects.\n\n### read (streamParams) (readParams)\n`read :: StreamParams -\u003e ReadParams -\u003e Promise Array StoredEvent`\n\nThis function retrieves events from the underlying storage engine. It's partially applied by the library deferring the database connection and query execution until the main `read()` function of the library is called. This helps to avoid premature resource allocation.\n\n#### Arguments:\n- **streamParams** is of type `StreamParams` and supplied by the `stream()` function of the library.\n- **readParams** is of type `ReadParams` and provided by the `read()` function of the library.\n\n#### Returns:\nA `Promise` containing an array of `StoredEvent` objects.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpolad%2Fkylfil","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpolad%2Fkylfil","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpolad%2Fkylfil/lists"}