{"id":16639857,"url":"https://github.com/dzoukr/cosmostore","last_synced_at":"2025-03-16T22:31:21.452Z","repository":{"id":44689631,"uuid":"144550544","full_name":"Dzoukr/CosmoStore","owner":"Dzoukr","description":"F# Event store for Azure Cosmos DB, Table Storage, Postgres, LiteDB \u0026 ServiceStack","archived":false,"fork":false,"pushed_at":"2022-01-31T06:27:36.000Z","size":654,"stargazers_count":168,"open_issues_count":13,"forks_count":21,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-10-13T07:07:19.228Z","etag":null,"topics":["cosmosdb","eventsourcing","eventstore","fsharp","litedb","postgres","servicestack","tablestorage"],"latest_commit_sha":null,"homepage":"","language":"F#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Dzoukr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-08-13T08:25:38.000Z","updated_at":"2023-05-20T19:36:18.000Z","dependencies_parsed_at":"2022-08-12T11:21:22.253Z","dependency_job_id":null,"html_url":"https://github.com/Dzoukr/CosmoStore","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dzoukr%2FCosmoStore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dzoukr%2FCosmoStore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dzoukr%2FCosmoStore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Dzoukr%2FCosmoStore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Dzoukr","download_url":"https://codeload.github.com/Dzoukr/CosmoStore/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221668510,"owners_count":16860664,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cosmosdb","eventsourcing","eventstore","fsharp","litedb","postgres","servicestack","tablestorage"],"created_at":"2024-10-12T07:07:14.462Z","updated_at":"2024-10-27T11:27:52.992Z","avatar_url":"https://github.com/Dzoukr.png","language":"F#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CosmoStore\r\n\r\n\u003cp align=\"center\"\u003e\r\n\u003cimg src=\"https://github.com/Dzoukr/CosmoStore/raw/master/logo.png\" width=\"150px\"/\u003e\r\n\u003c/p\u003e\r\n\r\nF# Event Store library for various storage providers (Cosmos DB, Table Storage, Marten, InMemory and LiteDB)\r\n\r\n## Features\r\n- Storage agnostic F# API\r\n- Support for Azure Cosmos DB\r\n- Support for Azure Table Storage\r\n- Support for Marten\r\n- Support for In-memory\r\n- Support for LiteDB\r\n- Optimistic concurrency\r\n- ACID compliant\r\n- Simple Stream querying\r\n\r\n\r\n## Available storage providers\r\n\r\n| Storage Provider | Payload type | Package | Version | Author\r\n|---|---|---|---|---|\r\n| none (API definition only) | - | CosmoStore | [![NuGet](https://img.shields.io/nuget/v/CosmoStore.svg?style=flat)](https://www.nuget.org/packages/CosmoStore/) | @dzoukr |\r\n| Azure Cosmos DB | `Newtonsoft.Json` | CosmoStore.CosmosDb | [![NuGet](https://img.shields.io/nuget/v/CosmoStore.CosmosDb.svg?style=flat)](https://www.nuget.org/packages/CosmoStore.CosmosDb/) |@dzoukr |\r\n| Azure Table Storage | `Newtonsoft.Json` | CosmoStore.TableStorage  | [![NuGet](https://img.shields.io/nuget/v/CosmoStore.TableStorage.svg?style=flat)](https://www.nuget.org/packages/CosmoStore.TableStorage/) | @dzoukr |\r\n| InMemory | `Newtonsoft.Json` | CosmoStore.InMemory  | [![NuGet](https://img.shields.io/nuget/v/CosmoStore.InMemory.svg?style=flat)](https://www.nuget.org/packages/CosmoStore.InMemory/) | @kunjee\r\n| Marten | `Newtonsoft.Json` | CosmoStore.Marten  | [![NuGet](https://img.shields.io/nuget/v/CosmoStore.Marten.svg?style=flat)](https://www.nuget.org/packages/CosmoStore.Marten/) | @kunjee\r\n| LiteDB | `BsonValue` / `BsonDocument` | CosmoStore.LiteDb  | [![NuGet](https://img.shields.io/nuget/v/CosmoStore.LiteDb.svg?style=flat)](https://www.nuget.org/packages/CosmoStore.LiteDb/) | @kunjee\r\n| ServiceStack | `'a` | CosmoStore.ServiceStack  | [![NuGet](https://img.shields.io/nuget/v/CosmoStore.ServiceStack.svg?style=flat)](https://www.nuget.org/packages/CosmoStore.ServiceStack/) | @kunjee\r\n\r\n## What is new in version 3\r\n\r\nAll previous version of CosmoStore were tightly connected with `Newtonsoft.Json` library and used its `JToken` as default payload for events. Since version 3.0 this does not apply anymore.\r\nWhole definition of `EventStore` was rewritten to be fully generic on payload and also on version level. Why? Some libraries not only use different payload than `JToken`, but possibly use\r\ndifferent type for `Version` then `int64` (default before version 3). Authors of libraries using `CosmoStore` API now can use any payload and any version type that fits best their storage mechanism.\r\n\r\n## Event store\r\n\r\nEvent store (defined as F# record) is by design *storage agnostic* which means that no matter if you use Cosmos DB or Table Storage, the API is the same.\r\n\r\n```fsharp\r\ntype EventStore\u003c'payload,'version\u003e = {\r\n    AppendEvent : StreamId -\u003e ExpectedVersion\u003c'version\u003e -\u003e EventWrite\u003c'payload\u003e -\u003e Task\u003cEventRead\u003c'payload,'version\u003e\u003e\r\n    AppendEvents : StreamId -\u003e ExpectedVersion\u003c'version\u003e -\u003e EventWrite\u003c'payload\u003e list -\u003e Task\u003cEventRead\u003c'payload,'version\u003e list\u003e\r\n    GetEvent : StreamId -\u003e 'version -\u003e Task\u003cEventRead\u003c'payload,'version\u003e\u003e\r\n    GetEvents : StreamId -\u003e EventsReadRange\u003c'version\u003e -\u003e Task\u003cEventRead\u003c'payload,'version\u003e list\u003e\r\n    GetEventsByCorrelationId : Guid -\u003e Task\u003cEventRead\u003c'payload,'version\u003e list\u003e\r\n    GetStreams : StreamsReadFilter -\u003e Task\u003cStream\u003c'version\u003e list\u003e\r\n    GetStream : StreamId -\u003e Task\u003cStream\u003c'version\u003e\u003e\r\n    EventAppended : IObservable\u003cEventRead\u003c'payload,'version\u003e\u003e\r\n}\r\n```\r\n\r\nEach function on record is explained in separate chapter.\r\n\r\n\r\n## Initializing Event store for Azure Cosmos DB\r\n\r\nCosmos DB Event store has own configuration type that follows some specifics of database like *Request units* throughput.\r\n\r\n```fsharp\r\ntype Configuration = {\r\n    DatabaseName : string\r\n    ContainerName : string\r\n    ConnectionString : string\r\n    Throughput : int\r\n    InitializeContainer : bool\r\n}\r\n```\r\n\r\n\u003e Note: If you don't know these terms check [official documentation](https://docs.microsoft.com/en-us/azure/cosmos-db/request-units)\r\n\r\nConfiguration can be created \"manually\" or use default setup (fixed collection with 400 RU/s)\r\n\r\n```fsharp\r\nopen CosmoStore\r\n\r\nlet cosmosDbUrl = Uri \"https://mycosmosdburl\" // check Keys section on Azure portal\r\nlet cosmosAuthKey = \"VeryPrivateKeyValue==\" // check Keys section on Azure portal\r\nlet myConfig = CosmosDb.Configuration.CreateDefault cosmosDbUrl cosmosAuthKey\r\n\r\nlet eventStore = myConfig |\u003e CosmosDb.EventStore.getEventStore\r\n```\r\n\r\n\r\n## Initializing Event store for Azure Table Storage\r\n\r\nConfiguration for Table Storage is much easier since we need only *account name* and *authentication key*.\r\n\r\n```fsharp\r\ntype StorageAccount =\r\n    | Cloud of accountName:string * authKey:string\r\n    | LocalEmulator\r\n\r\ntype Configuration = {\r\n    DatabaseName : string\r\n    Account : StorageAccount\r\n}\r\n\r\n```\r\n\r\nAs for Cosmos DB, you can easily create default configuration.\r\n\r\n```fsharp\r\nopen CosmoStore\r\n\r\nlet storageAccountName = \"myStoreageAccountName\" // check Keys section on Azure portal\r\nlet storageAuthKey = \"VeryPrivateKeyValue==\" // check Keys section on Azure portal\r\nlet myConfig = TableStorage.Configuration.CreateDefault storageAccountName storageAuthKey\r\n\r\nlet eventStore = myConfig |\u003e TableStorage.EventStore.getEventStore\r\n```\r\n\r\n## Writing Events to Stream\r\n\r\nEvents are data structures you want to write (append) to some \"shelf\" also known as *Stream*. Event for writing is defined as this type:\r\n\r\n```fsharp\r\ntype EventWrite\u003c'payload\u003e = {\r\n    Id : Guid\r\n    CorrelationId : Guid option\r\n    CausationId : Guid option\r\n    Name : string\r\n    Data : 'payload\r\n    Metadata : 'payload option\r\n}\r\n```\r\n\r\nWhen writing Events to some Stream, you usually expect them to be written having some version hence you must specify *optimistic concurrency* strategy. For this purpose the type `ExpectedVersion` exists:\r\n\r\n```fsharp\r\ntype ExpectedVersion\u003c'version\u003e =\r\n    | Any\r\n    | NoStream\r\n    | Exact of 'version\r\n```\r\n\r\nThere are two defined functions to write Event to Stream. `AppendEvent` for writing single Event and `AppendEvents` to write more Events.\r\n\r\n```fsharp\r\nlet expected = ExpectedVersion.NoStream // we are expecting brand new stream\r\nlet eventToWrite = ... // get new event to be written\r\nlet streamId = \"MyAmazingStream\"\r\n\r\n// writing first event\r\neventToWrite |\u003e eventStore.AppendEvent streamId expected \r\n\r\nlet moreEventsToWrite = ... // get list of another events\r\nlet newExpected = ExpectedVersion.Exact 2L // we are expecting next event to be in 2nd version\r\n\r\n// writing another N events\r\nmoreEventsToWrite |\u003e eventStore.AppendEvents streamId newExpected\r\n```\r\n\r\nIf everything goes well, you will get back list (in *Task*) of written events (type `EventRead` - explained in next chapter).\r\n\r\n\r\n## Reading Events from Stream\r\n\r\nWhen reading back Events from Stream, you'll a little bit more information than you wrote:\r\n\r\n```fsharp\r\ntype EventRead\u003c'payload,'version\u003e = {\r\n    Id : Guid\r\n    CorrelationId : Guid option\r\n    CausationId : Guid option\r\n    StreamId : StreamId\r\n    Version: 'version\r\n    Name : string\r\n    Data : 'payload\r\n    Metadata : 'payload option\r\n    CreatedUtc : DateTime\r\n}\r\n```\r\n\r\nYou have two options how to read back stored Events. You can read single Event by Version using `GetEvent` function:\r\n\r\n\r\n```fsharp\r\n// return 2nd Event from Stream\r\nlet singleEvent = 2L |\u003e eventStore.GetEvent \"MyAmazingStream\"\r\n```\r\n\r\nOr read list of Events using `GetEvents` function. For such reading you need to specify the *range*:\r\n\r\n```fsharp\r\n// return 1st-2nd Event from Stream\r\nlet firstTwoEvents = EventsReadRange.VersionRange(1,2) |\u003e eventStore.GetEvents \"MyAmazingStream\"\r\n\r\n// return all events\r\nlet allEvents = EventsReadRange.AllEvents |\u003e eventStore.GetEvents \"MyAmazingStream\"\r\n```\r\n\r\nTo fully understand what are the possibilities have a look at `EventsReadRange` definition:\r\n\r\n```fsharp\r\ntype EventsReadRange\u003c'version\u003e =\r\n    | AllEvents\r\n    | FromVersion of 'version\r\n    | ToVersion of 'version\r\n    | VersionRange of fromVersion:'version * toVersion:'version\r\n```\r\n\r\nIf you are interested in Events based on stored `CorrelationId`, you can use function introduced in version 2 - `GetEventsByCorrelationId`\r\n\r\n```fsharp\r\nlet myCorrelationId = ... // Guid value\r\nlet correlatedEvents = myCorrelationId |\u003e eventStore.GetEventsByCorrelationId\r\n```\r\n\r\n## Reading Streams from Event store\r\n\r\nEach Stream has own metadata:\r\n\r\n```fsharp\r\ntype Stream = {\r\n    Id : string\r\n    LastVersion : int64\r\n    LastUpdatedUtc : DateTime\r\n}\r\n```\r\n\r\nIf you know exact value of Stream `Id`, you can use function `GetStream`. To query more Streams, use `GetStreams` function. The querying works similar way as filtering Events by range, but here you can query Streams by `Id`:\r\n\r\n```fsharp\r\nlet allAmazingStream = StreamsReadFilter.StartsWith(\"MyAmazing\") |\u003e eventStore.GetStreams\r\nlet allStreams = StreamsReadFilter.AllStream |\u003e eventStore.GetStreams\r\n```\r\n\r\nThe complete possibilities are defined by `StreamsReadFilter` type:\r\n\r\n```fsharp\r\ntype StreamsReadFilter =\r\n    | AllStreams\r\n    | StartsWith of string\r\n    | EndsWith of string\r\n    | Contains of string\r\n```\r\n\r\n## Observing appended events\r\n\r\nSince version 1.4.0 you can observe appended events by hooking to `EventAppended` property `IObservable\u003cEventRead\u003e`. Use of [FSharp.Control.Reactive](https://www.nuget.org/packages/FSharp.Control.Reactive) library is recommended, but not required.\r\n\r\n## Comparison with Jet's Equinox?\r\nComing from Jet's Equinox? Please see amazing comment describing conceptual differences between `Equinox` and `CosmoStore` written by @bartelink: https://github.com/Dzoukr/CosmoStore/issues/6#issuecomment-477481777\r\n\r\n\r\n## Known issues (Azure Table Storage only)\r\n\r\nAzure Table Storage currently allows only *100 operations* (appends) in one batch. CosmoStore reserves one operation per batch for Stream metadata, so if you want to append more than *99 events* to single Stream, you will get `InvalidOperationException`.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdzoukr%2Fcosmostore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdzoukr%2Fcosmostore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdzoukr%2Fcosmostore/lists"}