{"id":13630439,"url":"https://github.com/ipfs/go-ds-crdt","last_synced_at":"2025-10-24T10:31:54.772Z","repository":{"id":37484481,"uuid":"172116110","full_name":"ipfs/go-ds-crdt","owner":"ipfs","description":"A distributed go-datastore implementation using Merkle-CRDTs.","archived":false,"fork":false,"pushed_at":"2025-03-24T12:37:39.000Z","size":661,"stargazers_count":414,"open_issues_count":8,"forks_count":46,"subscribers_count":24,"default_branch":"master","last_synced_at":"2025-04-11T00:52:15.894Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ipfs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-22T18:33:15.000Z","updated_at":"2025-04-04T02:15:58.000Z","dependencies_parsed_at":"2023-12-21T16:10:34.243Z","dependency_job_id":"c5df67b1-d0c4-4e1f-9a8c-24f89a40a003","html_url":"https://github.com/ipfs/go-ds-crdt","commit_stats":{"total_commits":260,"total_committers":16,"mean_commits":16.25,"dds":"0.27692307692307694","last_synced_commit":"ff922d8ad65a162c69e793b72f4f6622e7f88008"},"previous_names":[],"tags_count":60,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipfs%2Fgo-ds-crdt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipfs%2Fgo-ds-crdt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipfs%2Fgo-ds-crdt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ipfs%2Fgo-ds-crdt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ipfs","download_url":"https://codeload.github.com/ipfs/go-ds-crdt/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248322609,"owners_count":21084336,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T22:01:42.841Z","updated_at":"2025-10-24T10:31:54.709Z","avatar_url":"https://github.com/ipfs.png","language":"Go","funding_links":[],"categories":["Go","\u003ca name=\"Go\"\u003e\u003c/a\u003eGo"],"sub_categories":[],"readme":"# go-ds-crdt\n\n\u003e A distributed [go-datastore](https://github.com/ipfs/go-datastore)\n\u003e implementation using Merkle-CRDTs.\n\n`go-ds-crdt` is a key-value store implementation using Merkle CRDTs, as\ndescribed in\n[the paper by Héctor Sanjuán, Samuli Pöyhtäri and Pedro Teixeira](https://arxiv.org/abs/2004.00107).\nIt satisfies the\n[`Datastore`](https://pkg.go.dev/github.com/ipfs/go-datastore#Datastore)\nand [`Batching`](https://pkg.go.dev/github.com/ipfs/go-datastore#Batching)\ninterfaces from `go-datastore`.\n\nThis means that you can create a network of nodes that use this datastore, and \nthat each key-value pair written to it will automatically replicate to every\nother node. Updates can be published by any node. Network messages can be dropped, \nreordered, corrupted or duplicated. It is not necessary to know beforehand\nthe number of replicas participating in the system. Replicas can join and leave \nat will, without informing any other replica. There can be network partitions \nbut they are resolved as soon as connectivity is re-established between replicas.\n\nInternally it uses a delta-CRDT Add-Wins Observed-Removed set. The current\nvalue for a key is the one with highest priority. Priorities are defined as\nthe height of the Merkle-CRDT node in which the key was introduced.\n\nImplementation is independent from Broadcaster and DAG syncer layers, although the \neasiest is to use out of the box components from the IPFS stack (see below).\n\n## Performance\n\nUsing batching, Any `go-ds-crdt` replica can easily process and sync 400 keys/s at least. The largest known deployment has 100M keys.\n\n`go-ds-crdt` is used in production as state-synchronization layer for [IPFS Clusters](https://ipfscluster.io).\n\n## Usage\n\n`go-ds-crdt` needs:\n  * A user-provided, thread-safe,\n    [`go-datastore`](https://github.com/ipfs/go-datastore) implementation to\n    be used as permanent storage. We recommend using the\n    [Badger implementation](https://pkg.go.dev/github.com/ipfs/go-ds-badger).\n  * A user-defined `Broadcaster` component to broadcast and receive updates\n    from a set of replicas. If your application uses\n    [libp2p](https://libp2p.io), you can use\n    [libp2p PubSub](https://pkg.go.dev/github.com/libp2p/go-libp2p-pubsub) and\n    the provided\n    [`PubsubBroadcaster`](https://pkg.go.dev/github.com/ipfs/go-ds-crdt?utm_source=godoc#PubSubBroadcaster).\n  * A user-defined \"DAG syncer\" component ([`ipld.DAGService`](https://pkg.go.dev/github.com/ipfs/go-ipld-format?utm_source=godoc#DAGService)) to publish and\n    retrieve Merkle DAGs to the network. For example, you can use\n    [IPFS-Lite](https://github.com/hsanjuan/ipfs-lite) which casually\n    satisfies this interface.\n\nThe permanent storage layout is optimized for KV stores with fast indexes and\nkey-prefix support.\n\nSee https://pkg.go.dev/github.com/ipfs/go-ds-crdt for more information.\n\n## Captain\n\nThis project is captained by @hsanjuan.\n\n## License\n\nThis library is dual-licensed under Apache 2.0 and MIT terms.\n\nCopyright 2019. Protocol Labs, Inc.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipfs%2Fgo-ds-crdt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fipfs%2Fgo-ds-crdt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fipfs%2Fgo-ds-crdt/lists"}