{"id":13605324,"url":"https://github.com/CharlieTap/synk","last_synced_at":"2025-04-12T05:32:33.663Z","repository":{"id":80602316,"uuid":"545061152","full_name":"CharlieTap/synk","owner":"CharlieTap","description":"A Kotlin multiplatform CRDT library for offline first applications","archived":false,"fork":false,"pushed_at":"2023-11-22T14:18:47.000Z","size":399,"stargazers_count":53,"open_issues_count":2,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-08-02T19:37:12.264Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CharlieTap.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-10-03T18:04:44.000Z","updated_at":"2024-08-01T11:35:32.000Z","dependencies_parsed_at":"2023-11-22T16:02:01.436Z","dependency_job_id":null,"html_url":"https://github.com/CharlieTap/synk","commit_stats":null,"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CharlieTap%2Fsynk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CharlieTap%2Fsynk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CharlieTap%2Fsynk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CharlieTap%2Fsynk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CharlieTap","download_url":"https://codeload.github.com/CharlieTap/synk/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223497744,"owners_count":17155199,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T19:00:57.403Z","updated_at":"2024-11-07T10:30:25.242Z","avatar_url":"https://github.com/CharlieTap.png","language":"Kotlin","funding_links":[],"categories":["分布式开发","Kotlin"],"sub_categories":["Spring Cloud框架"],"readme":"# Synk\n\n![badge][badge-android]\n![badge][badge-jvm]\n\n---\n\nA Kotlin multiplatform CRDT library for building offline/local first applications.\n\n\nSynk supercharges client side databases to have distributed database properties such as:\n\n- Conflict Resolution\n- Causal Ordering\n\n_Allowing you to build offline first applications with the technologies you're familiar with._\n\n\n\n# How does it work?\n\nSynk is a state based CRDT library, it monitors state changes over time using a special type of [timestamp](https://github.com/CharlieTap/hlc) which is capable\nof tracking events in a distributed system (your application). Synk maintains this data in its own persistent key value storage database locally on each client.\nIt's important to understand Synk **does not** store your data, merely it stores timestamps associated with it.\n\nUnlike most CRDT libraries Synk works entirely agnostic to your data persistence technology, you can think of it as\nsomething that can be bolted on to your application as opposed to something that would replace your core data layer\ntechnology. What this means in practice is you can use whatever data persistence technology you like (Maybe Room or\nSQLDelight) and Synk can complement it by tracking extra metadata which is crucial for resolving conflicts.\n\nSynk behaves as a middle man between your local database and other databases in your application, for this reason\nit exposes two functions:\n\n### Outbound\n\n```kotlin\nSynk.outbound(new: T, old: T? = null): Message\u003cT\u003e\n```\n\nWhenever a new record/object is created or updated in your application locally, give synk the latest version and the old version (if applicable) and synk will return you a message.\nThis message needs to be propagated to all other clients.\n\n### Inbound\n\n```kotlin\nSynk.inbound(message: Message\u003cT\u003e, old: T? = null): T\n```\n\nWhen receiving a Message from another client application, inbound needs to be called. This function will perform conflict resolution for you and return an instance of your object ready to\nbe persisted.\n\nTypical usage can be seen as a loop where Inbound completes the process started by Outbound, for example:\n\n1. Client A database creates a Foo record\n2. Synk running on Client A is alerted to this change through the `outbound` function creating a Message\n3. Message is propagated to the Server from Client A as part of a synchronisation process\n4. Client B runs syncronisation process and receives the Message\n5. Synk running on Client B is alerted to the message through the `inbound` function, this performs conflict resolution\nand returns an object\n6. Object is inserted into Client B's database, replacing any previous state that may exist\n\n```mermaid\ngraph TD;\n Server--InboundMessage--\u003eClientA;\n Server--InboundMessage--\u003eClientB;\n ClientA--OutboundMessage--\u003eServer;\n ClientB--OutboundMessage--\u003eServer;\n```\n\n\n# How should I architect my application using Synk\n\nSynk is intentionally minimal and unopinionated in design, but there are of course some constraints that come from building an application with it.\nThe two that stand out are the following:\n\n- Messages must be relayed to all nodes in order for state to be consistent\n- Data exposed to Synk can never be deleted, at least not in the short term, soft deletes using tombstone fields is the recommended solution.\n\n\nOffline first applications that mutate state are distributed systems, there's no two ways about it. For this reason Synk has no concept of server like central storage.\nSynk sees the world how any node in a distributed system would.\n\n```mermaid\ngraph LR;\n NodeA[(NodeA)]\n NodeB[(NodeB)]\n NodeC[(NodeC)]\n NodeA--Message--\u003eNodeB;\n NodeB--Message--\u003eNodeC;\n NodeC--Message--\u003eNodeA;\n```\n# API Usage\n\n---\n\n## Gradle\n\nSynk currently uses Jitpack to distribute artifacts, you'll need to ensure that the jitpack maven repo is configured\nin you dependency resolution management block.\n\n```kotlin\ndependencyResolutionManagement {\n    repositories {\n        ...\n        maven(url = \"https://jitpack.io\" )\n```\n\nYou'll need both Synk runtime and the delightful metastore artifacts to get started:\n\n```kotlin\ndependencies {\n    implementation(\"com.github.charlietap.synk:delight-metastore:xxx\")\n    implementation(\"com.github.charlietap.synk:synk:xxx\")\n}\n```\nAlternatively if you're working with a KMP project you can pull the specialised dependencies for the different targets:\n\n```kotlin\ndependencies {\n    implementation(\"com.github.charlietap.synk:synk-android:xxx\")\n    //or\n    implementation(\"com.github.charlietap.synk:synk-jvm:xxx\")\n}\n```\n\n## Synk\n\nSynk maintains two pieces of state in order to function:\n\n- A logical clock, the location of this file is configured through ClockStorageConfiguration\n- A key value database called the MetaStore, you can provide an instance of the DelightfulMetastoreFactory. This factory\nuses Sqldelight under the hood and needs a Sqldriver to function. Depending on your platform you will need to provide the\nappropriate driver, you can read how to do this [here](https://cashapp.github.io/sqldelight/1.5.4/multiplatform_sqlite/)\n\n```kotlin\nval clockStorageConfig = ClockStorageConfiguration(\n    filePath = \"/synk\".toPath(),\n    fileSystem = FileSystem.SYSTEM\n)\nval factory = DelightfulMetastoreFactory(driver)\nval synk = Synk.Builder(clockStorageConfig)\n    .metaStoreFactory(factory)\n    .build()\n```\n\nIf you're on Android a preset extension function exists for the builder which configures the clock storage configuration for you:\n\n```kotlin\nval synk = Synk.Builder.Presets.Android(context)\n.metaStoreFactory(metastoreFactory)\n.build()\n```\n\n\n### Synk Adapters\n\nSynk adapters tell synk how to serialize and deserialize types into generic maps it can perform conflict resolution on. For\nevery type T you intend to use with Synk you must provide a SynkAdapter\u003cT\u003e when constructing your synk instance.\n\n```kotlin\nval synk = Synk.Builder(...)\n    .registerSynkAdapter(adapter)\n    .registerSynkAdapter(adapter2)\n    .build()\n```\n\nFor more information on Synk adapters please visit the [documentation page](docs/synk-adapters.md)\n\n## Conflict Resolution, Causal Ordering and Messages\n\nSynk resolves conflicts by recording a causal order of events in the Metastore. Calls to the `inbound` and `outbound`\nfunctions are checked against the current state of the Metastore and conflict resolution is performed.\n\nLocal events are events that occur on the current node (client application), notify Synk of the event by passing the new\nand old (in the case of updates) to outbound once they have already been persisted to the database.\n\nSynk will return you a Message, it's your responsibility to ensure all nodes receive all messages relevant to them.\n\n```kotlin\nSynk.outbound(new: T, old: T? = null): Message\u003cT\u003e\n```\n\n`inbound` is the ying to `outbound`'s yang, and the destination for the Messages `outbound` creates. The result of inbound is an object ready to be inserted into a database, free of conflicts and consistent across all nodes.\n\n\n```kotlin\nSynk.inbound(message: Message\u003cT\u003e, old: T? = null): T\n```\n\n## Change Detection\n\nComing soon ...\n\n## Serialization\n\nTo aid the relay of messages between applications Synk provides methods for serializing Messages to and from json.\nThese serializers make use of the Synk adapters provided and tend to have better performance than reflections powered\nserializers like gson.\n\n```kotlin\nSynk.serialize(messages: List\u003cMessage\u003cT\u003e\u003e): String\n```\n\n```kotlin\nSynk.deserialize(encoded: String): List\u003cMessage\u003cT\u003e\u003e\n```\n\nFor those looking to serialize messages themselves, Synk exposes Message/Meta Serializers for popular libs:\n\n- [Kotlin Serialization](docs/extensions.md#kotlinx-serialization)\n\n## Compaction\n\nMessages generated by Synk are commutative, associative and idempotent. This means that you can combine the messages in any order you want,\nhowever many times as you want, switching which messages combine with each other, and you will always deterministically get the same result.\n\nThis affords us the ability to merge a series of messages for a particular object into just one, it's latest state. You could for example batch all the messages\nregarding a particular object whilst offline, then compact all the changes into one Message before relaying this information to other nodes.\n\nTaking this idea further you could store all the Messages ever created in an Event Store, and compact them to derive the current state of the system.\n\n\n```kotlin\nSynk.compact(messages: List\u003cMessage\u003cT\u003e\u003e): List\u003cMessage\u003cT\u003e\u003e\n```\n\n## Testing\n\nSynk is designed from the ground up to be testable, by default Synk will use in memory metastore for metadata persistence.\nYou can easily configure a test friendly instance as follows:\n\n```kotlin\nval storageConfiguration = StorageConfiguration(\n    filePath = \"/test\".toPath(),\n    fileSystem = FakeFileSystem()\n)\n\nval testSynk = Synk.Builder(storageConfiguration)\n    .build()\n```\n\n## Additional Reading\n\n- [Synking all the things with CRDTs](https://dev.to/charlietap/synking-all-the-things-with-crdts-local-first-development-3241)\n\n\n[badge-android]: http://img.shields.io/badge/-android-6EDB8D.svg?style=flat\n[badge-jvm]: http://img.shields.io/badge/-jvm-DB413D.svg?style=flat\n[badge-js]: http://img.shields.io/badge/-js-F8DB5D.svg?style=flat\n[badge-linux]: http://img.shields.io/badge/-linux-2D3F6C.svg?style=flat\n[badge-windows]: http://img.shields.io/badge/-windows-4D76CD.svg?style=flat\n[badge-ios]: http://img.shields.io/badge/-ios-CDCDCD.svg?style=flat\n[badge-mac]: http://img.shields.io/badge/-macos-111111.svg?style=flat\n\n\n## License\n\nThis project is dual-licensed under both the MIT and Apache 2.0 licenses. You can choose which one you want to use the software under.\n\n- For details on the MIT license, please see the [LICENSE-MIT](LICENSE-MIT) file.\n- For details on the Apache 2.0 license, please see the [LICENSE-APACHE](LICENSE-APACHE) file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCharlieTap%2Fsynk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FCharlieTap%2Fsynk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCharlieTap%2Fsynk/lists"}