{"id":15235515,"url":"https://github.com/nikgraf/secsync","last_synced_at":"2025-08-23T02:32:44.482Z","repository":{"id":38329689,"uuid":"440305390","full_name":"nikgraf/secsync","owner":"nikgraf","description":"Architecture for end-to-end encrypted CRDTs","archived":false,"fork":false,"pushed_at":"2024-09-21T19:42:31.000Z","size":11759,"stargazers_count":217,"open_issues_count":13,"forks_count":9,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-08-20T16:59:40.158Z","etag":null,"topics":["crdt","crdts","encryption","encryption-decryption","libsodium","protocol"],"latest_commit_sha":null,"homepage":"https://www.secsync.com/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nikgraf.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"nikgraf"}},"created_at":"2021-12-20T21:00:20.000Z","updated_at":"2025-08-13T19:26:11.000Z","dependencies_parsed_at":"2023-10-10T13:41:45.141Z","dependency_job_id":"3fb35bf4-a1e6-4187-b733-673428977cf6","html_url":"https://github.com/nikgraf/secsync","commit_stats":null,"previous_names":["serenity-kit/secsync","nikgraf/secsync","serenity-kit/naisho"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nikgraf/secsync","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikgraf%2Fsecsync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikgraf%2Fsecsync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikgraf%2Fsecsync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikgraf%2Fsecsync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nikgraf","download_url":"https://codeload.github.com/nikgraf/secsync/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikgraf%2Fsecsync/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271732383,"owners_count":24811312,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-23T02:00:09.327Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crdt","crdts","encryption","encryption-decryption","libsodium","protocol"],"created_at":"2024-09-29T08:01:46.726Z","updated_at":"2025-08-23T02:32:44.428Z","avatar_url":"https://github.com/nikgraf.png","language":"TypeScript","funding_links":["https://github.com/sponsors/nikgraf"],"categories":["TypeScript"],"sub_categories":[],"readme":"# secsync\n\nIs an architecture to relay end-to-end encrypted CRDTs over a central service.\n\nIt was created out of the need to have an end-to-end encrypted protocol to allow data synchronization/fetching incl. real-time updates to support [local-first](https://www.inkandswitch.com/local-first/) apps in combination with a web clients without locally stored data.\n\n**WARNING**: This is beta software.\n\n## Examples\n\n- End-to-end encrypted document using [Yjs](https://github.com/yjs/yjs) incl. Cursor Awareness\n- End-to-end encrypted todo list using [Automerge](https://github.com/automerge/automerge)\n\nTry them out at [https://www.secsync.com/](https://www.secsync.com/)\n\n## Concept\n\nThe architecture is built upon 4 building blocks:\n\n1. Document\n2. Snapshot\n3. Update\n4. Ephemeral message\n\nA _Document_ is defined by an **ID** and the **active Snapshot**.\n\nA _Snapshot_ includes the **encrypted CRDT document** at a certain time.\n\nAn _Update_ includes one or multiple **encrypted CRDT updates** referencing a snapshot.\n\nAn _Ephemeral message_ includes **encrypted data** referencing a document.\n\nIf you look at it from a perspective of the current state of one document it looks like this:\n\n\u003cimg src=\"./documentation/public/secsync-document-representation.png?raw=true\" width=\"323\" height=\"339\" alt=\"State of one document as snapshots and updates.\" /\u003e\n\nIf you look at it over time it looks like a tree that that always comes together once a snapshot is created:\n\n\u003cimg src=\"./documentation/public/secsync-time-representation.png?raw=true\" width=\"309\" height=\"521\" alt=\"State of one document as snapshots and updates.\" /\u003e\n\nWhen the server service persists an update it stores it with an integer based version number which is returned to every client. This way clients efficiently can ask for only the updates they haven't received.\n\n## Use Cases\n\n### Local-first only app\n\nIn this case each client per document has to keep the\n\n- Document ID\n- CRDT document\n- Active Snapshot ID\n- Active Snapshot latest server version integer\n\nBy sending the document ID, active Snapshot ID and the active Snapshot version integer the client will receive only the latest changes. This can be:\n\n- Updates\n- New active Snapshot + Updates\n\nIf all clients stay relatively up to date all the time new snapshots would be inefficient and not necessary. They might still be relevant e.g. when a collaborator is removed from the document and the encryption key is rotated.\n\n### Cloud based app\n\nIn this case the client only needs to know the document ID and can fetch the latest snapshot incl. the referencing snapshots to construct the document. Here it makes sense to regularly create new snapshots to avoid longer loading times.\n\n### Mixed app (local first + cloud)\n\nSince it's the same API both can be supported. Creating snapshots regularly is probably the favorable way to go in this case.\n\n## Encryption\n\nEach Snapshot and Update is encrypted using an AEAD constructions. Specifically [XChaCha20-Poly1305-IETF](https://doc.libsodium.org/secret-key_cryptography/aead#availability-and-interoperability). Exchange incl. rotation of the secret key is not part of this protocol and could be done by using the Signal Protocol or lockboxes based on an existing Public key infrastructure (PKI).\n\nEach Snapshot also includes unencrypted but authenticated data so that the server and other clients can verify that authenticity of the Document \u0026 Snapshot ID relation. Unencrypted data:\n\n- Document ID\n- Snapshot ID\n- Public Key of the Client\n\nEach Update also includes unencrypted but authenticated data so that the server and other clients can verify that authenticity of the relationship to a Snapshot and Document.Unencrypted data:\n\n- Document ID\n- Snapshot ID\n- Public Key of the Client\n- Clock\n\nThe clock property is an incrementing integer that serves multiple purposes:\n\n- That the central service does not persist an update if the previous one wasn't persisted.\n- Each client to verify that it receives all updates per snapshot per client.\n\nThe data (encrypted and unencrypted) of each Snapshot and Update further is signed with the public key of the client using a ED2559 Signature. This ensures the authenticity of the data per client and is relevant to make sure to relate the incrementing clock to client.\n\nThe public keys further could be use to verify that only collaborators with the authorization have to make changes to a document actually do so. Serenity will use a [signed hash chain](https://github.com/serenity-kit/Serenity/tree/main/packages/workspace-chain) that's currently in development to ensure the authenticity of all collaborators.\n\nThere are use-cases where the public keys and signatures are only used to verify the updates per client e.g. a short term shared document in a video call.\n\n## Data Integrity\n\nIt's the responsibility of the central service to ensure the data integrity based on the Snapshot ID and Update clocks. This means updates are only persisted when the previous update per snapshot per client is persisted and snapshot is only persisted if it references the previous snapshot incl. all it's related updates.\n\nThe server can't verify if the encrypted data is corrupt. Clients have to trust each other or verify the changes. There are certain limitations to verifying the data integrity and if the central service cooperates with one participant the can be broken in various ways.\n\n## Near Real-time\n\nUpdates can be sent immediately when they happen (to reduce overhead for real-time communication), but will only be persisted in the right order and clients will only apply them in the right order.\n\n## Meta Data\n\nThis protocol doesn't hide meta data from the server. This means the relay service is aware on which documents a client has access to and when and how roughly how much someone contributed to a document.\n\n## High level threat model \u0026 trust model\n\n- The central relay service can not inject any participants nor data into a document.\n- The server can completely cut out one user at any point without being detected. We are thinking to mitigate this with read notifications in a future version of Secsync.\n- Clients have to trust each other.\n\nNote: Instantly removing access can also be seen as an advantage. In in a decentralized system you can have the issue that a collaborator is removed, but until this information is propagate all participants they will continue to share updates with the remove collaborator.\n\n## Further Reading\n\nMore documentation can be found in the [docs](./docs) folder.\n\n- [Requirements](./documentation/public/requirements.md)\n- [Specification](./documentation/public/specification.md)\n- [Security \u0026 Privacy Considerations](./documentation/public/security_and_privacy_considerations.md)\n- [Threat Library](./documentation/public/threat-library.md)\n\n## FAQ \u0026 Background\n\n### Why Snapshots and not only rely on encrypted Updates?\n\nDocuments that are a couple of pages long can included several thousand changes. When loading a document this would mean downloading, decrypting and applying all of them to a CRDT document. This is an UX issue and here snapshots can help, because if downloading one Snapshot, decrypting it and loading the compressed CRDT document is way faster.\n\nNote: We plan to add benchmarks for comparison in the future.\n\n### Why use a central relay service?\n\nThe main reason is to exchange data asynchronously. Meaning one client can create an update and another one retrieve it later when the first one is not online anymore.\n\nA second reason is that\n\n- To create a single stream of snapshots and updates for easier synchronization. (avoids handling complex problems like topological sorting for DAGs … for now)\n\n### Can I move my document to another central service?\n\nYes. You can fetch the essential data (document ID and CRDT document) and upload it to another service. Note: This functionality currently doesn't exist in the prototype.\n\n### What happens when a client has an out of date Snapshot?\n\nIt will receive a new snapshot which will be merged into the local data.\n\n- Automerge: `doc2 = Automerge.merge(doc2, doc1)`\n- Yjs: `Yjs.applyUpdateV2(yDocRef.current, snapshotResult, null);`\n\n### When to create a new Snapshot?\n\nThis highly depends on the use-case e.g. the amount of data per update and frequency. Apart from the there are\n\n- A collaborator is added to the document. When creating a new snapshot the new collaborator only sees the current state from now on. This only works if the CRDT implementation supports tombstones e.g. [Yjs Algorithm](https://github.com/yjs/yjs#yjs-crdt-algorithm).\n- The symmetric encryption is being rotated. Then the relay service can remove the previous snapshot and all related updates.\n\n## Possible Improvements in the Future\n\n- Add optional read notifications. From a UX perspective there can be value in being aware who has received which updates. From the perspective of the threat model it can be a tool to identify if the central relay service is excluding a collaborator.\n- Using a cryptographic ratchet based on a [key derivation function (KDF)](https://en.wikipedia.org/wiki/Key_derivation_function)\n- Leverage Zero-knowledge proofs to hide meta data. Inspired by [Signal's Private Group feature](https://signal.org/blog/private-groups/).\n\nIf you have any further ideas or suggestions please let us know.\n\n## Setup and Run the Example\n\n```sh\npnpm install\ncp examples/backend/.env.example examples/backend/.env\ndocker-compose up\n# in another tab\ncd examples/backend\npnpm prisma migrate dev\npnpm dev\n# in another tab\ncd examples/frontend\npnpm dev\n```\n\n## Run the example with ngrok\n\n```sh\n# get your authtoken from https://dashboard.ngrok.com/get-started/your-authtoken\nngrok start --config=ngrok.yml --all --authtoken=\u003cauthtoken\u003e\n# replace the localhost urls in the code with the ngrok urls\n```\n\n## Setup fly.io deployment\n\nUpdate app name inside fly.toml\n\n```sh\nfly postgres create\n# store the connection string\nflyctl secrets set DATABASE_URL=\u003cdb_connection_url\u003e/secsync\n```\n\nUpdate DATABASE_URL in Github secrets with \u003cdb_connection_url\u003e/secsync\n\n## Wipe the DB @ fly.io\n\n```sh\nflyctl postgres connect --app secsync-db\n# in the psql console run\nSELECT pg_terminate_backend(pg_stat_activity.pid)\nFROM pg_stat_activity\nWHERE pg_stat_activity.datname = 'secsync';\n\ndrop database secsync;\n```\n\n## Credits\n\nSecsync is proudly sponsored by [NGI Assure](https://nlnet.nl/assure/) via [NLNet](https://nlnet.nl).\n\n\u003ca href=\"https://nlnet.nl/assure/\"\u003e\u003cimg src=\"https://nlnet.nl/image/logos/NGIAssure_tag.svg\" alt=\"NLNet\" width=\"100\"\u003e\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikgraf%2Fsecsync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnikgraf%2Fsecsync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikgraf%2Fsecsync/lists"}