{"id":13550088,"url":"https://github.com/mycelial/mycelite","last_synced_at":"2025-05-16T15:03:12.912Z","repository":{"id":65630774,"uuid":"573433074","full_name":"mycelial/mycelite","owner":"mycelial","description":"Mycelite is a SQLite extension that allows you to synchronize changes from one instance of SQLite to another.","archived":false,"fork":false,"pushed_at":"2023-09-25T11:26:31.000Z","size":193,"stargazers_count":1085,"open_issues_count":6,"forks_count":16,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-05-16T15:02:33.866Z","etag":null,"topics":["database-replication","replication","sqlite3","sqlite3-extension"],"latest_commit_sha":null,"homepage":"https://mycelial.com","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mycelial.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-12-02T13:02:10.000Z","updated_at":"2025-05-02T16:38:54.000Z","dependencies_parsed_at":"2023-03-10T18:52:58.076Z","dependency_job_id":"3ddf5131-20db-4523-a0fa-5a4aaa15e85a","html_url":"https://github.com/mycelial/mycelite","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mycelial%2Fmycelite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mycelial%2Fmycelite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mycelial%2Fmycelite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mycelial%2Fmycelite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mycelial","download_url":"https://codeload.github.com/mycelial/mycelite/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254553936,"owners_count":22090415,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database-replication","replication","sqlite3","sqlite3-extension"],"created_at":"2024-08-01T12:01:28.834Z","updated_at":"2025-05-16T15:03:12.852Z","avatar_url":"https://github.com/mycelial.png","language":"Rust","readme":"![tests](https://github.com/mycelial/mycelite/actions/workflows/tests.yml/badge.svg)\n\n# Mycelite\n\nMycelite implements physical, single-writer replication for SQLite. \n\n### Technical details\n\n- Mycelite is a [VFS](https://www.sqlite.org/vfs.html) extension, which acts \nas a proxy for the OS filesystem. \n- Mycelite intercepts page writes and creates a binary diff with the old version. \n- The binary diffs are then stored in the [journal](./journal/README.md). They can also be sent\nover the network to another machine.\n- The diffs in the journal can then be sequentially applied, thus achieving \nbit-perfect copy of the original database.\n\nFor more details on SQLite binary format see [sqlite-parser-nom](https://github.com/mycelial/sqlite-parser-nom).\nIn principle, it could be illustrated in the following way:\n\n```\n┌───────────┐ VFS write                     ┌────────────┐ apply ┌────────────────┐\n│ db.sqlite ├──────────┐                    │ db.journal ├───────► replica.sqlite │\n├───────────┤          │                    ├────────────┤       ├────────────────┤\n│  header   │   ┌──────▼─────┐            ┌─►  diff 0    │       │    header      │\n├───────────┤   │ page 0 new ├─┐          │ ├────────────┤       ├────────────────┤\n│  page 0   ├─┐ └────────────┘ │ ┌──────┐ │ │    ...     │       │    page 0      │\n├───────────┤ │                ├─► diff ├─┘ └────────────┘       ├────────────────┤\n│  page 1   │ │ ┌────────────┐ │ └──────┘                        │      ...       │\n└───────────┘ └─► page 0 old ├─┘                                 └────────────────┘\n                └────────────┘\n```\n\nThis approach comes with both significant upsides and downsides:\n- Replica will contain exactly the same object in exactly the same order as in original.\n- Out-of-the-box non-deterministic DDLs (e.g., UPDATE with RANDOM() or CURRENT_TIMESTAMP).\n- Physical replication is less resource-intensive than logical replication, resulting in\nhigher throughput with no penalty as the number of replicas grows.\n- Time travel by hydrating up to any previous timestamp.\n- As there is no locking mechanism currently implemented, only a single writer is supported.\n- Replica journal grows linearly, unless compacted.\n- VACUUM operation might result in significantly sized journal entry without \nactual changes to accessible data.\n- Currently, [WAL](https://www.sqlite.org/wal.html)-enabled databases are not supported.\n\n### Usage\nRefer to the [Quickstart Documentation](https://mycelial.com/docs/get-started/quick-start).\n\n### A new type of application\n\nThere is a new type of application called local-first, which combines many of\nthe best features from both local and client/server applications.\n\n### What does local-first offer?\n\nWith local-first applications, you get the speed and responsiveness of a local\napplication, but at the same time, you get many of the desirable features from\nclient/server systems.\n\n### What do local-first applications look like?\n\nA good example of a local-first application is [Actual\nBudget](https://github.com/actualbudget/actual), an open-source personal finance\napplication.\n\nWhat makes Actual Budget different from its competitors?\n\nFirst of all, it's very fast because all the application data is on the local\ndevice - in a SQLite database - but what's most interesting about this app is\nthat it works on multiple devices. In other words, it has apps for iOS, Android,\nWindows, Mac and the Web and it allows you to make concurrent changes on\nmultiple devices and synchronize those changes to all your devices.\n\n### Why aren't more developers creating local-first applications?\n\nActual Budget is a good example of a local-first application, but it wasn't very\neasy to build. The authors had to write a bunch of synchronization-related code,\nthat implements and uses\n[CRDTs](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type), and\nthis start-from-scratch approach just isn't practical for most situations.\nBuilding local-first applications today is too difficult, but we're going to\nchange that.\n","funding_links":[],"categories":["Rust","others","backup and replicate"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmycelial%2Fmycelite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmycelial%2Fmycelite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmycelial%2Fmycelite/lists"}