{"id":45575717,"url":"https://github.com/marcelocantos/sqlpipe","last_synced_at":"2026-05-09T08:16:36.639Z","repository":{"id":339894200,"uuid":"1163751597","full_name":"marcelocantos/sqlpipe","owner":"marcelocantos","description":"Streaming replication protocol for SQLite","archived":false,"fork":false,"pushed_at":"2026-03-29T13:51:19.000Z","size":3216,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-03-29T16:34:07.927Z","etag":null,"topics":["cpp","database","replication","sqlite","streaming"],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcelocantos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-22T04:47:14.000Z","updated_at":"2026-03-29T13:51:22.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/marcelocantos/sqlpipe","commit_stats":null,"previous_names":["marcelocantos/sqlpipe"],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/marcelocantos/sqlpipe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelocantos%2Fsqlpipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelocantos%2Fsqlpipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelocantos%2Fsqlpipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelocantos%2Fsqlpipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcelocantos","download_url":"https://codeload.github.com/marcelocantos/sqlpipe/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcelocantos%2Fsqlpipe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31292347,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T21:15:39.731Z","status":"ssl_error","status_checked_at":"2026-04-01T21:15:34.046Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","database","replication","sqlite","streaming"],"created_at":"2026-02-23T09:48:21.137Z","updated_at":"2026-05-09T08:16:36.632Z","avatar_url":"https://github.com/marcelocantos.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# sqlpipe\n\nUnified SQLite library: replication, schema migration, and query transpilation.\n\nsqlpipe is a C++ library that combines three capabilities in a single\ntwo-file distribution (`dist/sqlpipe.h` / `dist/sqlpipe.cpp`):\n\n- **sqlpipe core** — keeps SQLite databases in sync over any transport layer.\n  A **Master** tracks changes and produces compact binary changesets; a\n  **Replica** applies them, emitting per-row change events and query\n  subscription updates. A **Peer** wraps both behind a symmetric API for\n  bidirectional replication with table-level ownership. A **Relay** enables\n  chain replication (source → relay → sink).\n- **sqlift** — declarative schema migration via structural diffing. The\n  `Database` constructor auto-migrates your schema on open; the\n  `generate_migration()` free function produces migration SQL from any two\n  DDL strings.\n- **sqldeep** — JSON5-like SQL transpiler. All SQL passed through `Database`\n  methods is transpiled automatically. Extended syntax like\n  `SELECT {id, name}` (JSON object construction) works out of the box.\n\nThe `Database` class is the primary entry point for most users. `Master`,\n`Replica`, and `Peer` are the lower-level building blocks for custom\nreplication topologies.\n\nThe library is transport-agnostic: it defines a message-in / message-out API.\nYou decide how messages travel between peers (TCP, WebSocket, QUIC, serial,\nshared memory, datagrams, etc.). The convergence loop makes every message\nregenerable, so the protocol works over pure datagrams — any message can be\nlost and recovered by the next convergence round.\n\n## Features\n\n- **Unified Database class** — opens SQLite, auto-migrates the schema via\n  sqlift, provides `exec`/`query`/`subscribe` with automatic change\n  notification. The primary API for most use cases.\n- **RAII Subscription** — `Database::subscribe` returns a `Subscription`\n  object that auto-unsubscribes on destruction.\n- **sqldeep syntax everywhere** — all SQL in `Database` methods is transpiled\n  automatically. Write `SELECT {id, name} FROM items` and get back a JSON\n  object per row with no extra wiring.\n- **Bundled distribution** — `dist/sqlpipe.h` + `dist/sqlpipe.cpp` include\n  sqlpipe core, sqlift, and sqldeep. No separate dependency installation\n  needed beyond SQLite itself.\n- **Schema migration via sqlift** — `Database` constructor auto-migrates on\n  open; `Database::migration(from_ddl, to_ddl)` static method; and the\n  `generate_migration(old_ddl, new_ddl)` free function for offline use.\n- **Convergence loop** — `Replica::converge()` provides stateless,\n  loss-tolerant sync. The replica computes bucket hashes and sends them\n  directly; the master responds with the delta. Works entirely over\n  datagrams. No handshake required. Call it periodically or on reconnect.\n- **Bidirectional replication** via the Peer API — each side owns a disjoint\n  set of tables (glob patterns supported, e.g. `\"*\"` owns all tables), with\n  server-authoritative ownership negotiation\n- **Chain replication** via the Relay class — source → relay → sink\n  topologies for fan-out or geographic distribution\n- **Incremental replication** via SQLite's session extension (compact binary\n  changesets)\n- **Efficient diff sync** on reconnect — bucketed row hashes identify what\n  differs, then only the delta is transferred\n- **Changeset queue** — master retains recent changesets\n  (`changeset_queue_size`, default 64) for fast reconnect replay without\n  full diff sync\n- **Predicate-aware query subscriptions** — register SQL queries on the\n  replica; receive updated result sets only when incoming changes match\n  extracted predicates. Queries are parsed via liteparser into relational\n  algebra; predicates are propagated through equijoins and evaluated by a\n  bytecode VM. Supports equality, inequality, range, IS NULL, IN, NOT IN,\n  BETWEEN, and OR-of-equalities.\n- **Prediction API** — `begin_prediction`/`commit_prediction`/`rollback_prediction`\n  for optimistic local updates with automatic rollback on server response\n- **Auto-flush** — `MasterConfig::on_flush` callback fires on commit, so\n  callers never need to call `flush()` explicitly\n- **Per-row change events** (insert/update/delete) on the receiving side\n- **Conflict callbacks** for custom resolution logic\n- **LZ4 changeset compression** — automatic, with uncompressed fallback\n- **Schema fingerprinting** via structural hashing (sqlift) to detect mismatches\n- **Single header + source** (`sqlpipe.h` / `sqlpipe.cpp`) for easy integration\n- **Formally verified** — the convergence protocol is modelled in\n  [TLA+](formal/Convergence.tla) and checked with TLC\n\n## Language bindings\n\n| Language | Location | Install |\n|---|---|---|\n| C++ | `dist/sqlpipe.h` + `dist/sqlpipe.cpp` | Copy two files (includes sqlpipe + sqlift + sqldeep) |\n| Go | `go/sqlpipe/` | `go get github.com/marcelocantos/sqlpipe/go/sqlpipe` |\n| Swift | `swift/` | SPM package with `CSqlpipe` and `Sqlpipe` targets |\n| TypeScript/Wasm | `web/` | `npm install` (builds SQLite + sqlpipe to Wasm) |\n\n## Requirements\n\n- C++23 compiler\n- SQLite 3 compiled with `-DSQLITE_ENABLE_SESSION\n  -DSQLITE_ENABLE_PREUPDATE_HOOK`\n\nAll tables must have explicit `PRIMARY KEY`s (required by SQLite's session\nextension). `WITHOUT ROWID` tables are not supported.\n\n## Quick start\n\n### Database API (recommended)\n\n```cpp\n#include \u003csqlpipe.h\u003e\nusing namespace sqlpipe;\n\n// Open database with schema (auto-creates/migrates via sqlift).\nDatabase db(\":memory:\", \"CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT, qty INTEGER)\");\n\n// Subscribe — fires immediately with initial result, then on every change.\n// Subscription auto-unsubscribes when it goes out of scope (RAII).\nauto sub = db.subscribe(\"SELECT {id, name, qty} FROM items ORDER BY id\",\n    [](const QueryResult\u0026 r) {\n        for (auto\u0026 row : r.rows)\n            std::cout \u003c\u003c std::get\u003cstd::string\u003e(row[0]) \u003c\u003c \"\\n\";\n    });\n\n// Insert — subscription fires automatically.\ndb.exec(\"INSERT INTO items VALUES (1, 'Widget', 10)\");\n\n// sqldeep syntax works everywhere.\nauto r = db.query(\"SELECT {id, name} FROM items WHERE qty \u003e 5\");\n```\n\n### Replication with Database\n\n`Database` integrates cleanly with `Master` and `Replica` by exposing the\nunderlying `sqlite3*` handle. After applying a changeset, call\n`replica_db.notify()` to fire subscriptions.\n\n```cpp\nDatabase master_db(\":memory:\", schema);\nDatabase replica_db(\":memory:\", schema);\n\nMaster master(master_db.handle());\nReplica replica(replica_db.handle());\nsync_handshake(master, replica);\n\n// Subscribe on replica side.\nauto sub = replica_db.subscribe(\"SELECT count(*) FROM items\",\n    [](const QueryResult\u0026 r) { /* ... */ });\n\n// Insert on master, flush, apply on replica.\nmaster_db.exec(\"INSERT INTO items VALUES (1, 'hello')\");\nfor (auto\u0026 msg : master.flush())\n    replica.handle_message(msg);\nreplica_db.notify();  // fires subscriptions registered on replica_db\n```\n\n### Lower-level API (Master/Replica directly)\n\n```cpp\n#include \u003csqlpipe.h\u003e\nusing namespace sqlpipe;\n\n// Open two databases with matching schemas.\nsqlite3 *master_db, *replica_db;\nsqlite3_open(\":memory:\", \u0026master_db);\nsqlite3_open(\":memory:\", \u0026replica_db);\nsqlite3_exec(master_db,\n    \"CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT)\", 0, 0, 0);\nsqlite3_exec(replica_db,\n    \"CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT)\", 0, 0, 0);\n\n// Create master and replica.\nMaster master(master_db);\nReplica replica(replica_db);\n\n// Handshake (exchange messages until replica reaches Live state).\nsync_handshake(master, replica);  // convenience for in-process use\n\n// Make changes on the master, then flush.\nsqlite3_exec(master_db, \"INSERT INTO t VALUES (1, 'hello')\", 0, 0, 0);\nauto msgs = master.flush();  // returns vector\u003cMessage\u003e\nfor (auto\u0026 msg : msgs) {\n    auto result = replica.handle_message(msg);\n    // result.messages    — vector\u003cMessage\u003e to send back\n    // result.changes     — per-row ChangeEvents\n    // result.subscriptions — updated query results\n}\n// replica_db now has the row.\n```\n\nSee [`examples/loopback.cpp`](examples/loopback.cpp) for a complete working\nexample including handshake and change event handling.\n\n## Building\n\n```sh\ngit clone --recurse-submodules https://github.com/marcelocantos/sqlpipe.git\ncd sqlpipe\nmk test     # build and run tests (146 test cases)\nmk example  # build and run the loopback demo\nmk wasm     # build Wasm module (requires emscripten)\n```\n\nIf you use an agentic coding tool (Claude Code, Cursor, Copilot, etc.), include\n[`dist/sqlpipe-agents-guide.md`](dist/sqlpipe-agents-guide.md) in your project\ncontext for a condensed API reference.\n\n## Protocol overview\n\nTwo sync paths are available. The convergence loop is preferred for most use\ncases; the legacy handshake is available for environments that require\nordered reliable delivery.\n\n**Convergence loop** (preferred — stateless, works over datagrams):\n\n```mermaid\nsequenceDiagram\n    participant R as Replica\n    participant M as Master\n\n    Note over R: converge() — no prior hello needed\n    R-\u003e\u003eM: BucketHashesMsg\n    Note over M: compare bucket hashes\n    M-\u003e\u003eR: NeedBucketsMsg (skipped if all match)\n    R-\u003e\u003eM: RowHashesMsg\n    Note over M: compute diff\n    M-\u003e\u003eR: DiffReadyMsg (patchset + deletes)\n    R-\u003e\u003eM: AckMsg\n\n    rect rgb(240, 248, 255)\n        Note over R,M: Live streaming\n        M-\u003e\u003eR: ChangesetMsg (master.flush())\n        R-\u003e\u003eM: AckMsg\n    end\n```\n\nEvery message in the convergence loop is regenerable. If any message is\nlost, call `converge()` again — the loop is idempotent. The master\nprocesses `BucketHashesMsg` directly without requiring a prior `HelloMsg`.\n\n**Legacy handshake** (ordered reliable channel):\n\n```mermaid\nsequenceDiagram\n    participant R as Replica\n    participant M as Master\n\n    R-\u003e\u003eM: HelloMsg\n    M-\u003e\u003eR: HelloMsg (or ErrorMsg on schema mismatch)\n    R-\u003e\u003eM: BucketHashesMsg\n    M-\u003e\u003eR: NeedBucketsMsg\n    R-\u003e\u003eM: RowHashesMsg\n    M-\u003e\u003eR: DiffReadyMsg\n    R-\u003e\u003eM: AckMsg\n\n    rect rgb(240, 248, 255)\n        Note over R,M: Live streaming\n    end\n```\n\n## API\n\n### Database\n\nThe `Database` class is the recommended starting point. It wraps a `sqlite3*`\nhandle, auto-migrates on open, and wires up subscriptions. All SQL is\ntranspiled through sqldeep before execution.\n\n```cpp\nclass Database {\npublic:\n    // Open (or create) a database at path, auto-migrating to schema_ddl.\n    Database(const std::string\u0026 path, const std::string\u0026 schema_ddl);\n\n    // Execute a statement (sqldeep transpiled). Fires notify() on success.\n    void exec(const std::string\u0026 sql);\n\n    // Run a query (sqldeep transpiled). Returns the full result set.\n    QueryResult query(const std::string\u0026 sql);\n\n    // Register a callback query (sqldeep transpiled). The callback fires\n    // immediately with the current result, then again after each exec() or\n    // notify() call that touches a table the query reads from.\n    // Returns a Subscription that unsubscribes on destruction.\n    Subscription subscribe(const std::string\u0026 sql,\n                           std::function\u003cvoid(const QueryResult\u0026)\u003e callback);\n\n    // Manually trigger subscription re-evaluation (all tables).\n    void notify();\n\n    // Trigger subscription re-evaluation for a specific set of tables.\n    void notify(const std::set\u003cstd::string\u003e\u0026 tables);\n\n    // Access the raw sqlite3* handle (e.g. to pass to Master/Replica/Peer).\n    sqlite3* handle() const;\n\n    // Generate migration SQL from one DDL to another (uses sqlift).\n    static std::string migration(const std::string\u0026 from_ddl,\n                                 const std::string\u0026 to_ddl);\n};\n\n// Free function: generate migration SQL between two DDL strings.\nstd::string generate_migration(const std::string\u0026 old_ddl,\n                               const std::string\u0026 new_ddl);\n```\n\n`Subscription` is a RAII handle: when it is destroyed, the underlying\nsubscription is automatically removed.\n\n### Master\n\n```cpp\nstruct MasterConfig {\n    std::optional\u003cstd::set\u003cstd::string\u003e\u003e table_filter;  // nullopt = all tables\n    std::int64_t bucket_size = 1024;\n    ProgressCallback on_progress = nullptr;\n    SchemaMismatchCallback on_schema_mismatch = nullptr;\n    FlushCallback on_flush = nullptr;        // auto-flush on commit (takes std::vector\u003cMessage\u003e)\n    std::size_t changeset_queue_size = 64;   // 0 = disable queue replay\n    LogCallback on_log = nullptr;\n};\n\nclass Master {\npublic:\n    explicit Master(sqlite3* db, MasterConfig config = {});\n    void exec(const std::string\u0026 sql);                     // auto-flushes if on_flush set\n    std::vector\u003cMessage\u003e flush();                          // manual flush\n    std::vector\u003cMessage\u003e handle_message(const Message\u0026 msg);\n    Seq current_seq() const;\n    SchemaVersion schema_version() const;\n};\n```\n\n### Replica\n\n```cpp\nstruct ReplicaConfig {\n    ConflictCallback on_conflict = nullptr;\n    std::optional\u003cstd::set\u003cstd::string\u003e\u003e table_filter;\n    std::int64_t bucket_size = 1024;\n    ProgressCallback on_progress = nullptr;\n    SchemaMismatchCallback on_schema_mismatch = nullptr;\n    LogCallback on_log = nullptr;\n};\n\nstruct HandleResult {\n    std::vector\u003cMessage\u003e      messages;       // protocol responses\n    std::vector\u003cChangeEvent\u003e  changes;        // row-level changes applied\n    std::vector\u003cQueryResult\u003e  subscriptions;  // invalidated query results\n};\n\nclass Replica {\npublic:\n    explicit Replica(sqlite3* db, ReplicaConfig config = {});\n    Message hello() const;\n    std::vector\u003cMessage\u003e converge();                       // stateless sync\n    HandleResult handle_message(const Message\u0026 msg);\n    HandleResult handle_messages(std::span\u003cconst Message\u003e msgs);  // batched\n    SubscriptionId subscribe(const std::string\u0026 sql);\n    void unsubscribe(SubscriptionId id);\n    void begin_prediction();       // optimistic local update\n    void commit_prediction();      // finalise prediction\n    void rollback_prediction();    // cancel prediction\n    void reset();                  // back to Init; preserves subscriptions\n    Seq current_seq() const;\n    SchemaVersion schema_version() const;\n    State state() const;  // Init, Handshake, DiffBuckets, DiffRows, Live, Error\n};\n```\n\n### Peer (bidirectional)\n\n```cpp\nenum class PeerRole : std::uint8_t { Client, Server };\n\nstruct PeerConfig {\n    PeerRole role = PeerRole::Client;\n    std::set\u003cstd::string\u003e owned_tables;  // glob patterns; e.g. \"*\" owns all tables\n    std::optional\u003cstd::set\u003cstd::string\u003e\u003e table_filter;\n    ApproveOwnershipCallback approve_ownership = nullptr;  // server only\n    ConflictCallback on_conflict = nullptr;\n    ProgressCallback on_progress = nullptr;\n    SchemaMismatchCallback on_schema_mismatch = nullptr;\n    LogCallback on_log = nullptr;\n};\n\nstruct PeerHandleResult {\n    std::vector\u003cPeerMessage\u003e  messages;\n    std::vector\u003cChangeEvent\u003e  changes;\n    std::vector\u003cQueryResult\u003e  subscriptions;\n};\n\nclass Peer {\npublic:\n    explicit Peer(sqlite3* db, PeerConfig config = {});\n    std::vector\u003cPeerMessage\u003e start();      // client initiates\n    std::vector\u003cPeerMessage\u003e flush();\n    PeerHandleResult handle_message(const PeerMessage\u0026 msg);\n    SubscriptionId subscribe(const std::string\u0026 sql);\n    void unsubscribe(SubscriptionId id);\n    void reset();\n    State state() const;  // Init, Negotiating, Diffing, Live, Error\n    const std::set\u003cstd::string\u003e\u0026 owned_tables() const;\n    const std::set\u003cstd::string\u003e\u0026 remote_tables() const;\n};\n```\n\n### Relay (chain replication)\n\n```cpp\nclass Relay {\npublic:\n    explicit Relay(sqlite3* db, RelayConfig config = {});\n    std::size_t add_sink(SinkCallback cb);             // register downstream (takes const Message\u0026)\n    void remove_sink(std::size_t id);\n    Message hello();                                   // send to upstream\n    std::vector\u003cMessage\u003e handle_upstream(const Message\u0026 msg);\n    std::vector\u003cMessage\u003e handle_downstream(const Message\u0026 msg);\n    SubscriptionId subscribe(const std::string\u0026 sql);\n    void unsubscribe(SubscriptionId id);\n    void reset();\n};\n```\n\n### Query subscriptions\n\nRegister SQL queries on the replica to receive updated results when incoming\nchanges match the query's predicates:\n\n```cpp\nauto id = replica.subscribe(\"SELECT id, val FROM t1 WHERE val \u003e 10 ORDER BY id\");\n\n// After applying a changeset:\nauto result = replica.handle_message(changeset_msg);\nfor (const auto\u0026 sub : result.subscriptions) {\n    // sub.id      — which subscription fired\n    // sub.columns — column names\n    // sub.rows    — the full updated result set\n}\n\nreplica.unsubscribe(id);\n```\n\nPredicates are extracted from WHERE clauses and propagated through equijoins.\nA bytecode VM evaluates predicates against changeset rows, so subscriptions\nwhose predicates don't match are skipped entirely — no SQL re-evaluation\nneeded.\n\n### Prediction API\n\nOptimistic local updates with automatic rollback:\n\n```cpp\nreplica.begin_prediction();\n// Write optimistically to the local database.\nsqlite3_exec(replica_db, \"INSERT INTO items VALUES (99, 'pending')\", 0, 0, 0);\n// Subscriptions now reflect the predicted state.\nreplica.commit_prediction();\n// Send the corresponding action to the server.\n// When the server's changeset arrives via handle_message(), the prediction\n// savepoint is automatically rolled back and the server's state applied.\n```\n\n### Reconnection\n\n**Convergence loop** (preferred): Call `converge()` at any time to sync\nwithout a handshake. Works from any state — Init, Live, or after `reset()`.\n\n```cpp\nreplica.reset();\nauto msgs = replica.converge();  // returns BucketHashesMsg\n// Send msgs to master, process responses normally.\n// If a message is lost, just call converge() again.\n```\n\n**Legacy handshake**: For ordered reliable channels.\n\n```cpp\nreplica.reset();\nauto hello = replica.hello();\n// ... exchange messages until Live ...\n```\n\n**Peer reconnection**:\n\n```cpp\npeer.reset();              // preserves table ownership\nauto msgs = peer.start();  // re-initiate handshake\n```\n\n## Error handling\n\nAll operations may throw `sqlpipe::Error`, which carries an `ErrorCode` and a\nhuman-readable message:\n\n```cpp\ntry {\n    auto msgs = master.flush();  // std::vector\u003cMessage\u003e\n} catch (const sqlpipe::Error\u0026 e) {\n    // e.code()  — ErrorCode enum\n    // e.what()  — descriptive string\n}\n```\n\n| ErrorCode | Meaning | Recommended action |\n|---|---|---|\n| `SqliteError` | An underlying SQLite call failed | Check the message; may indicate corruption or constraint violation |\n| `ProtocolError` | Malformed or unexpected message | Disconnect and reconnect |\n| `SchemaMismatch` | Master and replica schemas differ | Install `on_schema_mismatch`, or migrate offline and reconnect |\n| `InvalidState` | Operation not valid in current state | Bug in calling code |\n| `OwnershipRejected` | Peer ownership request rejected | Server's `approve_ownership` returned false |\n| `WithoutRowidTable` | Table uses `WITHOUT ROWID` | Use regular rowid tables |\n\n## Schema migration\n\n`Database` handles migration automatically on construction. For custom\nmigration workflows, use `generate_migration()` or the static\n`Database::migration()` method, which both delegate to sqlift's structural\nschema diffing:\n\n```cpp\n// Generate migration SQL between two DDL strings.\nauto sql = generate_migration(old_schema_ddl, new_schema_ddl);\nsqlite3_exec(db, sql.c_str(), 0, 0, 0);\n\n// Or via the static method:\nauto sql = Database::migration(old_schema_ddl, new_schema_ddl);\n```\n\nFor lower-level use, install an `on_schema_mismatch` callback to run\nmigrations on schema mismatch instead of erroring:\n\n```cpp\nReplicaConfig rc;\nrc.on_schema_mismatch = [\u0026](SchemaVersion remote, SchemaVersion local,\n                            const std::string\u0026 remote_schema_sql) {\n    // remote_schema_sql has the master's CREATE TABLE statements.\n    sqlite3_exec(replica_db, \"ALTER TABLE t ADD COLUMN new_col TEXT\", 0, 0, 0);\n    return true;  // reset to Init; re-handshake\n};\n```\n\nThe same callback is available on `MasterConfig` and `PeerConfig`.\n\n## Transport wiring\n\nsqlpipe is transport-agnostic. The wire format is length-prefixed:\n\n```cpp\n// Sending:\nauto buf = sqlpipe::serialize(msg);   // or serialize(peer_msg)\nsend(socket, buf.data(), buf.size());\n\n// Receiving:\nuint8_t hdr[4];\nrecv(socket, hdr, 4);\nuint32_t len = hdr[0] | (hdr[1]\u003c\u003c8) | (hdr[2]\u003c\u003c16) | (hdr[3]\u003c\u003c24);\nstd::vector\u003cuint8_t\u003e buf(4 + len);\nmemcpy(buf.data(), hdr, 4);\nrecv(socket, buf.data() + 4, len);\nauto msg = sqlpipe::deserialize(buf);    // or deserialize_peer(buf)\n```\n\nThe Go wrapper provides a `Transport` interface in\n`go/sqlpipe/transport` for pluggable transport implementations.\n\n## Thread safety\n\n`Master`, `Replica`, `Peer`, `Relay`, and `Database` are **not thread-safe**.\nEach instance must be accessed from a single thread at a time. The `sqlite3*`\nhandle must not be used concurrently during sqlpipe operations.\n\n## Message size limits\n\n- **`kMaxMessageSize`** (64 MB) — maximum serialized message size\n- **`kMaxArrayCount`** (10 M) — maximum elements in any array field\n\nMessages exceeding these limits cause `deserialize()` to throw `ProtocolError`.\n\n## Related projects\n\nsqldeep and sqlift are bundled into the sqlpipe distribution and require no\nseparate installation. Their standalone repos remain available for use outside\nof sqlpipe:\n\n- **[sqldeep](https://github.com/marcelocantos/sqldeep)** — JSON5-like SQL syntax transpiler for SQLite JSON functions (bundled into sqlpipe)\n- **[sqlift](https://github.com/marcelocantos/sqlift)** — Declarative SQLite schema migrations via structural diffing (bundled into sqlpipe)\n\n## License\n\nApache 2.0. See [LICENSE](LICENSE) for details.\n\nThird-party dependencies:\n- **SQLite** — public domain\n- **LZ4** — BSD 2-Clause\n- **spdlog** — MIT\n- **nlohmann/json** — MIT\n- **liteparser** — MIT\n- **sqlift** — Apache 2.0\n- **sqldeep** — Apache 2.0\n- **doctest** — MIT (test only)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcelocantos%2Fsqlpipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcelocantos%2Fsqlpipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcelocantos%2Fsqlpipe/lists"}