{"id":44833032,"url":"https://github.com/ezmode-games/smuggler","last_synced_at":"2026-02-17T00:35:28.725Z","repository":{"id":336424997,"uuid":"1149586236","full_name":"ezmode-games/smuggler","owner":"ezmode-games","description":"Smuggle data between SQLite and Cloudflare D1. Han Solo approved.","archived":false,"fork":false,"pushed_at":"2026-02-04T21:23:59.000Z","size":297,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-04T22:55:19.551Z","etag":null,"topics":["cloudeflare","d1-database","rust","sqlite"],"latest_commit_sha":null,"homepage":"https://ezmode.games/oss","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ezmode-games.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-04T09:33:30.000Z","updated_at":"2026-02-04T13:53:07.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ezmode-games/smuggler","commit_stats":null,"previous_names":["ezmode-games/smuggler"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/ezmode-games/smuggler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ezmode-games%2Fsmuggler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ezmode-games%2Fsmuggler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ezmode-games%2Fsmuggler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ezmode-games%2Fsmuggler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ezmode-games","download_url":"https://codeload.github.com/ezmode-games/smuggler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ezmode-games%2Fsmuggler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29526945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-16T21:45:09.491Z","status":"ssl_error","status_checked_at":"2026-02-16T21:44:58.452Z","response_time":115,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudeflare","d1-database","rust","sqlite"],"created_at":"2026-02-17T00:35:28.241Z","updated_at":"2026-02-17T00:35:28.717Z","avatar_url":"https://github.com/ezmode-games.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"smuggler.webp\" alt=\"Smuggler\" width=\"400\"\u003e\n\u003c/p\u003e\n\n# Smuggler\n\n\u003e \"Look, I ain't in this for your revolution, and I'm not in it for you, Princess. I expect to be well paid. I'm in it for the money.\" - Han Solo\n\nSmuggle data between SQLite and Cloudflare D1. Fast. Stateless. Questionable life choices.\n\n## Status: Alpha (Han Solo Approved)\n\nWe use this in production at [huttspawn.com](https://huttspawn.com). It works. Mostly.\n\nShould *you* use it? That depends. Do you:\n- Read the manual before assembling furniture? **Maybe wait for 1.0**\n- Shoot first? **Welcome aboard**\n\nThere are [known issues](https://github.com/ezmode-games/smuggler/issues). We're shaving parsecs, not days.\n\n## What It Does\n\nD1 is SQLite at the edge, but Cloudflare doesn't give you a way to sync your local dev database with production. Smuggler fills that gap:\n\n- **Content hashing** - Compares actual row data, not just timestamps\n- **Delta sync** - Only moves rows that changed\n- **Bidirectional** - Push to D1, pull from D1, or both\n- **No state files** - Fresh comparison every run, no stale state to haunt you\n\n## Installation\n\n### Quick install (recommended)\n\n```bash\ncurl -fsSL https://raw.githubusercontent.com/ezmode-games/smuggler/main/install.sh | bash\n```\n\nDetects your platform, downloads the right binary, verifies the checksum, and installs to `~/.local/bin/`. Supports Linux x64, macOS x64, and macOS ARM64.\n\nInstall a specific version:\n\n```bash\ncurl -fsSL https://raw.githubusercontent.com/ezmode-games/smuggler/main/install.sh | bash -s v0.1.0\n```\n\n### Manual download\n\n| Platform | Download |\n|----------|----------|\n| Linux x64 | [smuggler-linux-x64.tar.gz](https://github.com/ezmode-games/smuggler/releases/latest/download/smuggler-linux-x64.tar.gz) |\n| macOS x64 | [smuggler-macos-x64.tar.gz](https://github.com/ezmode-games/smuggler/releases/latest/download/smuggler-macos-x64.tar.gz) |\n| macOS ARM64 | [smuggler-macos-arm64.tar.gz](https://github.com/ezmode-games/smuggler/releases/latest/download/smuggler-macos-arm64.tar.gz) |\n\n### From source\n\n```bash\ncargo install --git https://github.com/ezmode-games/smuggler\n```\n\n## Quick Start\n\n1. Copy the example config:\n\n```bash\ncp config.example.toml config.toml\n```\n\n2. Add your credentials (don't commit this file, genius):\n\n```toml\ncloudflare_account_id = \"your-account-id\"\ncloudflare_api_token = \"your-api-token\"\ndatabase_id = \"your-d1-database-id\"\nlocal_db = \".wrangler/state/v3/d1/miniflare-D1DatabaseObject/xxx.sqlite\"\n```\n\n3. Check if you can reach D1:\n\n```bash\nsmuggler status\n```\n\n4. See what's different:\n\n```bash\nsmuggler diff\n```\n\n5. Push your local changes (point of no return):\n\n```bash\nsmuggler push\n```\n\n## Commands\n\n```\nsmuggler status    # Can we phone home?\nsmuggler diff      # What's different?\nsmuggler push      # Local -\u003e D1 (YOLO)\nsmuggler pull      # D1 -\u003e Local (safer YOLO)\n```\n\n### Options\n\n```\n-c, --config \u003cFILE\u003e   Config file [default: config.toml]\n-v, --verbose         See what's happening under the hood\n--dry-run             Coward mode (just kidding, it's smart)\n--table \u003cNAME\u003e        Sync one table only (validated against schema)\n```\n\n## How It Works\n\nFor each table, Smuggler:\n\n1. Grabs all primary keys from both databases\n2. SHA256 hashes each row's content (excluding timestamp columns)\n3. When content differs, compares timestamps to determine which side is newer\n4. Sorts rows into buckets:\n\n| Bucket | What it means | Push | Pull |\n|--------|--------------|------|------|\n| `local_only` | You added it locally | Insert to D1 | - |\n| `remote_only` | Someone else added it | - | Insert locally |\n| `local_newer` | Your timestamp wins | Update D1 | - |\n| `remote_newer` | Their timestamp wins | - | Update local |\n| `content_differs` | Same timestamp, different data | Configurable | Configurable |\n| `identical` | Exactly the same | Skip | Skip |\n\n### Why content hashing?\n\nTimestamps lie. Clocks drift. Bulk imports set everything to \"now\". Content hashing catches actual changes regardless of what the timestamps say.\n\n## Configuration\n\n```toml\ncloudflare_account_id = \"abc123\"\ncloudflare_api_token = \"your-token-with-d1-permissions\"\ndatabase_id = \"your-d1-uuid\"\nlocal_db = \"/path/to/local.sqlite\"\n\n[sync]\n# Empty = sync all tables except excluded\ntables = []\n\n# Things you definitely don't want to sync\nexclude_tables = [\n    \"sqlite_sequence\",\n    \"_cf_KV\",\n    \"__drizzle_migrations\",\n]\n\n# Optional column for timestamp ordering when content differs\ntimestamp_column = \"updated_at\"\n\n# When both sides changed: \"local_wins\", \"remote_wins\", \"newer_wins\"\nconflict_resolution = \"local_wins\"\n```\n\n### Finding Your Local D1 Database\n\nWrangler hides it at:\n\n```\n.wrangler/state/v3/d1/miniflare-D1DatabaseObject/\u003chash\u003e.sqlite\n```\n\nThe hash is derived from your binding name. If you have multiple databases, may the Force be with you.\n\n### API Token\n\nGet one from [Cloudflare Dashboard](https://dash.cloudflare.com/profile/api-tokens):\n\n- D1:Read - for `diff`, `pull`, `status`\n- D1:Write - for `push`\n\nPro tip: Create one token with both permissions. Fewer tokens to lose.\n\n## Limitations\n\nThings we don't do:\n\n- **Schema sync** - Run your migrations separately, we're data movers not DDL runners\n- **Transactions** - Each batch is atomic, but the whole sync isn't. Re-run if interrupted.\n- **BLOB wizardry** - Binary data compared as hex strings. It works but it's not pretty.\n- **Tables without primary keys** - We need something to compare. Add a PK.\n\n## Troubleshooting\n\n### \"401 Unauthorized\"\nYour token is expired or wrong. Make a new one.\n\n### \"Invalid table name\"\nSmuggler validates `--table` input against your database schema before touching any SQL. If the table doesn't exist, you'll get a list of available tables. Run your migrations on both databases first. We don't create tables.\n\n### All rows show as \"content_differs\"\nCheck that column order and types match. NULL vs empty string will cause hash mismatches.\n\n## Development\n\n```bash\ncargo test                       # Run the tests\ncargo fmt                        # Format code\ncargo clippy --all-targets       # Lint (including tests)\nRUST_LOG=debug cargo run -- diff # Debug output\n```\n\n## Related Projects\n\nPart of the [ezmode-games](https://github.com/ezmode-games) toolchain, built for [huttspawn.com](https://huttspawn.com):\n\n- More tools coming when we get around to it\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md). TL;DR:\n\n1. Fork it\n2. Branch it\n3. Fix it / Build it\n4. Test it\n5. PR it\n\n## License\n\nMIT. Do whatever you want. Not our fault if it deletes your production database.\n\n---\n\n*\"Never tell me the odds.\"*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fezmode-games%2Fsmuggler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fezmode-games%2Fsmuggler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fezmode-games%2Fsmuggler/lists"}