{"id":18952395,"url":"https://github.com/tailscale/squibble","last_synced_at":"2025-04-16T01:33:58.330Z","repository":{"id":222572508,"uuid":"757779368","full_name":"tailscale/squibble","owner":"tailscale","description":"A lightweight schema manager for SQLite databases.","archived":false,"fork":false,"pushed_at":"2024-09-09T23:14:15.000Z","size":75,"stargazers_count":32,"open_issues_count":0,"forks_count":1,"subscribers_count":16,"default_branch":"main","last_synced_at":"2024-09-10T03:35:51.292Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tailscale.png","metadata":{"files":{"readme":"README.md","changelog":"history.sql","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-15T00:39:18.000Z","updated_at":"2024-09-09T23:14:18.000Z","dependencies_parsed_at":"2024-02-15T01:37:55.022Z","dependency_job_id":"8043220e-76a3-4590-8f97-7f9bb1199089","html_url":"https://github.com/tailscale/squibble","commit_stats":null,"previous_names":["tailscale/squibble"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tailscale%2Fsquibble","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tailscale%2Fsquibble/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tailscale%2Fsquibble/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tailscale%2Fsquibble/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tailscale","download_url":"https://codeload.github.com/tailscale/squibble/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223692193,"owners_count":17186948,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T13:33:09.748Z","updated_at":"2024-11-08T13:33:10.247Z","avatar_url":"https://github.com/tailscale.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# squibble\n\nPackage squibble provides a schema migration assistant for SQLite databases.\n\n[![GoDoc](https://img.shields.io/static/v1?label=godoc\u0026message=reference\u0026color=white)](https://pkg.go.dev/github.com/tailscale/squibble)\n[![CI](https://github.com/tailscale/squibble/actions/workflows/go-presubmit.yml/badge.svg?event=push\u0026branch=main)](https://github.com/tailscale/squibble/actions/workflows/go-presubmit.yml)\n\nA `Schema` value manages the schema of a SQLite database that will be modified\nover time.  The current database schema is stored in the Current field, and\nmigrations from previous versions are captured as `UpdateRules`.\n\n## Example\n\n```go\n//go:embed schema.sql\nvar dbSchema string\n\nvar schema = \u0026squibble.Schema{\n\tCurrent: dbSchema,\n\n\tUpdates: []squibble.UpdateRule{\n\t\t// Each update gives the digests of the source and target schemas,\n\t\t// and a function to modify the first into the second.\n\t\t// The digests act as a version marker.\n\t\t{\"a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447\",\n\t\t\t\"727e2659ac457a3c86da2203ebd2e7387767ffe9a93501def5a87034ee672750\",\n\t\t\tsquibble.Exec(`CREATE TABLE foo (bar TEXT)`),\n\t\t},\n\t\t// The last update must end with the current schema.\n\t\t// Note that multiple changes are permitted in a rule.\n\t\t{\"727e2659ac457a3c86da2203ebd2e7387767ffe9a93501def5a87034ee672750\",\n\t\t\t\"f18496b875133e09906a26ba23ef0e5f4085c1507dc3efee9af619759cb0fafe\",\n\t\t\tsquibble.Exec(\n\t\t\t\t`ALTER TABLE foo ADD COLUMN baz INTEGER NOT NULL`,\n\t\t\t\t`DROP VIEW quux`,\n\t\t\t),\n\t\t},\n\t},\n}\n\nfunc main() {\n   flag.Parse()\n\n   // Open the database as usual.\n   db, err := sql.Open(\"sqlite\", \"test.db\")\n   if err != nil {\n      log.Fatalf(\"Open db: %v\", err)\n   }\n\n   // Apply any schema migrations needed.\n   if err := schema.Apply(context.Background(), db); err != nil {\n      log.Fatalf(\"Apply schema: %v\", err)\n   }\n\n   // ...how you do\n}\n```\n\n## Usage Outline\n\nFor the following, assume your schema is defined in a file `schema.sql` and the\ncurrent database is `data.db`.\n\n1. Modify `schema.sql` to look like the schema you want the database to end up\n   with.\n\n2. Run `squibble diff data.db schema.sql`. This will print out the difference\n   between the database schema and the update, including the computed digests.\n\n   ```\n   db:  b9062f812474223063c121d058e23823bf750074d1eba26605bbebbc9fd20dbe\n   sql: 76a0ed44d8ad976d1de83bcb67d549dee2ab5bfb5af7d597d2548119e7359455\n   \u003c human-readable-ish diff \u003e\n   ```\n\n3. Using these digests, a new rule to the end of the `Upgrades` list like:\n\n   ```go\n   {\n     Source: \"b9062f812474223063c121d058e23823bf750074d1eba26605bbebbc9fd20dbe\",  // from the db\n     Target: \"76a0ed44d8ad976d1de83bcb67d549dee2ab5bfb5af7d597d2548119e7359455\",  // from the schema\n     Apply: squibble.Exec(`\n        ALTER TABLE foo ADD COLUMN bar TEXT UNIQUE NOT NULL DEFAULT 'xyzzy';\n        DROP VIEW IF EXISTS fuzzypants;\n        CREATE INDEX horse_index ON animal (species) WHERE (species = 'horse');\n     `),\n   }\n   ```\n\n   Use `squibble diff --rule data.db schema.sql` to generate a copyable Go\n   source text in this format. For example:\n\n   ```go\n   {\n       Source: \"8d4f9b3e29aeca09e891460bf5ed08f12b84f6887b46a61082c339d49d7e0be8\",\n       Target: \"b196954e613b770a4a1c0a07b96f6e03cb86923a226c2b53bd523fb759fef3d6\",\n       Apply: func(ctx context.Context, db squibble.DBConn) error {\n           /* Schema diff:\n\n           \u003e\u003e Modify table \"Templates\"\n            ! replace column \"raw\" BLOB\n              with \"raw\" BLOB not null\n            + add column \"count\" INTEGER not null default=0\n\n           \u003e\u003e Add table \"lard\"\n            + CREATE TABLE lard (z integer, s text unique)\n\n           */\n           panic(\"not implemented\")\n       },\n   },\n   ```\n\n   You will still need to fill in the update rule implementation, but a\n   human-readable summary of the changes will be included as a comment to make\n   it easier to figure out what to write.  As shown in the example above, the\n   `squibble.Exec` function can be helpful for simple changes.\n\n   You should delete the comment before merging the rule, for legibility.\n\n## Mixing Migration and In-Place Updates\n\nSome schema changes can be done \"in-place\", simply by re-applying the schema\nwithout any other migration steps. Typical examples include the addition or\nremoval of whole tables, views, indexes, or triggers, which can be applied\nconditionally with statements like:\n\n```sql\nCREATE TABLE IF NOT EXISTS Foo ( ... )\n\nDROP VIEW IF EXISTS Bar;\n```\n\nI generally recommend you _not_ combine this style of update with use of the\nschema migrator. It works fine to do so, but adds extra friction.\n\nIf you do want to manage schema changes this way, you should apply the updated\nschema _before_ calling the `Apply` method of the `squibble.Schema`.  If the\nnew schema has changes that are not compatible with the known migration state,\nthe `Apply` method will report an error, and you can add an appropriate\nmigration step.\n\nFor example, suppose you have this schema:\n\n```sql\n-- Schema 1\nCREATE TABLE IF NOT EXISTS Foo (\n  id INTEGER PRIMARY KEY,\n  name TEXT NOT NULL\n);\n```\n\nAfter executing Schema 1, the migrator will be satisfied: The schema before\nmigration already looks like Schema 1, so there is nothing to do.\n\nNow say you add a new column:\n\n```sql\n-- Schema 2\nCREATE TABLE IF NOT EXISTS Foo (\n  id INTEGER PRIMARY KEY,\n  name TEXT NOT NULL,\n  important BOOL   -- new column\n);\n```\n\nWhen executing Schema 2, the database does not change: Table `Foo` already\nexists, so SQLite does not do anything. But the migrator sees that the schema\nhas changed and it doesn't have a migration rule, so you will have to add one:\n\n```go\nUpdates: []squibble.UpdateRule{{\n   Source: \"7e4799f89f03e9913d309f50c4cc70963fc5607fb335aa318f9c246fdd336488\",\n   Target: \"dee76ad0f980b8a5b419c4269559576d8413666adfe4a882e77f17b5792cca01\",\n   Apply:  squibble.Exec(`ALTER TABLE Foo ADD COLUMN important BOOL`),\n}}\n```\n\nand the migrator will be happy. Now say you add a new table:\n\n```sql\n-- Schema 3\nCREATE TABLE IF NOT EXISTS Foo (\n  id INTEGER PRIMARY KEY,\n  name TEXT NOT NULL,\n  important BOOL   -- added in schema 2\n);\n\nCREATE TABLE IF NOT EXISTS Bar (comment TEXT NOT NULL);\n```\n\nThis executes just fine, but now the state of the database seen by the migrator\nis different from the last state it has an update for: It has no migration rule\nto go from dee76ad0f980b8a5b419c4269559576d8413666adfe4a882e77f17b5792cca01 to\n30233f4462f18d591795b1f8b455a5daf3b19c8786e90ec94daf8d3825de0320, which is the\nstate of the database after Schema 3 was applied.\n\nThe migrator needs a rule for this, but the rule can be a no-op:\n\n```sql\nUpdates: []squibble.UpdateRule{{\n   Source: \"7e4799f89f03e9913d309f50c4cc70963fc5607fb335aa318f9c246fdd336488\",\n   Target: \"dee76ad0f980b8a5b419c4269559576d8413666adfe4a882e77f17b5792cca01\",\n   Apply:  squibble.Exec(`ALTER TABLE Foo ADD COLUMN important BOOL`),\n}, {\n   // This rule tells the migrator how to get to the current state, but\n   // the change was already handled by the schema directly.\n   Source: \"dee76ad0f980b8a5b419c4269559576d8413666adfe4a882e77f17b5792cca01\",\n   Target: \"30233f4462f18d591795b1f8b455a5daf3b19c8786e90ec94daf8d3825de0320\",\n   Apply:  squibble.NoAction, // does nothing, just marks an update\n}}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftailscale%2Fsquibble","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftailscale%2Fsquibble","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftailscale%2Fsquibble/lists"}