{"id":30748274,"url":"https://github.com/marklynch/simple-schema-versions","last_synced_at":"2025-09-04T05:22:56.098Z","repository":{"id":311360389,"uuid":"1043466144","full_name":"marklynch/simple-schema-versions","owner":"marklynch","description":"A simple schema versioning for Postgres that scales from basic projects to large scale systems","archived":false,"fork":false,"pushed_at":"2025-08-24T00:17:02.000Z","size":7,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-24T10:34:56.480Z","etag":null,"topics":["postgres","postgresql","sql"],"latest_commit_sha":null,"homepage":"","language":"PLpgSQL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marklynch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-23T23:13:21.000Z","updated_at":"2025-08-24T00:17:06.000Z","dependencies_parsed_at":"2025-08-24T10:43:26.914Z","dependency_job_id":"87ce794c-4623-4b8d-b3da-1e8db2bb12f6","html_url":"https://github.com/marklynch/simple-schema-versions","commit_stats":null,"previous_names":["marklynch/simple-schema-versions"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/marklynch/simple-schema-versions","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marklynch%2Fsimple-schema-versions","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marklynch%2Fsimple-schema-versions/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marklynch%2Fsimple-schema-versions/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marklynch%2Fsimple-schema-versions/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marklynch","download_url":"https://codeload.github.com/marklynch/simple-schema-versions/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marklynch%2Fsimple-schema-versions/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273555459,"owners_count":25126316,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-04T02:00:08.968Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["postgres","postgresql","sql"],"created_at":"2025-09-04T05:22:52.703Z","updated_at":"2025-09-04T05:22:56.069Z","avatar_url":"https://github.com/marklynch.png","language":"PLpgSQL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Simple Schema Versions\n\nThis project provides simple rules and utilities for managing schema changes in a PostgreSQL database. It helps track and manage schema versions, making migrations and upgrades easier and safer.\n\nIt uses pure SQL functions and does not rely on any external functionality.\n\n## Features\n\n- **Schema Type Identification:**  \n  Use the `get_schema_type()` function to identify the type of schema in multi-schema environments.\n\n- **Schema Version Tracking:**  \n  The `db_schema_version` table stores the history of schema versions applied to the database.\n\n- **Version Management Functions:**  \n  - `get_schema_version()`: Returns the current schema version in `major.minor.patch` format.\n  - `set_schema_version(major, minor, patch)`: Updates the schema version after migrations.\n\n## Setup\n\n1. **Initialize the schema versioning system:**  \n   Run the SQL statements in [`001_add_schema_versioning.sql`](sql/migrate/0.1/001_add_schema_versioning.sql) to set up the required functions and tables.\n\n2. **Check the current schema version:**  \n   ```sql\n   SELECT get_schema_version() AS version;\n   ```\n\n## Usage\n\nChanges to schemas are built up as a sequence of changes that can be applied, and an optional set of rollback scripts.\n\nThe changes should be decoupled from the code changes - so that old code can run with the the new schema changes, and \nthen updated code can be rolled out, before cleaning up unused columns.\n\nThe [`initdb.sql`](sql/initdb.sql) should be kept up to date as a current snapshot of the full DB for easy loading in new environments.\nThe *migrate* folder should be a complete record of changes.\n\nThe initdb.sql file can easily be updated with the following script (run from the sql directory)\n```bash\nfind migrate -name \"*.sql\" -type f | sort -V | while read file; do echo \"-- source: $file\";  cat \"$file\"; echo \"\\n\"; done \u003e initdb.sql\n```\n\nEach of the sql scripts in the migrate folder can be run manually or it can be scripted to run them in a sequence.\n\n\n## FAQs\n### Can I add this to an existing project?\nYes - add in the functions and pick a version number that makes sense.  It's a good idea to do a pgdump of the schema \nand set that as the initial file.  Then simply iterate version numbers.\n\n### Should this match the version number for my application?\nIn general No - as the DB and the application tend to evolve at different speeds and trying to match the version\ntends to make more useless busywork.\n\n### Do I need a rollback file for every change?\nIf you are using this in a major production application then you really should.  If you are working on a smaller\nor early stage project then it can be beneficial to skip them for flexibility.\n\n\n## Schema evolution best practice and rules.\n\nThis section gives concrete, actionable guidelines for evolving schemas while maintaining backward compatibility and minimizing risk.\n\n### Principles\n- Prefer additive, backward-compatible changes whenever possible.\n- Stage breaking changes: introduce new artifacts, shift traffic, then remove old ones.\n- Keep migrations small, reversible, and well-tested.\n- Record applied migrations in `db_schema_version` and keep the migration scripts in source control.\n\n### Backward compatibility rules\n- Add columns or tables instead of modifying/removing existing ones.\n- New columns should be nullable or have safe server-side defaults. Convert to NOT NULL only after backfill and validation.\n- Don't change column names or types in-place. Use dual-write/shadow columns and cutover later.\n- Keep stable function and view signatures; change implementation behind the stable API.\n\n### Safe-change patterns\n\n- Adding a column (recommended flow)\n  1. Add the column nullable with no heavy default.\n  2. Deploy application code that writes/reads the new column defensively.\n  3. Backfill values in a controlled batch job.\n  4. Validate backfill and then add NOT NULL / default in a separate migration.\n\n  Example (Postgres):\n  ```sql\n  ALTER TABLE users ADD COLUMN bio TEXT;\n  -- backfill as a separate job\n  UPDATE users SET bio = '' WHERE bio IS NULL;\n  ALTER TABLE users ALTER COLUMN bio SET NOT NULL;\n  ```\n\n- Renaming a column (safe approach)\n  1. Add the new column.\n  2. Update application to write to both old and new columns.\n  3. Backfill new column from old.\n  4. Switch reads to the new column.\n  5. Drop old column only after all clients use the new one.\n\n- Changing a type\n  1. Add a new column with target type.\n  2. Backfill using safe casts in batches.\n  3. Update app to read/write new column.\n  4. Remove old column once cutover is complete.\n\n- Constraints and indexes\n  - Create large indexes CONCURRENTLY (Postgres) to avoid blocking writes.\n  - Add constraints using NOT VALID, backfill, then VALIDATE:\n    ```sql\n    ALTER TABLE orders ADD CONSTRAINT chk_amount_positive CHECK (amount \u003e 0) NOT VALID;\n    -- backfill/fix rows\n    ALTER TABLE orders VALIDATE CONSTRAINT chk_amount_positive;\n    ```\n\n### Migrations \u0026 rollbacks\n- Write idempotent migrations that can be re-run safely.\n- Provide explicit rollback scripts; test both up and down paths in staging.\n- Break complex changes into multiple migrations (add, backfill, validate, cleanup).\n\n### Deployment and rollout\n- Follow a three-phase deployment: migrate (add artifacts) -\u003e deploy compatible app -\u003e cleanup (remove old artifacts).\n- Use feature flags to control behavior during rollout.\n- Run long-running migrations at low-traffic times or on replicas when feasible.\n- For distributed systems, ensure all services tolerate mixed schemas during transition.\n\n### Testing and observability\n- Test migrations locally and in staging with representative data sizes.\n- Canary changes on a small subset of instances or replicas.\n- Monitor migration duration, locks, errors, and application error rates.\n- Log every migration step and the db_schema_version changes.\n\n### Checklist for each schema change\n- [ ] Is the change additive or staged for compatibility?\n- [ ] Will applications tolerate both old and new schema shapes?\n- [ ] Is there a backfill plan and validation step?\n- [ ] Are migrations idempotent and reversible?\n- [ ] Have migrations been tested on staging with representative data?\n- [ ] Is monitoring and alerting configured for the migration?\n\nKeep migration scripts in the repository, include rationale and rollback plans in the commit message, and use `db_schema_version` to track progress. These practices reduce risk and enable predictable, low-downtime schema evolution.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarklynch%2Fsimple-schema-versions","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarklynch%2Fsimple-schema-versions","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarklynch%2Fsimple-schema-versions/lists"}