{"id":14069287,"url":"https://github.com/blainehansen/postgres_migrator","last_synced_at":"2025-04-04T11:12:49.802Z","repository":{"id":53492528,"uuid":"427857492","full_name":"blainehansen/postgres_migrator","owner":"blainehansen","description":"A postgres migration generator and runner that uses raw declarative sql.","archived":false,"fork":false,"pushed_at":"2024-10-21T16:19:19.000Z","size":89,"stargazers_count":91,"open_issues_count":0,"forks_count":6,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-10-22T04:28:37.190Z","etag":null,"topics":["database","database-migrations","migration","migrations","postgres","raw-sql","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/blainehansen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["blainehansen"]}},"created_at":"2021-11-14T06:44:41.000Z","updated_at":"2024-10-21T16:19:23.000Z","dependencies_parsed_at":"2024-09-11T03:40:11.733Z","dependency_job_id":"7ca23192-f827-45c8-8d34-7cc2e9b4ec8d","html_url":"https://github.com/blainehansen/postgres_migrator","commit_stats":{"total_commits":49,"total_committers":1,"mean_commits":49.0,"dds":0.0,"last_synced_commit":"7ceb50a17a03526f8affa0d7ed3e9dbd07404699"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blainehansen%2Fpostgres_migrator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blainehansen%2Fpostgres_migrator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blainehansen%2Fpostgres_migrator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blainehansen%2Fpostgres_migrator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/blainehansen","download_url":"https://codeload.github.com/blainehansen/postgres_migrator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247166168,"owners_count":20894654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","database-migrations","migration","migrations","postgres","raw-sql","rust"],"created_at":"2024-08-13T07:06:48.588Z","updated_at":"2025-04-04T11:12:49.775Z","avatar_url":"https://github.com/blainehansen.png","language":"Rust","readme":"# `postgres_migrator`\n\n`postgres_migrator` allows you to write your postgres schema directly in *declarative sql*, to automatically generate migrations when you change that declarative schema, and to apply those migrations with rigorous version tracking and consistency checks.\n\n**No more orms!** Use the full power of postgres directly without having to manually write migrations.\n\n`postgres_migrator` is able to:\n\n- Automatically generate raw sql migrations by diffing the sql in a migrations folder to that in a schema folder.\n- Apply those migrations and save migration version numbers in the database, so you can know what state a database is supposed to be in.\n- Run consistency checks on migrations, so that they are only ever applied in the order they were generated. This can prevent very painful debugging and database corruption.\n\n`postgres_migrator` intentionally doesn't do the following:\n\n- Create \"down\" versions of migrations. If you want to undo something in production, just make a new migration (that's best practice anyway). In dev just force the database into the right state.\n- Allow running migrations only up to a certain version. `postgres_migrator` will always apply all available unapplied migrations. If you don't want to apply some migrations, move them to a different folder or change their extension to something other than `.sql`.\n- Figure out database diffs itself, but instead uses the well-establised [`migra`](https://github.com/djrobstep/migra) under the hood.\n\n# Example\n\nIf your `schema` directory contains a sql file like this:\n\n```sql\ncreate table fruit (\n  id serial primary key,\n  name text not null unique,\n  color text not null default ''\n);\n```\n\nThen running `postgres_migrator generate 'add fruit table'` will generate a migration called `$new_version.$previous_version.add_fruit_table.sql` in the `migrations` folder.\n\nYou can then run `postgres_migrator migrate` to run this migration (and any others that haven't been run).\n\nIf you then change your schema sql to this:\n\n```sql\ncreate type flavor_type as enum('SWEET', 'SAVORY');\n\ncreate table fruit (\n  id serial primary key,\n  name text not null unique,\n  flavor flavor_type not null default 'SWEET'\n);\n```\n\nThen running `postgres_migrator generate 'remove color add flavor'` will generate `$new_version.$previous_version.remove_color_add_flavor.sql` that will go from the previous state to the new state.\n\n# Usage\n\nFirst, place your declarative sql files in the `schema` directory and create a directory for migrations called `migrations`. You can customize these with `--schema-directory` and `--migrations-directory`.\n\nYou can put any valid sql in the `migrations` files, you don't have to accept the automatically generated sql as is. All this tool cares about is the difference between the final sql objects created by sql in the `schema` and `migrations` directories, [and however you achieve that is up to you!](https://github.com/blainehansen/postgres_migrator/issues/4)\n\n## As a standalone docker image\n\n`postgres_migrator` is distributed as a docker image, `blainehansen/postgres_migrator`. You can run it using `docker run`, and since the cli needs to interact with a postgres database, read schema files, and read/write migration files, it needs quite a few options:\n\n\n```bash\ndocker run --rm -it --network host -u $(id -u ${USER}):$(id -g ${USER}) -v $(pwd):/working blainehansen/postgres_migrator \u003cargs\u003e\n```\n\nTo make this easier to manage, you can package that command in a function, alias, or script:\n\n```bash\nfunction postgres_migrator {\n  local result=$(docker run --rm -it --network host -u $(id -u ${USER}):$(id -g ${USER}) -v $(pwd):/working -e PG_URL=$PG_URL blainehansen/postgres_migrator \"$@\")\n  echo $result\n  return $?\n}\n\n# or\nalias postgres_migrator=\"docker run --rm -it --network host -u $(id -u ${USER}):$(id -g ${USER}) -v $(pwd):/working -e PG_URL=$PG_URL blainehansen/postgres_migrator\"\n\n# or in it's own executable file\ndocker run --rm -it --network host -u $(id -u ${USER}):$(id -g ${USER}) -v $(pwd):/working -e PG_URL=$PG_URL blainehansen/postgres_migrator \"$@\"\n\n# now you can call it more cleanly\npostgres_migrator generate 'adding users table'\npostgres_migrator migrate\n```\n\n## With docker compose\n\nMany people run postgres as one service in a docker compose setup, either using a `postgres` image or something like `cloud-sql-proxy`.\n\n```yml\nversion: '3'\nservices:\n  postgres_migrator:\n    image: blainehansen/postgres_migrator\n    container_name: postgres_migrator\n    depends_on:\n      - db\n    environment:\n      - PG_URL=postgres://experiment_user:asdf@db:5432/experiment_db?sslmode=disable\n    volumes:\n      - .:/working/\n    tty: true\n    entrypoint: tail -F anything\n\n  db:\n    image: postgres:alpine\n    container_name: db\n    environment:\n      - POSTGRES_DB=experiment_db\n      - POSTGRES_USER=experiment_user\n      - POSTGRES_PASSWORD=asdf\n    ports:\n      - \"5432:5432\"\n    command: postgres -c 'max_wal_size=2GB'\n```\n\nIf that is running, a command like this should work:\n\n```sh\ndocker exec -it -u $(id -u ${USER}):$(id -g ${USER}) postgres_migrator postgres_migrator migrate\n```\n\n## As a Rust binary\n\nThis package is published to [crates.io](https://crates.io/crates/postgres_migrator), so you can use `cargo install postgres_migrator` to install it.\n\nThe package calls the [`migra`](https://github.com/djrobstep/migra) command, so that must be installed and runnable.\n\n---\n\nCli usage:\n\n```\nUSAGE:\n    postgres_migrator [OPTIONS] --pg-url \u003cPG_URL\u003e \u003cSUBCOMMAND\u003e\n\nOPTIONS:\n    -h, --help\n            Print help information\n\n        --pg-url \u003cPG_URL\u003e\n            postgres connection string, in the form postgres://user:password@host:port/database\n            can also be loaded from the environment variable PG_URL [env: PG_URL=]\n\n        --migrations-directory \u003cMIGRATIONS_DIRECTORY\u003e\n            directory where migrations are stored [default: migrations]\n\n        --schema-directory \u003cSCHEMA_DIRECTORY\u003e\n            directory where the declarative schema is located [default: schema]\n\n    -V, --version\n            Print version information\n\nSUBCOMMANDS:\n    generate    generate new migration and place in migrations folder\n    migrate     apply all migrations to database\n    check       checks that `source` and `target` are in sync, throws error otherwise\n    diff        prints out the sql diff necessary to convert `source` to `target`\n    compact     ensure both database and migrations folder are current with schema and compact\n                to only one migration\n    clean       cleans the current instance of all temporary databases\n    help        Print this message or the help of the given subcommand(s)\n```\n\n## How to use with an existing database?\n\nIf you already have a database with an existing schema, you need to generate your first migration using the `--is-onboard` flag:\n\n```bash\npostgres_migrator generate 'first migration with postgres_migrator' --is-onboard\n# migrations folder should have migration named `$new_version.onboard.first_migration_with_postgres_migrator.sql`\n```\n\nThe `--is-onboard` flag changes the first migration to be an \"onboarding\" migration. When this migration is run, the actual sql in the migration won't be applied, and instead `postgres_migrator` will just create the `_schema_versions` table and insert the version of the migration.\n\nAfter you've created this first \"onboarding\" migration, and can just use `postgres_migrator` as usual!\n\n## What is `compact`?\n\nOver time a migrations folder can get large and unwieldy, with possibly hundreds of migrations. This long log gets less and less useful over time, especially for small teams. The `compact` command replaces all migrations with a single migration that creates the entire schema at once.\n\nSome teams will consider this dangerous and unnecessary, and they're free to not use it!\n\n# Credits\n\n- [`migra`](https://github.com/djrobstep/migra) for making it possible to diff schemas.\n- [`tusker`](https://github.com/bikeshedder/tusker) was the inspiration for using temporary databases as diff targets. `postgres_migrator` adds the ability to generate and run versioned migrations and to perform compaction.\n- Thank you [Rust](https://www.rust-lang.org/) for being so awesome! [clap](https://github.com/clap-rs/clap) and [rust-postgres](https://github.com/sfackler/rust-postgres) in particular made this way easier.\n\n# Contributing\n\nPull requests to make the script more ergonomic or robust are welcome.\n","funding_links":["https://github.com/sponsors/blainehansen"],"categories":["Rust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblainehansen%2Fpostgres_migrator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblainehansen%2Fpostgres_migrator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblainehansen%2Fpostgres_migrator/lists"}