{"id":13878663,"url":"https://github.com/fatkodima/online_migrations","last_synced_at":"2025-05-14T04:00:31.111Z","repository":{"id":37572589,"uuid":"445702063","full_name":"fatkodima/online_migrations","owner":"fatkodima","description":"Catch unsafe PostgreSQL migrations in development and run them easier in production (code helpers for table/column renaming, changing column type, adding columns with default, background migrations, etc).","archived":false,"fork":false,"pushed_at":"2025-05-08T12:30:59.000Z","size":714,"stargazers_count":668,"open_issues_count":2,"forks_count":19,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-05-08T12:42:34.038Z","etag":null,"topics":["activerecord","gem","migrations","postgresql","rails","ruby"],"latest_commit_sha":null,"homepage":"https://rubydoc.info/github/fatkodima/online_migrations/","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fatkodima.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-01-08T02:08:31.000Z","updated_at":"2025-05-08T12:31:01.000Z","dependencies_parsed_at":"2022-07-14T02:10:41.020Z","dependency_job_id":"b3bb5da9-92e0-45b6-a7a1-56fc29948e20","html_url":"https://github.com/fatkodima/online_migrations","commit_stats":{"total_commits":363,"total_committers":20,"mean_commits":18.15,"dds":0.05509641873278237,"last_synced_commit":"4cd3545099a75b8cc3b535b4391e6e92bcf72ba7"},"previous_names":[],"tags_count":51,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Fonline_migrations","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Fonline_migrations/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Fonline_migrations/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatkodima%2Fonline_migrations/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fatkodima","download_url":"https://codeload.github.com/fatkodima/online_migrations/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254067078,"owners_count":22009074,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["activerecord","gem","migrations","postgresql","rails","ruby"],"created_at":"2024-08-06T08:01:56.075Z","updated_at":"2025-05-14T04:00:31.084Z","avatar_url":"https://github.com/fatkodima.png","language":"Ruby","funding_links":[],"categories":["Ruby","Gems","Database Tools"],"sub_categories":["Database Structure and Schema Management"],"readme":"# OnlineMigrations\n\nCatch unsafe PostgreSQL migrations in development and run them easier in production.\n\n:white_check_mark: Detects potentially dangerous operations\\\n:white_check_mark: Prevents them from running by default\\\n:white_check_mark: Provides instructions and helpers on safer ways to do what you want\n\n**Note**: You probably don't need this gem for smaller projects, as operations that are unsafe at scale can be perfectly safe on smaller, low-traffic tables.\n\n[![Build Status](https://github.com/fatkodima/online_migrations/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/fatkodima/online_migrations/actions/workflows/test.yml)\n\n## Cool, but there is a `strong_migrations` already\n\nSee [comparison to `strong_migrations`](#comparison-to-strong_migrations)\n\n## Requirements\n\n- Ruby 3.1+\n- Rails 7.1+\n- PostgreSQL 12+\n\nFor older Ruby and Rails versions you can use older versions of this gem.\n\n**Note**: Since some migration helpers use database `VIEW`s to implement their logic, it is recommended to use `structure.sql` schema format, or otherwise add some gem (like [scenic](https://github.com/scenic-views/scenic)) to be able to dump them into the `schema.rb`.\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'online_migrations'\n```\n\nAnd then run:\n\n```sh\n$ bundle install\n$ bin/rails generate online_migrations:install\n$ bin/rails db:migrate\n```\n\n**Note**: If you do not have plans on using [background data migrations](docs/background_data_migrations.md) or [background schema migrations](docs/background_schema_migrations.md) features, then you can delete the generated migration and regenerate it later, if needed.\n\n### Upgrading\n\nIf you're already using [background data migrations](docs/background_data_migrations.md) or [background schema migrations](docs/background_schema_migrations.md), your background migrations tables may require additional columns. After every upgrade run:\n\n```sh\n$ bin/rails generate online_migrations:upgrade\n$ bin/rails db:migrate\n```\n\n## Motivation\n\nWriting a safe migration can be daunting. Numerous articles have been written on the topic and a few gems are trying to address the problem. Even for someone who has a pretty good command of PostgreSQL, remembering all the subtleties of explicit locking can be problematic.\n\n**Online Migrations** was created to catch dangerous operations and provide a guidance and code helpers to run them safely.\n\nAn operation is classified as dangerous if it either:\n\n- Blocks reads or writes for more than a few seconds (after a lock is acquired)\n- Has a good chance of causing application errors\n\n## Example\n\nWhen you run a migration that's potentially dangerous, you'll see an error message like:\n\n```txt\n⚠️  [online_migrations] Dangerous operation detected ⚠️\n\nActive Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.\nA safer approach is to:\n\n1. Ignore the column:\n\n  class User \u003c ApplicationRecord\n    self.ignored_columns += [\"name\"]\n  end\n\n2. Deploy\n3. Wrap column removing in a safety_assured { ... } block\n\n  class RemoveColumn \u003c ActiveRecord::Migration[8.0]\n    def change\n      safety_assured { remove_column :users, :name }\n    end\n  end\n\n4. Remove column ignoring from step 1\n5. Deploy\n```\n\n## Checks\n\nPotentially dangerous operations:\n\n- [removing a column](#removing-a-column)\n- [adding a column with a default value](#adding-a-column-with-a-default-value)\n- [backfilling data](#backfilling-data)\n- [changing the type of a column](#changing-the-type-of-a-column)\n- [renaming a column](#renaming-a-column)\n- [renaming a table](#renaming-a-table)\n- [creating a table with the force option](#creating-a-table-with-the-force-option)\n- [adding a check constraint](#adding-a-check-constraint)\n- [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column)\n- [executing SQL directly](#executing-SQL-directly)\n- [adding an index non-concurrently](#adding-an-index-non-concurrently)\n- [removing an index non-concurrently](#removing-an-index-non-concurrently)\n- [replacing an index](#replacing-an-index)\n- [adding a reference](#adding-a-reference)\n- [adding a foreign key](#adding-a-foreign-key)\n- [adding an exclusion constraint](#adding-an-exclusion-constraint)\n- [adding a unique constraint](#adding-a-unique-constraint)\n- [adding a json column](#adding-a-json-column)\n- [adding a stored generated column](#adding-a-stored-generated-column)\n- [using primary key with short integer type](#using-primary-key-with-short-integer-type)\n- [hash indexes](#hash-indexes)\n- [adding multiple foreign keys](#adding-multiple-foreign-keys)\n- [removing a table with multiple foreign keys](#removing-a-table-with-multiple-foreign-keys)\n- [mismatched reference column types](#mismatched-reference-column-types)\n- [adding a single table inheritance column](#adding-a-single-table-inheritance-column)\n\nConfig-specific checks:\n\n- [changing the default value of a column](#changing-the-default-value-of-a-column)\n\nYou can also add [custom checks](docs/configuring.md#custom-checks) or [disable specific checks](docs/configuring.md#disable-checks).\n\n### Removing a column\n\n:x: **Bad**\n\nActive Record caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.\n\n```ruby\nclass RemoveNameFromUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    remove_column :users, :name\n  end\nend\n```\n\n:white_check_mark: **Good**\n\n1. Ignore the column:\n\n   ```ruby\n   class User \u003c ApplicationRecord\n     self.ignored_columns += [\"name\"]\n   end\n   ```\n\n2. Deploy\n3. Wrap column removing in a `safety_assured` block:\n\n   ```ruby\n   class RemoveNameFromUsers \u003c ActiveRecord::Migration[8.0]\n     def change\n       safety_assured { remove_column :users, :name }\n     end\n   end\n   ```\n\n4. Remove column ignoring from step 1\n5. Deploy\n\n### Adding a column with a default value\n\n:x: **Bad**\n\nIn earlier versions of PostgreSQL adding a column with a non-null default value to an existing table blocks reads and writes while the entire table is rewritten.\n\n```ruby\nclass AddAdminToUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_column :users, :admin, :boolean, default: false\n  end\nend\n```\n\nIn PostgreSQL 11+ this no longer requires a table rewrite and is safe. Volatile expressions, however, such as `random()`, will still result in table rewrites.\n\n:white_check_mark: **Good**\n\nA safer approach is to:\n\n1. add the column without a default value\n2. change the column default\n3. backfill existing rows with the new value\n\n`add_column_with_default` helper takes care of all this steps:\n\n```ruby\nclass AddAdminToUsers \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def change\n    add_column_with_default :users, :admin, :boolean, default: false\n  end\nend\n```\n\n**Note**: If you forget `disable_ddl_transaction!`, the migration will fail.\n\n### Backfilling data\n\n:x: **Bad**\n\nActive Record wraps each migration in a transaction, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).\n\n```ruby\nclass AddAdminToUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_column :users, :admin, :boolean\n    User.update_all(admin: false)\n  end\nend\n```\n\nAlso, running a single query to update data can cause issues for large tables.\n\n:white_check_mark: **Good**\n\nThere are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use a `update_column_in_batches` helper in a separate migration with `disable_ddl_transaction!`.\n\n```ruby\nclass AddAdminToUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_column :users, :admin, :boolean\n  end\nend\n\nclass BackfillUsersAdminColumn \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def up\n    update_column_in_batches(:users, :admin, false, pause_ms: 10)\n  end\nend\n```\n\n**Note**: If you forget `disable_ddl_transaction!`, the migration will fail.\n**Note**: You may consider [background data migrations](#background-data-migrations) or [background schema migrations](#background-schema-migrations) to run data changes on large tables.\n\n### Changing the type of a column\n\n:x: **Bad**\n\nChanging the type of an existing column blocks reads and writes while the entire table is rewritten.\n\n```ruby\nclass ChangeFilesSizeType \u003c ActiveRecord::Migration[8.0]\n  def change\n    change_column :files, :size, :bigint\n  end\nend\n```\n\nA few changes don't require a table rewrite (and are safe) in PostgreSQL:\n\nType | Safe Changes\n--- | ---\n`bit` | Changing to `bit_varying`\n`bit_varying` | Increasing or removing `:limit`\n`cidr` | Changing to `inet`\n`citext` | Changing to `text` if not indexed, changing to `string` with no `:limit` if not indexed\n`datetime` | Increasing or removing `:precision`, changing to `timestamptz` when session time zone is UTC in PostgreSQL 12+\n`decimal` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`\n`interval` | Increasing or removing `:precision`\n`numeric` | Increasing `:precision` at same `:scale`, removing `:precision` and `:scale`\n`string` | Increasing or removing `:limit`, changing to `text`, changing to `citext` if not indexed\n`text` | Changing to `string` with no `:limit`, changing to `citext` if not indexed\n`timestamptz` | Increasing or removing `:limit`, changing to `datetime` when session time zone is UTC in PostgreSQL 12+\n`xml` | Changing to `text`, changing to `string` with no `:limit`\n\n:white_check_mark: **Good**\n\n#### \"Classic\" approach (abstract)\n\n1. Create a new column\n2. Write to both columns\n3. Backfill data from the old column to the new column\n4. Move reads from the old column to the new column\n5. Stop writing to the old column\n6. Drop the old column\n\n#### :bullettrain_side: Concrete steps for Active Record\n\n**Note**: The following steps can also be used to change the primary key's type (e.g., from `integer` to `bigint`).\n\nA safer approach can be accomplished in several steps:\n\n1. Create a new column and keep column's data in sync:\n\n   ```ruby\n   class InitializeChangeFilesSizeType \u003c ActiveRecord::Migration[8.0]\n     def change\n       initialize_column_type_change :files, :size, :bigint\n     end\n   end\n   ```\n\n   **Note**: `initialize_column_type_change` accepts additional options (like `:limit`, `:default` etc)\n   which will be passed to `add_column` when creating a new column, so you can override previous values.\n\n2. Backfill data from the old column to the new column:\n\n   ```ruby\n   class BackfillChangeFilesSizeType \u003c ActiveRecord::Migration[8.0]\n     disable_ddl_transaction!\n\n     def up\n       # You can use `backfill_column_for_type_change_in_background` if want to\n       # backfill using background migrations.\n       backfill_column_for_type_change :files, :size\n     end\n\n     def down\n       # no op\n     end\n   end\n   ```\n\n3. Make sure your application works with values in both formats (when read from the database, converting\nduring writes works automatically). For most column type changes, this does not need any updates in the app.\n4. Deploy\n5. Copy indexes, foreign keys, check constraints, NOT NULL constraint, swap new column in place:\n\n   ```ruby\n   class FinalizeChangeFilesSizeType \u003c ActiveRecord::Migration[8.0]\n     disable_ddl_transaction!\n\n     def change\n       finalize_column_type_change :files, :size\n     end\n   end\n   ```\n\n6. Deploy\n7. Finally, if everything works as expected, remove copy trigger and old column:\n\n   ```ruby\n   class CleanupChangeFilesSizeType \u003c ActiveRecord::Migration[8.0]\n     def up\n       cleanup_column_type_change :files, :size\n     end\n\n     def down\n       initialize_column_type_change :files, :size, :integer\n     end\n   end\n   ```\n\n8. Remove changes from step 3, if any\n9. Deploy\n\n### Renaming a column\n\n:x: **Bad**\n\nRenaming a column that's in use will cause errors in your application.\n\n```ruby\nclass RenameUsersNameToFirstName \u003c ActiveRecord::Migration[8.0]\n  def change\n    rename_column :users, :name, :first_name\n  end\nend\n```\n\n:white_check_mark: **Good**\n\n#### \"Classic\" approach (abstract)\n\n1. Create a new column\n2. Write to both columns\n3. Backfill data from the old column to the new column\n4. Move reads from the old column to the new column\n5. Stop writing to the old column\n6. Drop the old column\n\n#### :bullettrain_side: Enhanced approach (with concrete steps for Active Record)\n\nThe \"classic\" approach suggests creating a new column and copy data/indexes/etc to it from the old column. This can be costly for very large tables. There is a trick that helps to avoid such heavy operations.\n\nThe technique is built on top of database views, using the following steps:\n\n1. Rename the table to some temporary name\n2. Create a VIEW using the old table name with addition of a new column as an alias of the old one\n3. Add a workaround for Active Record's schema cache\n\nFor the previous example, to rename `name` column to `first_name` of the `users` table, we can run:\n\n```sql\nBEGIN;\nALTER TABLE users RENAME TO users_column_rename;\nCREATE VIEW users AS SELECT *, first_name AS name FROM users_column_rename;\nCOMMIT;\n```\n\nAs database views do not expose the underlying table schema (default values, not null constraints, indexes, etc), further steps are needed to update the application to use the new table name. Active Record heavily relies on this data, for example, to initialize new models.\n\nTo work around this limitation, we need to tell Active Record to acquire this information from original table using the new table name.\n\n**Online Migrations** provides several helpers to implement column renaming:\n\n1. Instruct Rails that you are going to rename a column:\n\n   ```ruby\n   OnlineMigrations.config.column_renames = {\n     \"users\" =\u003e {\n       \"name\" =\u003e \"first_name\"\n     }\n   }\n   ```\n\n   **Note**: You also need to temporarily enable partial writes (is disabled by default in Active Record \u003e= 7)\n   until the process of column rename is fully done.\n\n   ```ruby\n   # config/application.rb\n   # For Active Record \u003e= 7\n   config.active_record.partial_inserts = true\n\n   # Or for Active Record \u003c 7\n   config.active_record.partial_writes = true\n   ```\n\n2. Deploy\n3. Tell the database that you are going to rename a column. This will not actually rename any columns,\nnor any data/indexes/foreign keys copying will be made, so will be instantaneous.\nIt will use a combination of a VIEW and column aliasing to work with both column names simultaneously\n\n   ```ruby\n   class InitializeRenameUsersNameToFirstName \u003c ActiveRecord::Migration[8.0]\n     def change\n       initialize_column_rename :users, :name, :first_name\n     end\n   end\n   ```\n\n4. Replace usages of the old column with a new column in the codebase\n5. If you enabled Active Record `enumerate_columns_in_select_statements` setting in your application\n   (is disabled by default in Active Record \u003e= 7), then you need to ignore old column:\n\n   ```ruby\n   class User \u003c ApplicationRecord\n     self.ignored_columns += [\"name\"]\n   end\n   ```\n\n6. Deploy\n7. Remove the column rename config from step 1\n8. Remove the column ignore from step 5, if added\n9. Remove the VIEW created in step 3 and finally rename the column:\n\n   ```ruby\n   class FinalizeRenameUsersNameToFirstName \u003c ActiveRecord::Migration[8.0]\n     def change\n       finalize_column_rename :users, :name, :first_name\n     end\n   end\n   ```\n\n10. Deploy\n\n### Renaming a table\n\n:x: **Bad**\n\nRenaming a table that's in use will cause errors in your application.\n\n```ruby\nclass RenameClientsToUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    rename_table :clients, :users\n  end\nend\n```\n\n:white_check_mark: **Good**\n\n#### \"Classic\" approach (abstract)\n\n1. Create a new table\n2. Write to both tables\n3. Backfill data from the old table to new table\n4. Move reads from the old table to the new table\n5. Stop writing to the old table\n6. Drop the old table\n\n#### :bullettrain_side: Enhanced approach (with concrete steps for Active Record)\n\nThe \"classic\" approach suggests creating a new table and copy data/indexes/etc to it from the old table. This can be costly for very large tables. There is a trick that helps to avoid such heavy operations.\n\nThe technique is built on top of database views, using the following steps:\n\n1. Rename the database table\n2. Create a VIEW using the old table name by pointing to the new table name\n3. Add a workaround for Active Record's schema cache\n\nFor the previous example, to rename `clients` table to `users`, we can run:\n\n```sql\nBEGIN;\nALTER TABLE clients RENAME TO users;\nCREATE VIEW clients AS SELECT * FROM users;\nCOMMIT;\n```\n\nAs database views do not expose the underlying table schema (default values, not null constraints, indexes, etc), further steps are needed to update the application to use the new table name. Active Record heavily relies on this data, for example, to initialize new models.\n\nTo work around this limitation, we need to tell Active Record to acquire this information from original table using the new table name.\n\n**Online Migrations** provides several helpers to implement table renaming:\n\n1. Instruct Rails that you are going to rename a table:\n\n   ```ruby\n   OnlineMigrations.config.table_renames = {\n     \"clients\" =\u003e \"users\"\n   }\n   ```\n\n2. Deploy\n3. Create a VIEW:\n\n   ```ruby\n   class InitializeRenameClientsToUsers \u003c ActiveRecord::Migration[8.0]\n     def change\n       initialize_table_rename :clients, :users\n     end\n   end\n   ```\n\n4. Replace usages of the old table with a new table in the codebase\n5. Remove the table rename config from step 1\n6. Deploy\n7. Remove the VIEW created in step 3:\n\n   ```ruby\n   class FinalizeRenameClientsToUsers \u003c ActiveRecord::Migration[8.0]\n     def change\n       finalize_table_rename :clients, :users\n     end\n   end\n   ```\n\n8. Deploy\n\n### Creating a table with the force option\n\n:x: **Bad**\n\nThe `force` option can drop an existing table.\n\n```ruby\nclass CreateUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    create_table :users, force: true do |t|\n      # ...\n    end\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nCreate tables without the `force` option.\n\n```ruby\nclass CreateUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    create_table :users do |t|\n      # ...\n    end\n  end\nend\n```\n\nIf you intend to drop an existing table, run `drop_table` first.\n\n### Adding a check constraint\n\n:x: **Bad**\n\nAdding a check constraint blocks reads and writes while every row is checked.\n\n```ruby\nclass AddCheckConstraint \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_check_constraint :users, \"char_length(name) \u003e= 1\", name: \"name_check\"\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nAdd the check constraint without validating existing rows, and then validate them in a separate transaction:\n\n```ruby\nclass AddCheckConstraint \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def change\n    add_check_constraint :users, \"char_length(name) \u003e= 1\", name: \"name_check\", validate: false\n    validate_check_constraint :users, name: \"name_check\"\n  end\nend\n```\n\n**Note**: If you forget `disable_ddl_transaction!`, the migration will fail.\n\n### Setting NOT NULL on an existing column\n\n:x: **Bad**\n\nSetting `NOT NULL` on an existing column blocks reads and writes while every row is checked.\n\n```ruby\nclass ChangeUsersNameNull \u003c ActiveRecord::Migration[8.0]\n  def change\n    change_column_null :users, :name, false\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nInstead, add a check constraint and validate it in a separate transaction:\n\n```ruby\nclass ChangeUsersNameNull \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def change\n    add_not_null_constraint :users, :name, name: \"users_name_null\", validate: false\n    validate_not_null_constraint :users, :name, name: \"users_name_null\"\n  end\nend\n```\n\n**Note**: If you forget `disable_ddl_transaction!`, the migration will fail.\n\nA `NOT NULL` check constraint is functionally equivalent to setting `NOT NULL` on the column (but it won't show up in `schema.rb` in Rails \u003c 6.1). In PostgreSQL 12+, once the check constraint is validated, you can safely set `NOT NULL` on the column and drop the check constraint.\n\n```ruby\nclass ChangeUsersNameNullDropCheck \u003c ActiveRecord::Migration[8.0]\n  def change\n    # in PostgreSQL 12+, you can then safely set NOT NULL on the column\n    change_column_null :users, :name, false\n    remove_check_constraint :users, name: \"users_name_null\"\n  end\nend\n```\n\n### Executing SQL directly\n\nOnline Migrations does not support inspecting what happens inside an `execute` call, so cannot help you here. Make really sure that what you're doing is safe before proceeding, then wrap it in a `safety_assured { ... }` block:\n\n```ruby\nclass ExecuteSQL \u003c ActiveRecord::Migration[8.0]\n  def change\n    safety_assured { execute \"...\" }\n  end\nend\n```\n\n### Adding an index non-concurrently\n\n:x: **Bad**\n\nAdding an index non-concurrently blocks writes.\n\n```ruby\nclass AddIndexOnUsersEmail \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_index :users, :email, unique: true\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nAdd indexes concurrently.\n\n```ruby\nclass AddIndexOnUsersEmail \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def change\n    add_index :users, :email, unique: true, algorithm: :concurrently\n  end\nend\n```\n\n**Note**: If you forget `disable_ddl_transaction!`, the migration will fail. Also, note that indexes on new tables (those created in the same migration) don't require this.\n\n### Removing an index non-concurrently\n\n:x: **Bad**\n\nWhile actual removing of an index is usually fast, removing it non-concurrently tries to obtain an `ACCESS EXCLUSIVE` lock on the table, waiting for all existing queries to complete and blocking all the subsequent queries (even `SELECT`s) on that table until the lock is obtained and index is removed.\n\n```ruby\nclass RemoveIndexOnUsersEmail \u003c ActiveRecord::Migration[8.0]\n  def change\n    remove_index :users, :email\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nRemove indexes concurrently.\n\n```ruby\nclass RemoveIndexOnUsersEmail \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def change\n    remove_index :users, :email, algorithm: :concurrently\n  end\nend\n```\n\n**Note**: If you forget `disable_ddl_transaction!`, the migration will fail.\n\n### Replacing an index\n\n:x: **Bad**\n\nRemoving an old index before replacing it with the new one might result in slow queries while building the new index.\n\n```ruby\nclass AddIndexOnCreationToProjects \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def change\n    remove_index :projects, :creator_id, algorithm: :concurrently\n    add_index :projects, [:creator_id, :created_at], algorithm: :concurrently\n  end\nend\n```\n\n**Note**: If removed index is covered by any existing index, then it is safe to remove the index before replacing it with the new one.\n\n:white_check_mark: **Good**\n\nA safer approach is to create the new index and then delete the old one.\n\n```ruby\nclass AddIndexOnCreationToProjects \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def change\n    add_index :projects, [:creator_id, :created_at], algorithm: :concurrently\n    remove_index :projects, :creator_id, algorithm: :concurrently\n  end\nend\n```\n\n### Adding a reference\n\n:x: **Bad**\n\nRails adds an index non-concurrently to references by default, which blocks writes. Additionally, if `foreign_key` option (without `validate: false`) is provided, both tables are blocked while it is validated.\n\n```ruby\nclass AddUserToProjects \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_reference :projects, :user, foreign_key: true\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nMake sure the index is added concurrently and the foreign key is added in a separate migration.\nOr you can use `add_reference_concurrently` helper. It will create a reference and take care of safely adding index and/or foreign key.\n\n```ruby\nclass AddUserToProjects \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def change\n    add_reference_concurrently :projects, :user\n  end\nend\n```\n\n**Note**: If you forget `disable_ddl_transaction!`, the migration will fail.\n\n### Adding a foreign key\n\n:x: **Bad**\n\nAdding a foreign key blocks writes on both tables.\n\n```ruby\nclass AddForeignKeyToProjectsUser \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_foreign_key :projects, :users\n  end\nend\n```\n\nor\n\n```ruby\nclass AddReferenceToProjectsUser \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_reference :projects, :user, foreign_key: true\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nAdd the foreign key without validating existing rows:\n\n```ruby\nclass AddForeignKeyToProjectsUser \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_foreign_key :projects, :users, validate: false\n  end\nend\n```\n\nThen validate them in a separate migration:\n\n```ruby\nclass ValidateForeignKeyOnProjectsUser \u003c ActiveRecord::Migration[8.0]\n  def change\n    validate_foreign_key :projects, :users\n  end\nend\n```\n\n### Adding an exclusion constraint\n\n:x: **Bad**\n\nAdding an exclusion constraint blocks reads and writes while every row is checked.\n\n```ruby\nclass AddExclusionContraint \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_exclusion_constraint :users, \"number WITH =\", using: :gist\n  end\nend\n```\n\n:white_check_mark: **Good**\n\n[Let us know](https://github.com/fatkodima/online_migrations/issues/new) if you have a safe way to do this (exclusion constraints cannot be marked `NOT VALID`).\n\n### Adding a unique constraint\n\n:x: **Bad**\n\nAdding a unique constraint blocks reads and writes while the underlying index is being built.\n\n```ruby\nclass AddUniqueConstraint \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_unique_constraint :sections, :position, deferrable: :deferred\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nA safer approach is to create a unique index first, and then create a unique key using that index.\n\n```ruby\nclass AddUniqueConstraintAddIndex \u003c ActiveRecord::Migration[8.0]\n  disable_ddl_transaction!\n\n  def change\n    add_index :sections, :position, unique: true, name: \"index_sections_on_position\", algorithm: :concurrently\n  end\nend\n```\n\n```ruby\nclass AddUniqueConstraint \u003c ActiveRecord::Migration[8.0]\n  def up\n    add_unique_constraint :sections, :position, deferrable: :deferred, using_index: \"index_sections_on_position\"\n  end\n\n  def down\n    remove_unique_constraint :sections, :position\n  end\nend\n```\n\n### Adding a json column\n\n:x: **Bad**\n\nThere's no equality operator for the `json` column type, which can cause errors for existing `SELECT DISTINCT` queries in your application.\n\n```ruby\nclass AddSettingsToProjects \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_column :projects, :settings, :json\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nUse `jsonb` instead.\n\n```ruby\nclass AddSettingsToProjects \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_column :projects, :settings, :jsonb\n  end\nend\n```\n\n### Adding a stored generated column\n\n:x: **Bad**\n\nAdding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked.\n\n```ruby\nclass AddLowerEmailToUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_column :users, :lower_email, :virtual, type: :string, as: \"LOWER(email)\", stored: true\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nAdd a non-generated column and use callbacks or triggers instead.\n\n### Using primary key with short integer type\n\n:x: **Bad**\n\nWhen using short integer types as primary key types, [there is a risk](https://m.signalvnoise.com/update-on-basecamp-3-being-stuck-in-read-only-as-of-nov-8-922am-cst/) of running out of IDs on inserts. The default type in Active Record \u003c 5.1 for primary and foreign keys is `INTEGER`, which allows a little over of 2 billion records. Active Record 5.1 changed the default type to `BIGINT`.\n\n```ruby\nclass CreateUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    create_table :users, id: :integer do |t|\n      # ...\n    end\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nUse one of `bigint`, `bigserial`, `uuid` instead.\n\n```ruby\nclass CreateUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    create_table :users, id: :bigint do |t| # bigint is the default for Active Record \u003e= 5.1\n      # ...\n    end\n  end\nend\n```\n\n### Hash indexes\n\n:x: **Bad - PostgreSQL \u003c 10**\n\nHash index operations are not WAL-logged, so hash indexes might need to be rebuilt with `REINDEX` after a database crash if there were unwritten changes. Also, changes to hash indexes are not replicated over streaming or file-based replication after the initial base backup, so they give wrong answers to queries that subsequently use them. For these reasons, hash index use is discouraged.\n\n```ruby\nclass AddIndexToUsersOnEmail \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_index :users, :email, unique: true, using: :hash\n  end\nend\n```\n\n:white_check_mark: **Good - PostgreSQL \u003c 10**\n\nUse B-tree indexes instead.\n\n```ruby\nclass AddIndexToUsersOnEmail \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_index :users, :email, unique: true # B-tree by default\n  end\nend\n```\n\n### Adding multiple foreign keys\n\n:x: **Bad**\n\nAdding multiple foreign keys in a single migration blocks writes on all involved tables until migration is completed.\nAvoid adding foreign key more than once per migration file, unless the source and target tables are identical.\n\n```ruby\nclass CreateUserProjects \u003c ActiveRecord::Migration[8.0]\n  def change\n    create_table :user_projects do |t|\n      t.belongs_to :user, foreign_key: true\n      t.belongs_to :project, foreign_key: true\n    end\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nAdd additional foreign keys in separate migration files. See [adding a foreign key](#adding-a-foreign-key) for how to properly add foreign keys.\n\n```ruby\nclass CreateUserProjects \u003c ActiveRecord::Migration[8.0]\n  def change\n    create_table :user_projects do |t|\n      t.belongs_to :user, foreign_key: true\n      t.belongs_to :project, foreign_key: false\n    end\n  end\nend\n\nclass AddForeignKeyFromUserProjectsToProject \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_foreign_key :user_projects, :projects\n  end\nend\n```\n\n### Removing a table with multiple foreign keys\n\n:x: **Bad**\n\nRemoving a table with multiple foreign keys blocks reads and writes on all involved tables until migration is completed.\nRemove all the foreign keys first.\n\nAssuming, `projects` has foreign keys on `users.id` and `repositories.id`:\n\n```ruby\nclass DropProjects \u003c ActiveRecord::Migration[8.0]\n  def change\n    drop_table :projects\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nRemove all the foreign keys first:\n\n```ruby\nclass RemoveProjectsUserFk \u003c ActiveRecord::Migration[8.0]\n  def change\n    remove_foreign_key :projects, :users\n  end\nend\n\nclass RemoveProjectsRepositoryFk \u003c ActiveRecord::Migration[8.0]\n  def change\n    remove_foreign_key :projects, :repositories\n  end\nend\n```\n\nThen remove the table:\n\n```ruby\nclass DropProjects \u003c ActiveRecord::Migration[8.0]\n  def change\n    drop_table :projects\n  end\nend\n```\n\n### Mismatched reference column types\n\n:x: **Bad**\n\nReference columns should be of the same type as the referenced primary key.\nOtherwise, there's a risk of bugs caused by IDs representable by one type but not the other.\n\nAssuming, there is a `users` table with `bigint` primary key type:\n\n```ruby\nclass AddUserIdToProjects \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_column :projects, :user_id, :integer\n  end\nend\n```\n\n:white_check_mark: **Good**\n\nAdd a reference column of the same type as a referenced primary key.\n\nAssuming, there is a `users` table with `bigint` primary key type:\n\n```ruby\nclass AddUserIdToProjects \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_column :projects, :user_id, :bigint\n  end\nend\n```\n\n### Adding a single table inheritance column\n\n:x: **Bad**\n\nAdding a single table inheritance column might cause errors in old instances of your application.\n\n```ruby\nclass AddTypeToUsers \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_column :users, :string, :type, default: \"Member\"\n  end\nend\n```\n\nAfter the migration was ran and the column was added, but before the code is fully deployed to all instances, an old instance may be restarted (due to an error etc). And when it will fetch 'User' records from the database, 'User' will look for a 'Member' subclass (from the 'type' column) and fail to locate it unless it is already defined.\n\n:white_check_mark: **Good**\n\nA safer approach is to:\n\n1. ignore the column:\n\n   ```ruby\n   class User \u003c ApplicationRecord\n     self.ignored_columns += [\"type\"]\n   end\n   ```\n\n2. deploy\n3. remove the column ignoring from step 1 and apply initial code changes\n4. deploy\n\n### Changing the default value of a column\n\n:x: **Bad**\n\nActive Record \u003c 7 enables partial writes by default, which can cause incorrect values to be inserted when changing the default value of a column.\n\n```ruby\nclass ChangeSomeColumnDefault \u003c ActiveRecord::Migration[8.0]\n  def change\n    change_column_default :users, :some_column, from: \"old\", to: \"new\"\n  end\nend\n\nUser.create!(some_column: \"old\") # can insert \"new\"\n```\n\n:white_check_mark: **Good**\n\nDisable partial writes in `config/application.rb`. For Active Record \u003c 7, use:\n\n```ruby\nconfig.active_record.partial_writes = false\n```\n\nFor Active Record 7, use:\n\n```ruby\nconfig.active_record.partial_inserts = false\n```\n\n## Assuring Safety\n\nTo mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.\n\n```ruby\nclass MySafeMigration \u003c ActiveRecord::Migration[8.0]\n  def change\n    safety_assured { remove_column :users, :some_column }\n  end\nend\n```\n\nCertain methods like `execute` and `change_table` cannot be inspected and are prevented from running by default. Make sure what you're doing is really safe and use this pattern.\n\n## Configuring the gem\n\nRead [configuring.md](docs/configuring.md).\n\n## Background Data Migrations\n\nRead [background_data_migrations.md](docs/background_data_migrations.md) on how to perform data migrations on large tables.\n\n## Background Schema Migrations\n\nRead [background_schema_migrations.md](docs/background_schema_migrations.md) on how to perform background schema migrations on large tables.\n\n## Credits\n\nThanks to [strong_migrations gem](https://github.com/ankane/strong_migrations), [GitLab](https://gitlab.com/gitlab-org/gitlab) and [maintenance_tasks gem](https://github.com/Shopify/maintenance_tasks) for the original ideas.\n\n## Contributing\n\nBug reports and pull requests are welcome on GitHub at https://github.com/fatkodima/online_migrations.\n\n## Development\n\nAfter checking out the repo, run `bundle install` to install dependencies. Run `createdb online_migrations_test` to create a test database. Then, run `bundle exec rake test` to run the tests. This project uses multiple Gemfiles to test against multiple versions of Active Record; you can run the tests against the specific version with `BUNDLE_GEMFILE=gemfiles/activerecord_61.gemfile bundle exec rake test`.\n\nTo install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).\n\n## Additional resources\n\nAlternatives:\n\n- https://github.com/ankane/strong_migrations\n- https://github.com/LendingHome/zero_downtime_migrations\n- https://github.com/braintree/pg_ha_migrations\n- https://github.com/doctolib/safe-pg-migrations\n\nInteresting reads:\n\n- [Explicit Locking](https://www.postgresql.org/docs/current/explicit-locking.html)\n- [When Postgres blocks: 7 tips for dealing with locks](https://www.citusdata.com/blog/2018/02/22/seven-tips-for-dealing-with-postgres-locks/)\n- [PostgreSQL rocks, except when it blocks: Understanding locks](https://www.citusdata.com/blog/2018/02/15/when-postgresql-blocks/)\n- [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/paypal-tech/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680)\n- [Adding a NOT NULL CONSTRAINT on PG Faster with Minimal Locking](https://medium.com/doctolib-engineering/adding-a-not-null-constraint-on-pg-faster-with-minimal-locking-38b2c00c4d1c)\n- [Adding columns with default values to really large tables in Postgres + Rails](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/)\n- [Safe Operations For High Volume PostgreSQL](https://www.braintreepayments.com/blog/safe-operations-for-high-volume-postgresql/)\n- [Stop worrying about PostgreSQL locks in your Rails migrations](https://medium.com/doctolib/stop-worrying-about-postgresql-locks-in-your-rails-migrations-3426027e9cc9)\n- [Avoiding integer overflows with zero downtime](https://buildkite.com/blog/avoiding-integer-overflows-with-zero-downtime)\n\n## Comparison to `strong_migrations`\n\nThis gem was heavily inspired by the `strong_migrations` and GitLab's approaches to database migrations. This gem is a superset of `strong_migrations`, feature-wise, and has the same APIs.\n\nThe main differences are:\n\n1. `strong_migrations` provides you **text guidance** on how to run migrations safer and you should implement them yourself. This new gem has actual [**code helpers**](https://github.com/fatkodima/online_migrations/blob/master/lib/online_migrations/schema_statements.rb) (and suggests them when fails on unsafe migrations) you can use to do what you want. See [example](#example) for an example.\n\n   It has migrations helpers for:\n\n     * renaming tables/columns\n     * changing columns types (including changing primary/foreign keys from `integer` to `bigint`)\n     * adding columns with default values\n     * backfilling data\n     * adding different types of constraints\n     * and others\n\n2. This gem has a powerful internal framework for running [data migrations](docs/background_data_migrations.md) and [schema migrations](docs/background_schema_migrations.md) on very large tables in background.\n\n   For example, you can use background migrations to migrate data that’s stored in a single JSON column to a separate table instead; backfill values from one column to another (as one of the steps when changing column type); or backfill some column’s value from an API.\n\n3. Yet, it has more checks for unsafe changes (see [checks](#checks)).\n\n4. Currently, this gem supports only PostgreSQL, while `strong_migrations` also checks `MySQL` and `MariaDB` migrations.\n\n5. This gem is more flexible in terms of configuration - see [config file](https://github.com/fatkodima/online_migrations/blob/master/lib/online_migrations/config.rb) for additional configuration options.\n\n## License\n\nThe gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatkodima%2Fonline_migrations","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffatkodima%2Fonline_migrations","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatkodima%2Fonline_migrations/lists"}