{"id":13879219,"url":"https://github.com/gocardless/nandi","last_synced_at":"2025-05-15T22:12:11.445Z","repository":{"id":44644473,"uuid":"190399576","full_name":"gocardless/nandi","owner":"gocardless","description":"Fear free PostgreSQL migrations for Rails","archived":false,"fork":false,"pushed_at":"2025-03-26T15:31:34.000Z","size":360,"stargazers_count":124,"open_issues_count":5,"forks_count":3,"subscribers_count":56,"default_branch":"master","last_synced_at":"2025-05-09T11:06:21.582Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gocardless.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-05T13:25:37.000Z","updated_at":"2025-04-10T19:34:38.000Z","dependencies_parsed_at":"2025-02-23T07:00:26.578Z","dependency_job_id":"902d26e3-fc4e-494d-acec-ac29c524394d","html_url":"https://github.com/gocardless/nandi","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gocardless%2Fnandi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gocardless%2Fnandi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gocardless%2Fnandi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gocardless%2Fnandi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gocardless","download_url":"https://codeload.github.com/gocardless/nandi/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254430335,"owners_count":22069909,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-06T08:02:13.861Z","updated_at":"2025-05-15T22:12:06.436Z","avatar_url":"https://github.com/gocardless.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"# Nandi\n\nFriendly Postgres migrations for people who don't want to take down their database to add a column!\n\n## Supported\n\n- Ruby 3.2 or above\n- Rails 7.1 or above\n- Postgres 11 or above\n\n## What does it do?\n\nNandi provides an alternative API to ActiveRecord's built-in Migration DSL for defining changes to your database schema.\n\nActiveRecord makes many changes easy. Unfortunately, that includes things that should be done with great care. Consider this migration, for example:\n\n```rb\nclass AddBarIDToFoos \u003c ActiveRecord::Migration[8.0]\n  def change\n    add_reference :foos, :bars, foreign_key: true\n  end\nend\n```\n\nThis is a perfectly ordinary thing to want to do - add a reference from one table to another and add a foreign key constraint, so that `bar_id` will always contain a value that appears in `bars`. But this actually takes very strict locks on both tables, `foos` and `bars`, while it checks that the constraint is valid. Depending on how large a table `bars` is, that could take a while; and if it does, your app will basically grind to a halt if it needs to access these tables. There are many such pitfalls around; and they generally only become dangerous when your database hits a certain size. There is hopefully a grizzled veteran engineer on your team who has memorised all the danger through bitter experience of 3am pages and 10 page post-mortems. But shouldn't we be able to do this with sofware, instead of scar tissue?\n\nEnter Nandi!\n\n![nandi](https://user-images.githubusercontent.com/2285130/56881872-bef6f500-6a59-11e9-8936-04d3861b6dce.gif)\n\nNandi offers availability-safe implementations of most common schema changes. It produces plain old ActiveRecord migration files, so existing Rails tooling can be leveraged for everything apart from correctness.\n\n## Getting started\n\nAdd to your Gemfile:\n\n```rb\ngem 'nandi'\ngem 'activerecord-safer_migrations' # Also required\n```\n\nGenerate a new migration:\n\n```sh\nrails generate nandi:migration add_widgets\n```\n\nYou'll get a fresh file, by default in `db/safe_migrations`. Let's use it to create a table with two fields, a name and a price, and the standard timestamps:\n\n```rb\n# db/safe_migrations/20190606060606_add_widgets.rb\n\nclass AddWidgets \u003c Nandi::Migration\n  def up\n    create_table :widgets do |t|\n      t.text :name\n      t.integer :price\n\n      t.timestamps\n    end\n  end\n\n  def down\n    drop_table :widgets\n  end\nend\n```\n\nLooks good! So let's generate an actual runnable ActiveRecord migration file.\n\n```sh\nrails generate nandi:compile\n```\n\nThe result will sort of look like this:\n\n```rb\n# db/migrate/20190606060606_add_widgets.rb\n\nclass AddWidgets \u003c ActiveRecord::Migration[8.0]\n  set_lock_timeout(750)\n  set_statement_timeout(1500)\n\n  def up\n    create_table :widgets do |t|\n      t.column :name, :text\n      t.column :price, :integer\n      t.timestamps\n    end\n  end\n\n  def down\n    drop_table :widgets\n  end\nend\n```\n\n(But not quite - the indentation is likely to be skewiff and some syntax will be oddly formatted. We have focused on making sure that the output is correct, rather than readable, although the dream is to one day have the same files that you would write yourself if you knew exactly what you were doing.)\n\nNow we can run the migration as we normally would.\n\n```sh\nrails db:migrate\n```\n\nAnd we're done!\n\nNow in this case, Nandi hasn't done much for us. It's explicitly set reasonable timeouts, so slow operations won't block other work indefinitely, and that's that. Let's try another.\n\n```rb\n# db/safe_migrations/20190606060606_add_widgets_index_on_name_and_price.rb\n\nclass AddWidgetsIndexOnNameAndPrice \u003c Nandi::Migration\n  def up\n    add_index :widgets, [:name, :price]\n  end\n\n  def down\n    remove_index :widgets, [:name, :price]\n  end\nend\n\n# db/migrate/20190606060606_add_widgets_index_on_name_and_price.rb\n\nclass AddWidgetsIndexOnNameAndPrice \u003c ActiveRecord::Migration[8.0]\n  set_lock_timeout(750)\n  set_statement_timeout(1500)\n\n  disable_ddl_transaction!\n  def up\n    add_index(\n      :widgets,\n      %i[name price],\n      name: :idx_widgets_on_name_price,\n      algorithm: :concurrently,\n      using: :btree,\n    )\n  end\n\n  def down\n    remove_index(\n      :widgets,\n      column: %i[name price],\n      algorithm: :concurrently,\n    )\n  end\nend\n```\n\nNandi has added in the `algorithm: :concurrently` option, ensuring that the index is not built immediately with the table locked in the meantime (a common source of pain). You can't use that option within a transaction, however, so Nandi uses the `disable_ddl_transaction!` macro. And we're ready to go.\n\nBut wait a minute - what about the foreign key one we started out with? The grizzled veterans among you know the workaround: add the constraint with the `NOT VALID` flag set, and then - in a separate follow-up transaction - validate the constraint. Nandi makes this easy:\n\n```sh\nrails generate nandi:foreign_key foos bars\n```\n\nWe now have three new migration files:\n\n```rb\n# db/safe_migrations/20190611124816_add_reference_on_foos_to_bars.rb\n\nclass AddReferenceOnFoosToBars \u003c Nandi::Migration\n  def up\n    add_column :foos, :bar_id, :bigint\n  end\n\n  def down\n    remove_column :foos, :bar_id\n  end\nend\n\n# db/safe_migrations/20190611124817_add_foreign_key_on_foos_to_bars.rb\n\nclass AddForeignKeyOnFoosToBars \u003c Nandi::Migration\n  def up\n    add_foreign_key :foos, :bars\n  end\n\n  def down\n    drop_constraint :foos, :foos_bars_fk\n  end\nend\n\n# db/safe_migrations/20190611124818_validate_foreign_key_on_foos_to_bars.rb\n\nclass ValidateForeignKeyOnFoosToBars \u003c Nandi::Migration\n  def up\n    validate_constraint :foos, :foos_bars_fk\n  end\n\n  def down; end\nend\n```\n\nWhich, when compiled, takes care of things in the right order:\n\n```rb\n# db/migrate/20190611124816_add_reference_on_foos_to_bars.rb\n\nclass AddReferenceOnFoosToBars \u003c ActiveRecord::Migration[8.0]\n  set_lock_timeout(5_000)\n  set_statement_timeout(1_500)\n\n  def up\n    add_column(:foos, :bar_id, :bigint)\n  end\n  def down\n    remove_column(:foos, :bar_id)\n  end\nend\n\n# db/migrate/20190611124817_add_foreign_key_on_foos_to_bars.rb\n\nclass AddForeignKeyOnFoosToBars \u003c ActiveRecord::Migration[8.0]\n  set_lock_timeout(750)\n  set_statement_timeout(1500)\n\n  def up\n    add_foreign_key(\n      :foos,\n      :bars,\n      { name: :foos_bars_fk, validate: false },\n    )\n  end\n\n  def down\n    execute \u003c\u003c-SQL\n    ALTER TABLE foos DROP CONSTRAINT foos_bars_fk\n    SQL\n  end\nend\n\n# db/migrate/20190611124818_validate_foreign_key_on_foos_to_bars.rb\n\n# frozen_string_literal: true\n\nclass ValidateForeignKeyOnFoosToBars \u003c ActiveRecord::Migration[8.0]\n  set_lock_timeout(750)\n  set_statement_timeout(1500)\n\n  def up\n    execute \u003c\u003c-SQL\n    ALTER TABLE foos VALIDATE CONSTRAINT foos_bars_fk\n    SQL\n  end\nend\n\n```\n\n## Class methods\n\n### `.set_lock_timeout(timeout)`\n\nOverride the default lock timeout for the duration of the migration. For migrations that require AccessExclusive locks, this is limited to 750ms.\n\n### `.set_statement_timeout(timeout)`\n\nOverride the default statement timeout for the duration of the migration. For migrations that require AccessExclusive locks, this is limited to 1500ms.\n\n## Migration methods\n\n### `#add_column(table, name, type, **kwargs)`\n\nAdds a new column. Nandi will explicitly set the column to be NULL, as validating a new NOT NULL constraint can be very expensive on large tables and cause availability issues.\n\n### `#add_foreign_key(table, target, column: nil, name: nil)`\n\nAdd a foreign key constraint. The generated SQL will include the NOT VALID parameter, which will prevent immediate validation of the constraint, which locks the target table for writes potentially for a long time. Use the separate #validate_constraint method, in a separate migration; this only takes a row-level lock as it scans through.\n\n### `#add_index(table, fields, **kwargs)`\n\nAdds a new index to the database.\n\nNandi will\n\n- add the `CONCURRENTLY` option, which means the change takes a less restrictive lock at the cost of not running in a DDL transaction\n- default to the `BTREE` index type, as it is commonly a good fit.\n\nBecause index creation is particularly failure-prone, and because we cannot run in a transaction and therefore risk partially applied migrations that (in a Rails environment) require manual intervention, Nandi Validates that, if there is a add_index statement in the migration, it must be the only statement.\n\n### `#create_table(table) {|columns_reader| ... }`\n\nCreates a new table. Yields a ColumnsReader object as a block, to allow adding columns.\n\nExamples:\n\n```rb\ncreate_table :widgets do |t|\n  t.text :foo, default: true\nend\n```\n\n### `#add_reference(table, ref_name, **extra_args)`\n\nAdds a new reference column. Nandi will validate that the foreign key flag is not set to true; use `add_foreign_key` and `validate_foreign_key` instead! Nandi will also set the `index: false` flag, as index creation is unsafe unless done concurrently in a separate migration.\n\n### `#remove_reference(table, ref_name, **extra_args)`\n\nRemoves a reference column.\n\n### `#remove_column(table, name, **extra_args)`\n\nRemove an existing column.\n\n### `#drop_constraint(table, name)`\n\nDrops an existing constraint.\n\n### `#remove_not_null_constraint(table, column)`\n\nDrops an existing NOT NULL constraint. Please not that this migration is not safely reversible; to enforce NOT NULL like behaviour, use a CHECK constraint and validate it in a separate migration.\n\n### `#change_column_default(table, column, value)`\n\nChanges the default value for this column when new rows are inserted into the table.\n\n### `#remove_index(table, target)`\n\nDrop an index from the database.\n\nNandi will add the `CONCURRENTLY` option, which means the change takes a less restrictive lock at the cost of not running in a DDL transaction.\nBecause we cannot run in a transaction and therefore risk partially applied migrations that (in a Rails environment) require manual intervention, Nandi Validates that, if there is a remove_index statement in the migration, it must be the only statement.\n\n### `#drop_table(table)`\n\nDrops an existing table.\n\n### `#irreversible_migration`\n\nRaises `ActiveRecord::IrreversibleMigration` error.\n\n## Generators\n\nSome schema changes need to be split across two migration files. Whenever you want to add a constraint to a column, you'll have to do this to avoid locking the table while Postgres validates that all existing data meets the constraint.\n\nFor some of the most common cases, we provide a Rails generator that generates both files for you.\n\n### Not-null checks\n\nTo generate migration files for a not-null check, run this command:\n\n```bash\nrails generate nandi:not_null_check foos bar\n```\n\nThis will generate two files:\n\n```\ndb/safe_migrations/20190424123727_add_not_null_check_on_bar_to_foos.rb\ndb/safe_migrations/20190424123728_validate_not_null_check_on_bar_to_foos.rb\n```\n\nFrom there, you can simply `rails generate nandi:compile` as usual and you're done!\n\n### Foreign key constraints\n\nYou may have spotted this generator in our worked example above. We've added it to this reference section too for completeness.\n\nThe simplest version of our foreign key migration generator is:\n\n```\nrails generate nandi:foreign_key foos bars\n```\n\nIt will generate three files like these:\n\n```\ndb/safe_migrations/20190424123726_add_reference_on_foos_to_bars.rb\ndb/safe_migrations/20190424123727_add_foreign_key_on_bars_to_foos.rb\ndb/safe_migrations/20190424123728_validate_foreign_key_on_bars_to_foos.rb\n```\n\nIf you're adding the constraint to a column that already exists, you can use the `--no-create-column` flag to skip the first migration:\n\n```\nrails generate nandi:foreign_key foos bars --no-create-column\n```\n\nIf your foreign key column is named differently, you can override it with the `--column` flag as seen in this example:\n\n```\nrails generate nandi:foreign_key foos bar --no-create-column --column special_bar_ids\n```\n\nWe generate the name of your foreign key for you. If you want or need to override it (e.g. if it exceeds the max length of a constraint name in Postgres), you can use the `--name` flag:\n\n```\nrails generate nandi:foreign_key foos bar --name my_fk\n```\n\n## Configuration\n\nNandi can be configured in various ways, typically in an initializer:\n\n```rb\nNandi.configure do |config|\n  config.lock_timeout = 1_000\nend\n```\n\nThe configuration parameters are as follows.\n\n### `access_exclusive_lock_timeout_limit` (Integer)\n\nThe maximum lock timeout for migrations that take an ACCESS EXCLUSIVE lock and therefore block all reads and writes. Default: 5,000ms.\n\n### `access_exclusive_statement_timeout_limit` (Integer)\n\nThe maximum statement timeout for migrations that take an ACCESS EXCLUSIVE lock and therefore block all reads and writes. Default: 1,500ms.\n\n### `concurrent_statement_timeout_limit` (Integer)\n\nThe minimum statement timeout for migrations that take place concurrently. Default: 3,600,000ms (ie, 3 hours).\n\n### `lock_timeout` (Integer)\n\nThe default lock timeout for migrations. Can be overridden by way of the `set_lock_timeout` class method in a given migration. Default: 5,000ms.\n\n### `migration_directory` (String)\n\nThe directory for Nandi migrations. Default: `db/safe_migrations`\n\n### `output_directory` (String)\n\nThe directory for output files. Default: `db/migrate`\n\n### `renderer` (Class)\n\nThe rendering backend used to produce output. The only supported option at current is `Nandi::Renderers::ActiveRecord`, which produces ActiveRecord migrations.\n\n### `statement_timeout` (Integer)\n\nThe default statement timeout for migrations that take permissive locks. Can be overridden by way of the `set_statement_timeout` class method in a given migration. Default: 10,800,000ms (ie, 3 hours).\n\n### `access_exclusive_statement_timeout` (Integer)\n\nThe default statement timeout for migrations that take ACCESS EXCLUSIVE locks. Can be overridden by way of the `set_statement_timeout` class method in a given migration. Default: 1500ms.\n\n### `compile_files` (String)\n\nThe files to compile when the compile generator is run. Default: `all`\n\nMay be one of the following:\n\n- 'all' compiles all files\n- 'git-diff' only files changed since last commit\n- a full or partial version timestamp, eg '20190101010101', '20190101'\n- a timestamp range , eg '\u003e=20190101010101'\n\n### `lockfile_directory` (String)\n\nThe directory where .nandilock.yml will be stored. Default: `db/` in working directory.\n\n#post_process {|migration| ... }\n\nRegister a block to be called on output, for example a code formatter. Whatever is returned will be written to the output file.\n\n```rb\nconfig.post_process { |migration| MyFormatter.format(migration) }\n```\n\n#register_method(name, klass)\n\nRegister a custom DDL method.\n\nParameters:\n\n`name` (Symbol) - The name of the method to create. This will be monkey-patched into Nandi::Migration.\n\n`klass` (Class) — The class to initialise with the arguments to the method. It should define a `template` instance method which will return a subclass of Cell::ViewModel from the Cells templating library and a `procedure` method that returns the name of the method. It may optionally define a `mixins` method, which will return an array of `Module`s to be mixed into any migration that uses this method.\n\n## `.nandiignore`\n\nTo protect people from writing unsafe migrations, we provide a script [`nandi-enforce`](https://github.com/gocardless/nandi/blob/master/exe/nandi-enforce) that ensures all migrations in the specified directories are safe migrations generated by Nandi.\n\nIn the off cases where you need to write a migration by hand, add a `.nandiignore` to the root of your repository with your migration files:\n\n```\ndb/migrate/20190324144824_my_handwritten_migration.rb\ndb/migrate/20190327130801_another_handwritten_migration.rb\ndb/migrate/20190327134957_one_more_handwritten_migration.rb\n```\n\n## Why Nandi?\n\nYou may have noticed a GIF of an adorable baby elephant above. This elephant is called Nandi, and she was the star of many an internal presentation slide here at GoCardless. Of course, Postgres is elephant-themed; but it is sometimes an angry elephant, motivating the creation of gems like this one. What better mascot than a harmless, friendly calf?\n\n## Generate documentation\n\n```sh\nbundle exec yard\n```\n\n## Run tests\n\n```sh\nbundle exec rspec\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgocardless%2Fnandi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgocardless%2Fnandi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgocardless%2Fnandi/lists"}