{"id":22980528,"url":"https://github.com/kcartlidge/migratable","last_synced_at":"2025-07-04T06:37:01.775Z","repository":{"id":65533636,"uuid":"133154738","full_name":"kcartlidge/migratable","owner":"kcartlidge","description":"Simple and efficient database migrations supporting multiple database technologies.  Command line tool and optional dotnet nuget packages.","archived":false,"fork":false,"pushed_at":"2024-11-08T00:41:12.000Z","size":55696,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-12-08T07:32:46.561Z","etag":null,"topics":["database","migrations"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kcartlidge.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-12T14:42:06.000Z","updated_at":"2024-11-08T00:41:18.000Z","dependencies_parsed_at":"2024-10-23T13:00:38.448Z","dependency_job_id":null,"html_url":"https://github.com/kcartlidge/migratable","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcartlidge%2Fmigratable","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcartlidge%2Fmigratable/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcartlidge%2Fmigratable/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kcartlidge%2Fmigratable/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kcartlidge","download_url":"https://codeload.github.com/kcartlidge/migratable/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229773560,"owners_count":18122031,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","migrations"],"created_at":"2024-12-15T01:43:49.686Z","updated_at":"2024-12-15T01:43:50.046Z","avatar_url":"https://github.com/kcartlidge.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Migratable\n\nSimple and efficient database migrations supporting multiple database technologies.\n\nAvailable as both a **cross-platform CLI tool** suitable for any ecosystem (eg *DotNet, Go, Node etc*) and a set of **nuget packages** for inclusion in your own DotNet apps (eg to automate migrations).\n\n*Runs on Linux, Mac (Intel/ARM), and Windows. Currently supports PostgreSQL and MySQL.*\n\n## Contents\n\n- [About the cross-platform CLI tool](#about-the-cross-platform-cli-tool)\n- [Benefits of both the CLI and the Nuget packages](#benefits-of-both-the-cli-and-the-nuget-packages)\n- [What's available?](#whats-available)\n- [Requirements](#requirements)\n  - [A database connection string in an environment variable](#a-database-connection-string-in-an-environment-variable)\n  - [A folder of SQL migration scripts](#a-folder-of-sql-migration-scripts)\n- [Using Migratable via the CLI tool](#using-migratable-via-the-cli-tool)\n- [Using the Migratable Nuget packages](#using-the-migratable-nuget-packages)\n  - [How it works](#how-it-works)\n  - [Timeline Display](#timeline-display)\n- [Note about MySQL](#note-about-mysql)\n- [For developers working on Migratable itself](#for-developers-working-on-migratable-itself)\n  - [Running the tests](#running-the-tests)\n  - [Creating a new version for Nuget](#creating-a-new-version-for-nuget)\n  - [Forcing another project to get the latest from Nuget](#forcing-another-project-to-get-the-latest-from-nuget)\n- [MIT Licence](#mit-licence)\n\n---\n\n## About the cross-platform CLI tool\n\n- It's *not dependent on the DotNet platform*\n- It fits any ecosystem (eg *Go, Python, Node etc*)\n- It's easy to configure, needing only:\n  - A connection string in an environment variable\n  - A folder with named migrations using up/down SQL scripts\n\n## Benefits of both the CLI and the Nuget packages\n\n- Your database structure can be *version-controlled*\n- Roll your database backwards as well as forwards\n- Seed/pre-populate, update, or remove data\n- Run in *transactions* for atomic up/down\n- Uses the [MIT licence](./LICENCE)\n\n## What's available?\n\nCross-platform releases (*not dependent on DotNet*):\n\n- [Command-line tool](./builds)\n\nPackages available on *Nuget* for DotNet:\n\n- [Migratable](https://www.nuget.org/packages/Migratable)\n- [Migratable.PostgresProvider](https://www.nuget.org/packages/Migratable.PostgresProvider)\n- [Migratable.MySqlProvider](https://www.nuget.org/packages/Migratable.MySqlProvider)\n\n## Requirements\n\nThese requirements are the same whether you are using the command-line CLI tool or the Nuget packages.\n\nYou need a suitable database server installed (either MySQL or PostgreSQL). You also need the following:\n\n### A database connection string in an environment variable\n\nYou can use any environment variable name you like.\nHere's an example for Linux/Mac connecting to PostgreSQL:\n\n``` sh\nexport MY_CONNSTR=\"Server=127.0.0.1;Port=5432;Database=my_database;Search Path=my_schema;User Id=my_user;Password=my_password\"\n```\n\n*(This is an example, not a revealed secret.)*\n\nIt's similar for Windows, but you'd replace `export` with `set` instead.  However in Windows it's probably simpler to edit the environment variables from your Control Panel / Settings area.\n\n### A folder of SQL migration scripts\n\nA single folder holds all migration scripts.\nInside is a subfolder per migration, where the name begins with the migration sequence number (optional leading zeros) followed by a description.\nEach migration in turn consists of an `up.sql` file and a `down.sql` file.\n\n``` text\n/migrations\n  /001 Create account table\n    down.sql\n    up.sql\n  /002 Create news table\n    down.sql\n    up.sql\n  /003 Insert sample data\n    down.sql\n    up.sql\n```\n\n[There's an example folder here](./Example/migrations).\n\nYou must start at version one and you cannot omit a version in the sequence.  You may also not have duplicate version numbers.\n\n## Using Migratable via the CLI tool\n\n- [Download a release](./builds) and place it somewhere convenient\n- Add the connection string into your environment\n- Create the folder of SQL migration scripts\n- Run the CLI\n\nOn a Mac for example you could do:\n\n``` sh\n# Create a folder and copy the builds into it.\ncd \u003crepository_root\u003e\nsudo mkdir -p /usr/local/bin/Migratable\nsudo cp -r builds/macos-arm64/* /usr/local/bin/Migratable\n\n# Run it.\n/usr/local/bin/Migratable/Migratable.CLI postgres my_connstr ~/my-app/migrations --info\n```\n\nIn the above example the parameters are:\n\n- `postgres` is the database type (`postgres` or `mysql`)\n- `my_connstr` is the name of the environment variable containing your database connection string\n- `~/my-app/migrations` is the folder containing all your migrations\n- `--info` is the Migratable command to run\n\nYou can run without any command arguments to get a summary of usage:\n\n``` text\nUsage:\n  migratable \u003cdb\u003e \u003cenv_name\u003e \u003cmigrations\u003e \u003ccommand\u003e\n\n\n  db          database; either POSTGRES or MYSQL\n  env_name    the name of an environment variable\n              with a database connection string\n  migrations  folder containing migration scripts\n              (https://github.com/kcartlidge/migratable)\n  command     migration action to perform\n\nCommands:\n\n  info        Show migration status\n  list        List known migrations\n\n  reset       Remove all migrations\n  latest      Apply new migrations\n  next        Roll forward one migration\n  back        Roll backward one migration\n  target=0    Target specific migration\n```\n\n---\n\nIf it fails to run on Linux and Mac one of the following may help:\n\n- Run `chmod +x Migratable.CLI` to ensure the CLI tool is executable\n- Open it in Finder/Nemo/whatever first to deal with any security unlocking needed\n\n## Using the Migratable Nuget packages\n\nThere is an [Example project](./Example) in this solution.  It's totally self-contained as it uses an in-memory provider.\nIt also shows the use of a `Timeline` to show the known migrations and the current status.\n\nHere's some sample code to show the packages in use. `SampleProvider` is defined in the example project just mentioned. You could also look at the (simple and commented) [code for the CLI tool](./Migratable.CLI).\n\n``` csharp\n// Configure.\nvar provider = new SampleProvider();\nvar migrator = new Migratable.Migrator(provider);\nvar migrations = migrator.LoadMigrations(\"./migrations\");\n\n// Show the known migrations and current database position.\nvar timeline = new Timeline(provider, migrations, 3);\ntimeline.Show();\n\n// Confirm the connection.\nConsole.WriteLine(migrator.Describe());\nConsole.Write(\"Press enter/return to continue (or Ctrl+C) ... \");\nConsole.ReadLine();\n\n// Migrate from the current version to version 5.\nConsole.WriteLine($\"Old version: {migrator.GetVersion()}\");\nmigrator.SetVersion(5);\nConsole.WriteLine($\"New version: {migrator.GetVersion()}\");\n```\n\nThe `Describe()` method is designed to give confidence in proceeding. For MySQL/MariaDB it shows the server and database name.\n\n### How it works\n\nThere are 2 necessary components, with two optional ones:\n\n- *Migrator* - what your code should interact with to load/perform migrations\n- *Provider* - a utility package to support a particular database technology\n- *Notifier* - an *optional* class that can be sent progress messages for you to output to the console, log to a file, send via email, or whatever you choose\n  - The Notifier 'hook' is there but no delivery mechanism is provided, although the example project shows that a [console notifier](./Example/SampleNotifier.cs) is just one line of code\n- *Timeline* - an *optional* class that can show/return a list of migrations with the current migration position\n\nIn brief, you follow this process:\n\n- Create a Provider instance for your database\n- Create a Migrator and pass in your Provider\n- Optionally create a Notifier and pass that to the Migrator\n- Ask your Migrator to load your migrations\n- Ask your Migrator to perform versioned actions\n- Optionally use the Timeline before and/or after applying actions\n\nThat final Migrator step will result in your up/down SQL statements being issued as needed to transition from your current database version to your target one.\nThis is supported by an automatically created/updated `migratable_version` table.\n\n### Timeline Display\n\nThe `Timeline` class has options to either write a timeline to the `Console` or to return a `List\u003cstring\u003e` containing the equivalent text.\nIf you're writing to the screen a blank separator line is included before and after; this is not present if you request the text.\n\nWhen creating a new `Timeline` instance, the `2` is an optional indent for the text to ensure it fits with your own output. You can also provide overrides for the title and the position text.\n\n``` csharp\nvar timeline = new Timeline(provider, migrations, 2);\n...\nmigrator.SetVersion(2);\ntimeline.Show();\n```\n\nDepending upon the migrations (this is based on one of the test fixtures) it would give something like this:\n\n``` text\n  STATUS\n  001 Create accounts\n  002 Populate accounts\n  \u003c-- YOU ARE HERE\n  003 Create themes\n```\n\n## Note about MySQL\n\nMySQL has a habit of silently committing structural changes (add column, create table etc) mid-transaction.\nYou should therefore avoid using multiple statements in a single migration if any one of them is structural.\nIf you do, and one of the other statements fails, the transaction rollback may fail to undo a structural change that has already been applied in the current migration script.\n\n---\n\n## For developers working on Migratable itself\n\n*If you only intend making use of Migratable in your own projects read no further.*\n\n### Running the tests\n\n``` sh\ncd Migratable.Tests\ndotnet test\n```\n\n### Creating a new version for Nuget\n\nThe `Migratable/Migratable.csproj` file contains Nuget settings.\nWithin that file, update the version number then create the Nuget package.\nThe version exists twice, in `VersionPrefix` and `Version`.\n\nThe pack command that follows is done at the *solution* level not the project.\n\n``` sh\ncd \u003csolution\u003e\ndotnet build\ndotnet pack -c Release\n```\n\nThe `pack` command will mention issues with other projects in the solution; we are not packing them so it's fine to ignore the suggestions (eg a README for the example).\nWhen completed the *Successfully created package* line says where the `.nupkg` file is.\n\nIf you are changing the CLI that project contains its own version number.\nIt also has its own script for creating new cross-platform builds:\n\n``` sh\ncd \u003csolution\u003e\ncd Migratable.CLI\n./build.sh\n```\n\nOn Windows use `build.bat` rather than `build.sh`.\n\n### Forcing another project to get the latest from Nuget\n\nIt sometimes takes a while for a new version to be available after pushing.\nYou may be able to speed up the process:\n\n``` sh\ncd \u003cother-project\u003e\ndotnet restore --no-cache\n```\n\n---\n\n## MIT Licence\n\nCopyright (c) 2019-2024 K Cartlidge.\nSee the included [LICENCE file](./LICENCE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkcartlidge%2Fmigratable","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkcartlidge%2Fmigratable","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkcartlidge%2Fmigratable/lists"}