{"id":15003192,"url":"https://github.com/lukaszbudnik/migrator","last_synced_at":"2025-10-30T09:31:36.000Z","repository":{"id":4068443,"uuid":"51864287","full_name":"lukaszbudnik/migrator","owner":"lukaszbudnik","description":"Super fast and lightweight DB migration \u0026 evolution tool written in Go","archived":false,"fork":false,"pushed_at":"2025-01-26T07:13:36.000Z","size":4538,"stargazers_count":19,"open_issues_count":3,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-26T08:19:46.460Z","etag":null,"topics":["database","database-migrations","databases","db-evolution","db-migration","docker","evolution","go","golang","microsoft-sql-server","migration","multi-tenant-database","mysql","postgresql","schema","tenant"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lukaszbudnik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["lukaszbudnik"]}},"created_at":"2016-02-16T19:59:19.000Z","updated_at":"2025-01-26T07:13:39.000Z","dependencies_parsed_at":"2023-09-29T12:15:30.945Z","dependency_job_id":"d07b4626-9f3a-4bf5-8f1f-3036e53077da","html_url":"https://github.com/lukaszbudnik/migrator","commit_stats":{"total_commits":606,"total_committers":4,"mean_commits":151.5,"dds":0.4455445544554455,"last_synced_commit":"676e4fefa59f2d920dfac22d0f1241508f32628d"},"previous_names":[],"tags_count":27,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lukaszbudnik%2Fmigrator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lukaszbudnik%2Fmigrator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lukaszbudnik%2Fmigrator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lukaszbudnik%2Fmigrator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lukaszbudnik","download_url":"https://codeload.github.com/lukaszbudnik/migrator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238950485,"owners_count":19557533,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","database-migrations","databases","db-evolution","db-migration","docker","evolution","go","golang","microsoft-sql-server","migration","multi-tenant-database","mysql","postgresql","schema","tenant"],"created_at":"2024-09-24T18:56:54.661Z","updated_at":"2025-10-30T09:31:35.993Z","avatar_url":"https://github.com/lukaszbudnik.png","language":"Go","funding_links":["https://github.com/sponsors/lukaszbudnik"],"categories":[],"sub_categories":[],"readme":"# migrator ![Build](https://github.com/lukaszbudnik/migrator/workflows/Build/badge.svg) ![Docker](https://github.com/lukaszbudnik/migrator/workflows/Docker%20Image%20CI/badge.svg) [![Go Report Card](https://goreportcard.com/badge/github.com/lukaszbudnik/migrator)](https://goreportcard.com/report/github.com/lukaszbudnik/migrator) [![codecov](https://codecov.io/gh/lukaszbudnik/migrator/branch/main/graph/badge.svg)](https://codecov.io/gh/lukaszbudnik/migrator)\n\nSuper fast and lightweight DB migration tool written in Go. migrator outperforms other market-leading DB migration frameworks by a few orders of magnitude when comparing both execution time and memory consumption (see [PERFORMANCE.md](PERFORMANCE.md)).\n\n## ✨ Key Features\n\n- **🚀 Ultra Performance**: Orders of magnitude faster than other migration tools\n- **☁️ Multi-Cloud Storage**: Read migrations from local disk, AWS S3, or Azure Blob Storage\n- **🏢 Multi-Tenant Ready**: Built-in support for multi-schema, multi-tenant SaaS applications\n- **📡 GraphQL API**: Modern HTTP GraphQL service with comprehensive query capabilities\n- **📊 Observability**: Built-in Prometheus metrics and health checks\n- **🐳 Container Native**: Ultra-lightweight 30MB Docker image, perfect for microservices\n- **🔧 Legacy Migration**: Sync existing migrations from other frameworks seamlessly\n- **🎯 CI/CD Ready**: Easy integration into continuous deployment pipelines\n\n## 🗄️ Supported Databases\n\n- **PostgreSQL** 9.6+ (and flavours: Amazon RDS/Aurora, Google CloudSQL)\n- **MySQL** 5.7+ (and flavours: MariaDB, TiDB, Percona, Amazon RDS/Aurora, Google CloudSQL)\n- **Microsoft SQL Server** 2008+\n\n## 📦 Installation\n\nThe official docker image is available on:\n- Docker Hub: [lukasz/migrator](https://hub.docker.com/r/lukasz/migrator)\n- GitHub Container Registry: [ghcr.io/lukaszbudnik/migrator](https://github.com/lukaszbudnik/migrator/pkgs/container/migrator)\n\n```bash\ndocker pull lukasz/migrator:latest\n```\n\n## 🎯 Overview\n\nmigrator manages and versions all DB changes for you and completely eliminates manual and error-prone administrative tasks. migrator versions can be used for auditing and compliance purposes. migrator not only supports single schemas, but also comes with multi-schema support out of the box, making it an ideal DB migrations solution for multi-tenant SaaS products.\n\nmigrator runs as a HTTP GraphQL service and can be easily integrated into existing continuous integration and continuous delivery pipelines. migrator can also sync existing migrations from legacy frameworks making the technology switch even more straightforward.\n\nmigrator supports the following multi-tenant databases:\n\n- PostgreSQL and all its flavours\n- MySQL and all its flavours\n- Microsoft SQL Server\n\nmigrator supports reading DB migrations from:\n\n- local folder (any Docker/Kubernetes deployments)\n- AWS S3\n- Azure Blob Containers\n\nThe official docker image is available on:\n\n- docker hub at: [lukasz/migrator](https://hub.docker.com/r/lukasz/migrator)\n- alternative mirror at: [ghcr.io/lukaszbudnik/migrator](https://github.com/lukaszbudnik/migrator/pkgs/container/migrator)\n\nThe image is ultra lightweight and has a size of 30MB. Ideal for micro-services deployments!\n\n## 📋 Table of Contents\n\n- [🚀 Quick Start Guide](#-quick-start-guide)\n- [📡 API](#-api)\n- [⚙️ Configuration](#-configuration)\n- [📁 Source migrations](#-source-migrations)\n- [🗄️ Supported databases](#-supported-databases)\n- [🔧 Customisation and legacy frameworks support](#-customisation-and-legacy-frameworks-support)\n- [📊 Metrics](#-metrics)\n- [🏥 Health Checks](#-health-checks)\n- [📚 Tutorials](#-tutorials)\n- [⚡ Performance](#-performance)\n\n## 🚀 Quick Start Guide\n\nYou can apply your first migrations with migrator in literally a few seconds. There is a ready-to-use docker-compose file which sets up migrator and test databases.\n\n### 1. Get the migrator project\n\nGet the source code:\n\n```bash\ngit clone https://github.com/lukaszbudnik/migrator.git\ncd migrator\n```\n\n### 2. Start migrator and test DB containers\n\nStart migrator and setup test DB containers using docker-compose:\n\n```bash\ndocker-compose -f ./test/docker-compose.yaml up\n```\n\ndocker-compose will start and configure the following services:\n\n1. `migrator` - service using latest official migrator image, listening on port `8181`\n2. `migrator-dev` - service built from local branch, listening on port `8282`\n3. `postgres` - PostgreSQL service, listening on port `54325`\n4. `mysql` - MySQL service, listening on port `3306`\n5. `mssql` - MS SQL Server, listening on port `1433`\n\n### 3. Play around with migrator\n\nSet the port and create your first migration:\n\n```bash\nMIGRATOR_PORT=8181\nCOMMIT_SHA=\"your-version-here\"\n\n# Create new version\ncurl -s -d @- http://localhost:$MIGRATOR_PORT/v2/service \u003c\u003cEOF | jq\n{\n  \"query\": \"mutation CreateVersion(\\$input: VersionInput!) {\n    createVersion(input: \\$input) {\n      version { id, name }\n    }\n  }\",\n  \"variables\": {\n    \"input\": {\n      \"versionName\": \"$COMMIT_SHA\"\n    }\n  }\n}\nEOF\n\n# Fetch migrator versions\ncurl -s -d @- http://localhost:$MIGRATOR_PORT/v2/service \u003c\u003cEOF | jq -r \".data.versions\"\n{\n  \"query\": \"\n  query Versions {\n    versions {\n        id,\n        name,\n        created,\n      }\n  }\",\n  \"operationName\": \"Versions\"\n}\nEOF\n\n# Fetch tenants\ncurl -s -d @- http://localhost:$MIGRATOR_PORT/v2/service \u003c\u003cEOF | jq -r \".data.tenants\"\n{\n  \"query\": \"\n  query Tenants {\n    tenants {\n        name\n      }\n  }\",\n  \"operationName\": \"Tenants\"\n}\nEOF\n\n# Create new tenant\nTENANT_NAME=\"newcustomer$RANDOM\"\nVERSION_NAME=\"create-tenant-$TENANT_NAME\"\ncurl -s -d @- http://localhost:$MIGRATOR_PORT/v2/service \u003c\u003cEOF | jq -r '.data.createTenant'\n{\n  \"query\": \"\n  mutation CreateTenant(\\$input: TenantInput!) {\n    createTenant(input: \\$input) {\n      version {\n        id,\n        name,\n      }\n      summary {\n        startedAt\n        duration\n        tenants\n        migrationsGrandTotal\n        scriptsGrandTotal\n      }\n    }\n  }\",\n  \"operationName\": \"CreateTenant\",\n  \"variables\": {\n    \"input\": {\n      \"versionName\": \"$VERSION_NAME\",\n      \"tenantName\": \"$TENANT_NAME\"\n    }\n  }\n}\nEOF\n```\n\n\u003e **💡 Tip**: For a complete GraphQL schema and production deployment guides, see the [📡 API](#-api) and [📚 Tutorials](#-tutorials) sections below.\n\n## 📡 API\n\nTo return build information together with a list of supported API versions execute:\n\n```bash\nMIGRATOR_PORT=8181\ncurl http://localhost:${MIGRATOR_PORT}/\n```\n\nSample HTTP response:\n\n```json\n{\n  \"release\": \"refs/tags/v2023.0.1\",\n  \"sha\": \"43a6858b74832c185783969b3afbf7c2547a533a\",\n  \"apiVersions\": [\"v2\"]\n}\n```\n\n### /v2 - GraphQL API\n\nAPI v2 is a GraphQL API. API v2 was introduced in migrator v2020.1.0.\n\nAPI v2 introduced a formal concept of a DB version. Every migrator action creates a new DB version. Version logically groups all applied DB migrations for auditing and compliance purposes. You can browse versions together with executed DB migrations using the GraphQL API.\n\n### GET /v2/config\n\nReturns migrator's config as `application/yaml`.\n\nSample request:\n\n```bash\ncurl http://localhost:${MIGRATOR_PORT}/v2/config\n```\n\nSample HTTP response:\n\n```\nbaseLocation: test/migrations\ndriver: sqlserver\ndataSource: sqlserver://SA:YourStrongPassw0rd@127.0.0.1:32774/?database=migratortest\u0026connection+timeout=1\u0026dial+timeout=1\nsingleMigrations:\n- ref\n- config\ntenantMigrations:\n- tenants\npathPrefix: /\n```\n\n### GET /v2/schema\n\nReturns migrator's GraphQL schema as `plain/text`.\n\nAlthough migrator supports GraphQL introspection, it is much more convenient to get the schema in plain text.\n\nSample request:\n\n```bash\ncurl http://localhost:${MIGRATOR_PORT}/v2/schema\n```\n\nThe API v2 GraphQL schema and its description is as follows:\n\n```graphql\nschema {\n  query: Query\n  mutation: Mutation\n}\nenum MigrationType {\n  SingleMigration\n  TenantMigration\n  SingleScript\n  TenantScript\n}\nenum Action {\n  // Apply is the default action, migrator reads all source migrations and applies them\n  Apply\n  // Sync is an action where migrator reads all source migrations and marks them as applied in DB\n  // typical use cases are:\n  // importing source migrations from a legacy tool or synchronising tenant migrations when tenant was created using external tool\n  Sync\n}\nscalar Time\ninterface Migration {\n  name: String!\n  migrationType: MigrationType!\n  sourceDir: String!\n  file: String!\n  contents: String!\n  checkSum: String!\n}\ntype SourceMigration implements Migration {\n  name: String!\n  migrationType: MigrationType!\n  sourceDir: String!\n  file: String!\n  contents: String!\n  checkSum: String!\n}\ntype DBMigration implements Migration {\n  id: Int!\n  name: String!\n  migrationType: MigrationType!\n  sourceDir: String!\n  file: String!\n  contents: String!\n  checkSum: String!\n  schema: String!\n  created: Time!\n}\ntype Tenant {\n  name: String!\n}\ntype Version {\n  id: Int!\n  name: String!\n  created: Time!\n  dbMigrations: [DBMigration!]!\n}\ninput SourceMigrationFilters {\n  name: String\n  sourceDir: String\n  file: String\n  migrationType: MigrationType\n}\ninput VersionInput {\n  versionName: String!\n  action: Action = Apply\n  dryRun: Boolean = false\n}\ninput TenantInput {\n  tenantName: String!\n  versionName: String!\n  action: Action = Apply\n  dryRun: Boolean = false\n}\ntype Summary {\n  // date time operation started\n  startedAt: Time!\n  // how long the operation took in seconds\n  duration: Float!\n  // number of tenants in the system\n  tenants: Int!\n  // number of loaded and applied single schema migrations\n  singleMigrations: Int!\n  // number of loaded multi-tenant schema migrations\n  tenantMigrations: Int!\n  // number of applied multi-tenant schema migrations (equals to tenants * tenantMigrations)\n  tenantMigrationsTotal: Int!\n  // sum of singleMigrations and tenantMigrationsTotal\n  migrationsGrandTotal: Int!\n  // number of loaded and applied single schema scripts\n  singleScripts: Int!\n  // number of loaded multi-tenant schema scripts\n  tenantScripts: Int!\n  // number of applied multi-tenant schema migrations (equals to tenants * tenantScripts)\n  tenantScriptsTotal: Int!\n  // sum of singleScripts and tenantScriptsTotal\n  scriptsGrandTotal: Int!\n}\ntype CreateResults {\n  summary: Summary!\n  version: Version\n}\ntype Query {\n  // returns array of SourceMigration objects\n  // all parameters are optional and can be used to filter source migrations\n  // note that if the input query includes \"contents\" field this operation can produce large amounts of data\n  // if you want to return \"contents\" field it may be better to get individual source migrations using sourceMigration(file: String!)\n  sourceMigrations(filters: SourceMigrationFilters): [SourceMigration!]!\n  // returns a single SourceMigration\n  // this operation can be used to fetch a complete SourceMigration including \"contents\" field\n  // file is the unique identifier for a source migration file which you can get from sourceMigrations()\n  sourceMigration(file: String!): SourceMigration\n  // returns array of Version objects\n  // file is optional and can be used to return versions in which given source migration file was applied\n  // note that if input query includes DBMigration array and \"contents\" field this operation can produce large amounts of data\n  // if you want to return \"contents\" field it may be better to get individual versions using either\n  // version(id: Int!) or even get individual DB migration using dbMigration(id: Int!)\n  versions(file: String): [Version!]!\n  // returns a single Version\n  // id is the unique identifier of a version which you can get from versions()\n  // note that if input query includes \"contents\" field this operation can produce large amounts of data\n  // if you want to return \"contents\" field it may be better to get individual DB migration using dbMigration(id: Int!)\n  version(id: Int!): Version\n  // returns a single DBMigration\n  // this operation can be used to fetch a complete DBMigration including \"contents\" field\n  // id is the unique identifier of a DB migration which you can get from versions(file: String) or version(id: Int!)\n  dbMigration(id: Int!): DBMigration\n  // returns array of Tenant objects\n  tenants(): [Tenant!]!\n}\ntype Mutation {\n  // creates new DB version by applying all eligible DB migrations \u0026 scripts\n  createVersion(input: VersionInput!): CreateResults!\n  // creates new tenant by applying only tenant-specific DB migrations \u0026 scripts, also creates new DB version\n  createTenant(input: TenantInput!): CreateResults!\n}\n```\n\n### POST /v2/service\n\nThis is a GraphQL endpoint which handles both query and mutation requests.\n\nSee [Quick Start Guide](#-quick-start-guide) for a few curl examples to get you started.\n\nThe preferred way of consuming migrator's GraphQL endpoint is to use GraphQL clients. These clients can be generated from the GraphQL schema in any programming language you use (Java, Python, C#, JavaScript, Go, etc.).\n\n### /v1 - REST API\n\nAPI v1 was sunset in v2021.0.0.\n\nThe documentation is available in a separate document [API v1](APIv1.md).\n\n### Request tracing\n\nmigrator uses request tracing via `X-Request-ID` header. This header can be used with all requests for tracing and/or auditing purposes. If this header is absent migrator will generate one for you.\n\n## ⚙️ Configuration\n\nLet's see how to configure migrator.\n\n### migrator.yaml\n\nmigrator configuration file is a simple YAML file. Take a look at a sample `migrator.yaml` configuration file which contains the description, correct syntax, and sample values for all available properties.\n\n```yaml\n# required, location where all migrations are stored, see singleSchemas and tenantSchemas below\nbaseLocation: test/migrations\n# required, SQL go driver implementation used, see section \"Supported databases\"\ndriver: postgres\n# required, dataSource format is specific to SQL go driver implementation used, see section \"Supported databases\"\ndataSource: \"user=postgres dbname=migrator_test host=192.168.99.100 port=55432 sslmode=disable\"\n# optional, override only if you have a specific way of determining tenants, default is:\ntenantSelectSQL: \"select name from migrator.migrator_tenants\"\n# optional, override only if you have a specific way of creating tenants, default is:\ntenantInsertSQL: \"insert into migrator.migrator_tenants (name) values ($1)\"\n# optional, override only if you have a specific schema placeholder, default is:\nschemaPlaceHolder: { schema }\n# required, directories of single schema SQL migrations, these are subdirectories of baseLocation\nsingleMigrations:\n  - public\n  - ref\n  - config\n# optional, directories of tenant schemas SQL migrations, these are subdirectories of baseLocation\ntenantMigrations:\n  - tenants\n# optional, directories of single SQL scripts which are applied always, these are subdirectories of baseLocation\nsingleScripts:\n  - config-scripts\n# optional, directories of tenant SQL scripts which are applied always for all tenants, these are subdirectories of baseLocation\ntenantScripts:\n  - tenants-scripts\n# optional, default is 8080\nport: 8080\n# path prefix is optional and defaults to '/'\n# path prefix is used for application HTTP request routing by Application Load Balancers/Application Gateways\n# for example when deploying to AWS ECS and using AWS ALB the path prefix could be set as below\n# then all HTTP requests should be prefixed with that path, for example: /migrator/v1/config, /migrator/v1/migrations/source, etc.\npathPrefix: /migrator\n# the webhook configuration section is optional\n# the default Content-Type header is application/json but can be overridden via webHookHeaders below\nwebHookURL: https://your.server.com/services/TTT/BBB/XXX\n# if the webhook expects a payload in a specific format there is an option to provide a payload template\n# see webhook template for more information\nwebHookTemplate: '{\"text\": \"New version: ${summary.versionId} started at: ${summary.startedAt} and took ${summary.duration}. Full results are: ${summary}\"}'\n# should you need more control over HTTP headers use below\nwebHookHeaders:\n  - \"Authorization: Basic QWxhZGRpbjpPcGVuU2VzYW1l\"\n  - \"Content-Type: application/json\"\n  - \"X-Custom-Header: value1,value2\"\n# optional, allows to filter logs produced by migrator, valid values are: DEBUG, INFO, ERROR, PANIC\n# defaults to INFO\nlogLevel: INFO\n```\n\n### Env variables substitution\n\nmigrator supports env variables substitution in config file. All patterns matching `${NAME}` will look for env variable `NAME`. Below are some common use cases:\n\n```yaml\ndataSource: \"user=${DB_USER} password=${DB_PASSWORD} dbname=${DB_NAME} host=${DB_HOST} port=${DB_PORT}\"\nwebHookHeaders:\n  - \"X-Security-Token: ${SECURITY_TOKEN}\"\n```\n\n### WebHook template\n\nBy default when a webhook is configured migrator will post a JSON representation of `Summary` struct to its endpoint.\n\nIf your webhook expects a payload in a specific format (say Slack or MS Teams incoming webhooks) there is an option to configure a `webHookTemplate` property in migrator's configuration file. The template can have the following placeholders:\n\n- `${summary}` - will be replaced by a JSON representation of `Summary` struct, all double quotes will be escaped so that the template remains a valid JSON document\n- `${summary.field}` - will be replaced by a given field of `Summary` struct\n\nPlaceholders can be mixed:\n\n```yaml\nwebHookTemplate: '{\"text\": \"New version created: ${summary.versionId} started at: ${summary.startedAt} and took ${summary.duration}. Migrations/scripts total: ${summary.migrationsGrandTotal}/${summary.scriptsGrandTotal}. Full results are: ${summary}\"}'\n```\n\n## 📁 Source migrations\n\nMigrations can be read from local disk, AWS S3, Azure Blob Containers. I'm open to contributions to add more cloud storage options.\n\n### Local storage\n\nIf `baseLocation` property is a path (either relative or absolute) local storage implementation is used:\n\n```\n# relative path\nbaseLocation: test/migrations\n# absolute path\nbaseLocation: /project/migrations\n```\n\n### AWS S3\n\nIf `baseLocation` starts with `s3://` prefix, AWS S3 implementation is used. In such case the `baseLocation` property is treated as a bucket name followed by optional prefix:\n\n```\n# S3 bucket\nbaseLocation: s3://your-bucket-migrator\n# S3 bucket with optional prefix\nbaseLocation: s3://your-bucket-migrator/appcodename/prod/artefacts\n```\n\nmigrator uses official AWS SDK for Go and uses a well known [default credential provider chain](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html).\n\n### Azure Blob Containers\n\nIf `baseLocation` matches `^https://.*\\.blob\\.core\\.windows\\.net/.*` regex, Azure Blob implementation is used. In such case the `baseLocation` property is treated as a container URL. The URL can have optional prefix too:\n\n```\n# Azure Blob container URL\nbaseLocation: https://storageaccountname.blob.core.windows.net/mycontainer\n# Azure Blob container URL with optional prefix\nbaseLocation: https://storageaccountname.blob.core.windows.net/mycontainer/appcodename/prod/artefacts\n```\n\nmigrator uses official Azure SDK for Go and supports authentication using Storage Account Key (via `AZURE_STORAGE_ACCOUNT` and `AZURE_STORAGE_ACCESS_KEY` env variables) as well as much more flexible (and recommended) Azure Active Directory Managed Identity.\n\n## 🗄️ Supported databases\n\nCurrently migrator supports the following databases including their flavours (like Percona, MariaDB for MySQL, etc.). Please review the Go driver implementation for information about all supported features and how `dataSource` configuration property should look like.\n\n### PostgreSQL 9.6+\n\nSchema-based multi-tenant database, with transactions spanning DDL statements, driver used: https://github.com/lib/pq.\n\nThe following versions and flavours are supported by the driver:\n\n- PostgreSQL\n- Amazon RDS PostgreSQL - PostgreSQL-compatible relational database built for the cloud\n- Amazon Aurora PostgreSQL - PostgreSQL-compatible relational database built for the cloud\n- Google CloudSQL PostgreSQL - PostgreSQL-compatible relational database built for the cloud\n\n### MySQL 5.7+\n\nDatabase-based multi-tenant database, transactions do not span DDL statements, driver used: https://github.com/go-sql-driver/mysql.\n\nThe following versions and flavours are supported by the driver:\n\n- MySQL\n- MariaDB - enhanced near linearly scalable multi-master MySQL\n- TiDB - an open-source, cloud-native, distributed SQL database designed for high availability, horizontal and vertical scalability, strong consistency, and high performance.\n- Percona - an enhanced drop-in replacement for MySQL\n- Amazon RDS MySQL - MySQL-compatible relational database built for the cloud\n- Amazon Aurora MySQL - MySQL-compatible relational database built for the cloud\n- Google CloudSQL MySQL - MySQL-compatible relational database built for the cloud\n\n### Microsoft SQL Server 2008+\n\nA relational database management system developed by Microsoft, driver used: https://github.com/microsoft/go-mssqldb.\n\nThe Go driver supports all Microsoft SQL Server versions starting with 2008.\n\n## 🔧 Customisation and legacy frameworks support\n\nmigrator can be used with an already existing legacy DB migration framework.\n\n### Custom tenants support\n\nIf you have an existing way of storing information about your tenants you can configure migrator to use it.\nIn the config file you need to provide 2 configuration properties:\n\n- `tenantSelectSQL` - a select statement which returns names of the tenants\n- `tenantInsertSQL` - an insert statement which creates a new tenant entry, the insert statement should be a valid prepared statement for the SQL driver/database you use, it must accept the name of the new tenant as a parameter; finally should your table require additional columns you need to provide default values for them\n\nHere is an example:\n\n```yaml\ntenantSelectSQL: select name from global.customers\ntenantInsertSQL: insert into global.customers (name, active, date_added) values (?, true, NOW())\n```\n\n### Custom schema placeholder\n\nSQL migrations and scripts can use `{schema}` placeholder which will be automatically replaced by migrator with a current schema. For example:\n\n```sql\ncreate schema if not exists {schema};\ncreate table if not exists {schema}.modules ( k int, v text );\ninsert into {schema}.modules values ( 123, '123' );\n```\n\nIf you have an existing DB migrations legacy framework which uses different schema placeholder you can override the default one.\nIn the config file you need to provide `schemaPlaceHolder` configuration property:\n\nFor example:\n\n```yaml\nschemaPlaceHolder: :tenant\n```\n\n### Synchronising legacy migrations to migrator\n\nBefore switching from a legacy tool you need to synchronise source migrations to migrator. migrator has no knowledge of migrations applied by other tools and as such will attempt to apply all found source migrations.\n\nSynchronising will load all source migrations and mark them as applied. This can be done by `CreateVersion` operation with action set to `Sync`.\n\nOnce the initial synchronisation is done you can use migrator for all the consecutive DB migrations.\n\n### Final comments\n\nWhen using migrator please remember that:\n\n- migrator creates `migrator` schema together with `migrator_versions` and `migrator_migrations` tables automatically\n- if you're not using [Custom tenants support](#custom-tenants-support) migrator creates `migrator_tenants` table automatically\n- when adding a new tenant migrator creates a new DB schema and applies all tenant migrations and scripts\n- single schemas are not created automatically, you must add an initial migration with `create schema {schema}` SQL statement (see sample migrations in `test` folder)\n\n## 📊 Metrics\n\nmigrator exposes Prometheus metrics at `/metrics` endpoint. Apart from migrator-specific metrics, it exposes a lot of OS process and Go metrics.\n\nThe following metrics are available:\n\n- `go_gc_*` - Go garbage collection\n- `go_memstats_*` - Go memory\n- `process_*` - OS process\n- `migrator_gin_request_*` - Gin request metrics\n- `migrator_gin_response_*` - Gin response metrics\n- `migrator_gin_tenants_created` - migrator tenants created\n- `migrator_gin_versions_created` - migrator versions created\n- `migrator_gin_migrations_applied{type=\"single_migrations\"}` - migrator single migrations applied\n- `migrator_gin_migrations_applied{type=\"single_scripts\"}` - migrator single scripts applied\n- `migrator_gin_migrations_applied{type=\"tenant_migrations_total\"}` - migrator total tenant migrations applied (for all tenants)\n- `migrator_gin_migrations_applied{type=\"tenant_scripts_total\"}` - migrator total tenant scripts applied (for all tenants)\n\n## 🏥 Health Checks\n\nHealth checks are available at `/health` endpoint. migrator implements [Eclipse MicroProfile Health 3.0 RC4](https://download.eclipse.org/microprofile/microprofile-health-3.0-RC4/microprofile-health-spec.html) spec.\n\nA successful response returns HTTP 200 OK code:\n\n```json\n{\n  \"status\": \"UP\",\n  \"checks\": [\n    {\n      \"name\": \"DB\",\n      \"status\": \"UP\"\n    },\n    {\n      \"name\": \"Loader\",\n      \"status\": \"UP\"\n    }\n  ]\n}\n```\n\nIn case one of the checks has DOWN status then the overall status is DOWN. Failed check has `data` field which provides more information on why its status is DOWN. Health check will also return HTTP 503 Service Unavailable code:\n\n```json\n{\n  \"status\": \"DOWN\",\n  \"checks\": [\n    {\n      \"name\": \"DB\",\n      \"status\": \"DOWN\",\n      \"data\": {\n        \"details\": \"failed to connect to database: dial tcp 127.0.0.1:5432: connect: connection refused\"\n      }\n    },\n    {\n      \"name\": \"Loader\",\n      \"status\": \"DOWN\",\n      \"data\": {\n        \"details\": \"open /nosuchdir/migrations: no such file or directory\"\n      }\n    }\n  ]\n}\n```\n\n## 📚 Tutorials\n\nIn this section I provide links to more in-depth migrator tutorials.\n\n### Deploying migrator to AWS ECS\n\nThe goal of this tutorial is to deploy migrator to AWS ECS, load migrations from AWS S3 and apply them to AWS RDS DB while storing env variables securely in AWS Secrets Manager. The list of all AWS services used is: IAM, ECS, ECR, Secrets Manager, RDS, and S3.\n\nYou can find it in [tutorials/aws-ecs](tutorials/aws-ecs).\n\n### Deploying migrator to AWS EKS\n\nThe goal of this tutorial is to deploy migrator to AWS EKS, load migrations from AWS S3 and apply them to AWS RDS DB. The list of AWS services used is: IAM, EKS, ECR, RDS, and S3.\n\nYou can find it in [tutorials/aws-eks](tutorials/aws-eks).\n\n### Deploying migrator to Azure AKS\n\nThe goal of this tutorial is to publish migrator image to Azure ACR private container repository, deploy migrator to Azure AKS, load migrations from Azure Blob Container and apply them to Azure Database for PostgreSQL. The list of Azure services used is: AKS, ACR, Blob Storage, and Azure Database for PostgreSQL.\n\nYou can find it in [tutorials/azure-aks](tutorials/azure-aks).\n\n### Securing migrator with OAuth2\n\nThe goal of this tutorial is to secure migrator with OAuth2. It shows how to deploy oauth2-proxy in front of migrator which will off-load and transparently handle authorization for migrator end-users.\n\nYou can find it in [tutorials/oauth2-proxy](tutorials/oauth2-proxy).\n\n### Securing migrator with OIDC\n\nThe goal of this tutorial is to secure migrator with OAuth2 and OIDC. It shows how to deploy oauth2-proxy and haproxy in front of migrator which will off-load and transparently handle both authorization (oauth2-proxy) and authentication (haproxy with custom lua script) for migrator end-users.\n\nYou can find it in [tutorials/oauth2-proxy-oidc-haproxy](tutorials/oauth2-proxy-oidc-haproxy).\n\n## ⚡ Performance\n\nPerformance benchmarks were moved to a dedicated [PERFORMANCE.md](PERFORMANCE.md) document.\n\n## Change log\n\nPlease navigate to [migrator/releases](https://github.com/lukaszbudnik/migrator/releases) for a complete list of versions, features, and change log.\n\n## Contributing\n\nContributions are most welcomed!\n\nFor contributing, code style, running unit \u0026 integration tests please see [CONTRIBUTING.md](CONTRIBUTING.md).\n\n## License\n\nApache 2.0 License - see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flukaszbudnik%2Fmigrator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flukaszbudnik%2Fmigrator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flukaszbudnik%2Fmigrator/lists"}