{"id":24873724,"url":"https://github.com/rolang/dumbo","last_synced_at":"2025-10-15T17:31:06.760Z","repository":{"id":183195912,"uuid":"669683172","full_name":"rolang/dumbo","owner":"rolang","description":"Simple database migration tool for Postgres with skunk on JVM and Native","archived":false,"fork":false,"pushed_at":"2025-01-31T07:16:07.000Z","size":330,"stargazers_count":29,"open_issues_count":2,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-31T07:31:37.907Z","etag":null,"topics":["database-migration","flyway","jvm","native","postgres","scala","scala-native","skunk"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rolang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-23T05:00:09.000Z","updated_at":"2025-01-31T07:16:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"61a5612a-e24b-46fa-a4ff-65cbbc09b32e","html_url":"https://github.com/rolang/dumbo","commit_stats":null,"previous_names":["rolang/dumbo"],"tags_count":22,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rolang%2Fdumbo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rolang%2Fdumbo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rolang%2Fdumbo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rolang%2Fdumbo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rolang","download_url":"https://codeload.github.com/rolang/dumbo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236624527,"owners_count":19178982,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database-migration","flyway","jvm","native","postgres","scala","scala-native","skunk"],"created_at":"2025-02-01T06:16:51.332Z","updated_at":"2025-10-15T17:31:06.742Z","avatar_url":"https://github.com/rolang.png","language":"Scala","readme":"# Dumbo\n\n![Maven Central Version](https://img.shields.io/maven-central/v/dev.rolang/dumbo_3?label=maven-central)\n![Maven Snapshot Version](https://img.shields.io/maven-metadata/v?metadataUrl=https:%2F%2Fcentral.sonatype.com%2Frepository%2Fmaven-snapshots%2Fdev%2Frolang%2Fdumbo_3%2Fmaven-metadata.xml\u0026label=maven-snapshots)\n[![dumbo Scala version support](https://index.scala-lang.org/rolang/dumbo/dumbo/latest-by-scala-version.svg?platform=native0.4)](https://index.scala-lang.org/rolang/dumbo/dumbo)\n[![dumbo Scala version support](https://index.scala-lang.org/rolang/dumbo/dumbo/latest-by-scala-version.svg?platform=jvm)](https://index.scala-lang.org/rolang/dumbo/dumbo)\n\n![Logo](./docs/assets/logo.png)\n\nSimple database migration tool for [Postgres](https://www.postgresql.org) with [skunk](https://typelevel.org/skunk/).  \nUsable via command-line or as library in your Scala project targeting JVM or Native (see [usage example](#usage-example)).  \nSupports a subset of [Flyway](https://flywaydb.org) features and keeps a Flyway compatible history state to allow you to switch to Flyway if necessary.  \nYou might also be able to simply switch from Flyway to Dumbo without any changes in migration files or the history state, depending on used Flyway features.\n\n## Currently supports:\n\n### Versioned Migrations\n\nAs described in _[Flyway Versioned Migrations docs](https://documentation.red-gate.com/flyway/flyway-concepts/migrations/versioned-migrations)_:\n\nThe most common type of migration is a versioned migration. Each versioned migration has a version, a description and a checksum. The version must be unique. The description is purely informative for you to be able to remember what each migration does. The checksum is there to detect accidental changes. Versioned migrations are applied in order exactly once.\n\nEach versioned migration must be assigned a unique version.  \n A simple increasing integer or any version is valid as long as it conforms to the usual dotted notation:\n\n- 1\n- 001\n- 5.2\n- 1.2.3.4.5.6.7.8.9\n- 205.68\n- 20130115113556\n- 2013.1.15.11.35.56\n- 2013.01.15.11.35.56\n\n![Versioned Migrations](./docs/assets/versioned_migrations.png)\n\n### Repeatable Migrations\n\nAs described in _[Flyway Repeatable Migrations docs](https://documentation.red-gate.com/flyway/flyway-concepts/migrations/repeatable-migrations)_:\n\nRepeatable migrations have a description and a checksum, but no version. Instead of being run just once, they are (re-)applied every time their checksum changes.\n\n![Repeatable Migrations](./docs/assets/repeatable_migrations.png)\n\nThis is very useful for managing database objects whose definition can then simply be maintained in a single file in version control. They are typically used for\n \n(Re-)creating views/procedures/functions/packages/...\nBulk reference data reinserts\nWithin a single migration run, repeatable migrations are always applied last, after all pending versioned migrations have been executed. \nRepeatable migrations are applied in the order of their description.\n \nIt is your responsibility to ensure the same repeatable migration can be applied multiple times. This usually involves making use of CREATE OR REPLACE clauses in your DDL statements.\n\n\n### Script Config Files\n\nSimilar to [Flyway script config files](https://documentation.red-gate.com/flyway/reference/script-configuration) it's possible to configure migrations on a per-script basis.\n\nThis is achieved by creating a script configuration file in the same folder as the migration.  \nThe script configuration file name must match the migration file name, with the `.conf` suffix added.\n\nFor example a migration file `db/V1__my_script.sql` would have a script configuration file `db/V1__my_script.sql.conf`.\n\n#### Structure\n\nScript config files have the following structure:\n\n```\nkey=value\n```\n\n#### Reference\n\n- **executeInTransaction**  \n  Manually determine whether or not to execute this migration in a transaction.\n\n  This is useful where certain statements can only execute outside a transaction (like `CREATE INDEX CONCURRENTLY` etc.)  \n  Example:\n\n  ```\n  executeInTransaction=false\n  ```\n\n  ⚠️⚠️  \n  **NOTE**: Dumbo will attempt to execute each migration as a [simple query with multiple statements](https://www.postgresql.org/docs/current/protocol-flow.html#PROTOCOL-FLOW-MULTI-STATEMENT) \n  in a transaction by default (unlike Flyway which may decide not to do so).  \n  To disable it you need to set it explicitly using the configuration above.  \n\n  Use with care and try to avoid where possible. Partially applied migrations will require manual intervention.  \n  Dumbo is not going to update the history state in case of partial failures.  \n  If you re-run the migration process it will attempt to execute the script the same way it did before it failed.  \n  To fix it you'd need to roll back applied updates manually and then update the migration script and/or split it into multiple files before re-running the migration process.  \n  ⚠️⚠️  \n\n## Usage example\n\nFor usage via command line see [command-line](#command-line) section.\n\nIn a sbt project dumbo can be added like:\n\n```scala\nlibraryDependencies += \"dev.rolang\" %% \"dumbo\" % \"0.6.x\"\n```\n\n_For compatibility with skunk `0.6.x` / natchez / Scala 2.12.x use release series `0.0.x`_:\n\n```scala\nlibraryDependencies += \"dev.rolang\" %% \"dumbo\" % \"0.0.x\"\n```\n\nTo include snapshot releases, add snapshot resolver:\n\n```scala\nresolvers += \"Sonatype Central Snapshots\" at \"https://central.sonatype.com/repository/maven-snapshots\"\n```\n\nExamples can be viewed in [modules/example](./modules/example/).  \nSimilar to usage of the Flyway Java library, given versioned migrations in the resources folder:\n\n```\nexample\n  src\n    main\n      resources\n        db\n          migration\n            R__test_view.sql\n            V1__test.sql\n            V3__test_c.sql\n            V2__test_b.sql\n```\n\nThe migration can be executed like:\n\n```scala\n//\u003e using scala 3.7.1\n//\u003e using resourceDir ../resources\n//\u003e using dep \"dev.rolang::dumbo::0.6.0\"\n\nimport cats.effect.{IO, IOApp}\nimport dumbo.logging.Implicits.console\nimport dumbo.{ConnectionConfig, Dumbo}\nimport org.typelevel.otel4s.trace.Tracer.Implicits.noop\n\nobject ExampleApp extends IOApp.Simple:\n  def run = Dumbo\n    .withResourcesIn[IO](\"db/migration\")\n    .apply(\n      connection = ConnectionConfig(\n        host = \"localhost\",\n        port = 5432,\n        user = \"root\",\n        database = \"postgres\",\n        password = None,\n        ssl = ConnectionConfig.SSL.None,\n      )\n    )\n    .runMigration\n    .flatMap: result =\u003e\n      IO.println(s\"Migration completed with ${result.migrationsExecuted} migrations\")\n```\n\nTo run the example, start a Postgres server via docker:\n\n```shell\ndocker compose up pg_latest_1\n```\n\nExecute the example via [scala-cli](https://scala-cli.virtuslab.org):\n```shell\nscala-cli modules/example/src/main/scala/ExampleApp.scala\n```\n\nor via [sbt](https://www.scala-sbt.org):\n\n```shell\nsbt 'example/runMain ExampleApp'\n```\n\n## Configurations\n\n### Configure the resources\n\nTo read migration scripts from embedded resources:\n\n```scala\nval dumboWithResouces = Dumbo.withResourcesIn[IO](\"db/migration\")\n```\n\nNotes:\n\n- In Scala 3 the resource files will be listed / checked at compile time.\n  In case the resource location can't be found in the classpath or multiple locations were found, a compilation error will appear.\n- For Scala Native ensure to have [embedded resources](https://scala-native.org/en/stable/lib/javalib.html?highlight=resources#embedding-resources) enabled.\n- In Scala 2 the resource location will be checked at runtime\n- In Scala 2 + Native you'll be required to pass a list of resources as we can't list resources from a location at runtime, e.g. like:\n\n```scala\nval dumboWithResouces = Dumbo.withResources(\n  List(\n    ResourceFilePath(\"/db/migration/V1__test.sql\"),\n    ResourceFilePath(\"/db/migration/V2__test_b.sql\"))\n  )\n```\n\nTo read migration scripts from the files system use:\n\n```scala\nval dumboWithResouces = Dumbo.withFilesIn[IO](\n  fs2.io.file.Path(\"modules/example/src/main/resources/db/migration\")\n)\n```\n\n### Apply further configuration:\n\n```scala\ndumboWithResouces.apply(\n  // connection config\n  connection: dumbo.ConnectionConfig = dumbo.ConnectionConfig(\n    host = \"localhost\",\n    port = 5432,\n    user = \"postgres\",\n    database = \"postgres\",\n    password = Some(\"postgres\"),\n    ssl = dumbo.ConnectionConfig.SSL.None, // SSL config, default is dumbo.ConnectionConfig.SSL.None\n  ),\n\n  // default schema (the history state is going to be stored under that schema)\n  defaultSchema: String = \"public\",\n\n  // schemas to include in the search\n  schemas: Set[String] = Set.empty[String],\n\n  // migration history table name\n  schemaHistoryTable: String = \"flyway_schema_history\",\n\n  // compare migration files with applied migrations\n  // check e.g. for changed file content/description or missing files before migration\n  validateOnMigrate: Boolean = true\n)\n\n// migration progress logs can be added optionally in case you'd like dumbo to provide some feedback on longer running queries\n// it will perform requests to Postgres in given interval to check for queries that are causing the lock on migration history table\ndumboWithResouces.withMigrationStateLogAfter[IO](5.seconds)(\n  /* use config as above */\n)\n\n// in some cases you may want to provide a custom skunk session instead of connection config\n// NOTE: given a connection config, dumbo will construct a session that will include given schemas into the search path via session parameters\n// for custom sessions the search_path will be updated via the SET command if it doesn't match given schemas config which is not recommended (see https://typelevel.org/skunk/reference/Sessions.html#session-parameters)\n// you may consider adding the search_path to the parameters yourself in that case, dumbo will log it as a warning\ndumboWithResouces.withSession(\n  defaultSchema = \"schema_1\", // consider updating the search_path for non default schema settings\n  sessionResource = skunk.Session.single[IO](\n    host = \"localhost\",\n    port = 5432,\n    user = \"postgres\",\n    database = \"postgres\",\n    password = Some(\"postgres\"),\n    // add schemas to the search path\n    // those are added by default when using a dumbo.ConnectionConfig\n    parameters = skunk.Session.DefaultConnectionParameters ++ Map(\n      \"search_path\" -\u003e \"schema_1\"\n    ),\n    // a strategy other than BuiltinsOnly should not be required for running migrations\n    // you may want to keep that\n    strategy = skunk.util.Typer.Strategy.BuiltinsOnly,\n  )\n)\n```\n\n### Logging\nDumbo requires a `dumbo.logging.Logger` instance.  \nTo use a logger based on `cats.effect.std.Console` that is required by `skunk` add the import:\n```scala\nimport dumbo.logging.Implicits.console\n```\n\nOr provide a custom logger like:\n```scala\ngiven dumbo.logging.Logger[F] =\n  case (dumbo.logging.LogLevel.Info, message) =\u003e ???\n  case (dumbo.logging.LogLevel.Warn, message) =\u003e ???\n```\n\nFor an example based on [log4cats](https://github.com/typelevel/log4cats) with [Slf4J](https://slf4j.org/) see [ExampleLog4Cats.scala](modules/example/src/main/scala/ExampleLog4Cats.scala).\n\n## Command-line\n\nDumbo ships with a command line tool as a native binary.\n\n### Download and installation\n\nNote: the executable depends on [utf8proc](https://github.com/JuliaStrings/utf8proc) and [s2n-tls](https://github.com/aws/s2n-tls).\n\n#### Linux\n\n##### Arch\n\n```shell\n# install prerequisites\nsudo pacman -S s2n-tls libutf8proc\n\n# download and run dumbo\ncurl -L https://github.com/rolang/dumbo/releases/latest/download/dumbo-cli-x86_64-linux \u003e dumbo \u0026\u0026 chmod +x dumbo\n./dumbo -v\n```\n\n##### Alpine\n\n```shell\n# install prerequisites\napk update \u0026\u0026 apk add gcompat libgcc libstdc++ s2n-tls utf8proc\n\n# download and run dumbo\nwget https://github.com/rolang/dumbo/releases/latest/download/dumbo-cli-x86_64-linux -O dumbo \u0026\u0026 chmod +x dumbo\n./dumbo -v\n```\n\n##### Ubuntu / Debian\n\nAs of now `s2n-tls` is not available as apt package.\nFor utf8proc the package `libutf8proc3` is required which is currently only available from Ubuntu `24.04` / `noble` via apt.  \nAlternatively one can install the depenedncies via [homebrew](https://brew.sh) as follows:\n\n```shell\n# install homebrew if not already installed https://docs.brew.sh/Homebrew-on-Linux\nsudo apt-get install build-essential procps curl file git\n/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"\n\n# install prerequisites and include homebrew lib\n/home/linuxbrew/.linuxbrew/bin/brew install s2n utf8proc\nexport LD_LIBRARY_PATH=\"/home/linuxbrew/.linuxbrew/lib\"\n\n# download and run dumbo\ncurl -L https://github.com/rolang/dumbo/releases/latest/download/dumbo-cli-x86_64-linux \u003e dumbo \u0026\u0026 chmod +x dumbo\n./dumbo -v\n```\n\n#### macOS\n```shell\n# install prerequisites\nbrew install s2n utf8proc\n# download and run dumbo\n# on Apple silicon / ARM\ncurl -L https://github.com/rolang/dumbo/releases/latest/download/dumbo-cli-aarch64-macosx \u003e dumbo \u0026\u0026 chmod +x dumbo\n# on Intel / x86-64 use this instead:\n# curl -L https://github.com/rolang/dumbo/releases/latest/download/dumbo-cli-x86_64-macosx \u003e dumbo \u0026\u0026 chmod +x dumbo\n./dumbo -v\n```\n\n#### Docker\n\nA docker image with the command line is published to docker hub: [rolang/dumbo](https://hub.docker.com/r/rolang/dumbo).\n\nTo print the command line help run:\n\n```shell\ndocker run rolang/dumbo:latest-alpine help\n```\n\nTo run the example migrations in this repository, run from repository's root directory:\n\n1. Boot up a Postgres instance\n\n```shell\ndocker compose up pg_latest_1\n```\n\n2. Run example migration\n\n```shell\ndocker run --net=\"host\" \\\n  -v ./modules/example/src/main/resources/db/migration:/migration \\\n  rolang/dumbo:latest-alpine \\\n  -user=root \\\n  -url=postgresql://localhost:5432/postgres \\\n  -location=/migration \\\n  migrate\n```\n\n### Command-line usage\n\n```\ndumbo [options] [command]\n```\n\n##### Commands:\n\n| Command                | Description                                                       |\n| ---------------------- | ----------------------------------------------------------------- |\n| help                   | Print this usage info and exit                                    |\n| migrate                | Migrates the database                                             |\n| validate               | Validates the applied migrations against the ones in the location |\n| version, -v, --version | Print the Dumbo version                                           |\n\n##### Configuration parameters (Format: -key=value):\n\n| Configuration     | Description                                                                                               | Default                 |\n| ----------------- | --------------------------------------------------------------------------------------------------------- | ----------------------- |\n| location          | Path to directory to scan for migrations                                                                  |                         |\n| table             | The name of Dumbo's schema history table                                                                  | `flyway_schema_history` |\n| password          | Password to use to connect to the database                                                                |                         |\n| url               | Url to use to connect to the database                                                                     |                         |\n| validateOnMigrate | Validate when running migrate                                                                             | `true`                  |\n| user              | User to use to connect to the database                                                                    |                         |\n| schemas           | Comma-separated list of the schemas managed by Dumbo. First schema will be used as default schema if set. | `public`                |\n| ssl               | SSL mode to use: `none`, `trusted` or `system`.                                                           | `none`                  |\n\n##### Examples:\n\n```shell\ndumbo \\\n  -user=postgres \\\n  -password=\"my safe passw0rd\" \\\n  -url=postgresql://localhost:5432/postgres \\\n  -location=/path/to/db/migration \\\n  migrate\n```\n\n```shell\ndumbo help migrate\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frolang%2Fdumbo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frolang%2Fdumbo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frolang%2Fdumbo/lists"}