{"id":29162158,"url":"https://github.com/bearmug/erlang-pure-migrations","last_synced_at":"2025-08-23T01:33:10.506Z","repository":{"id":57537964,"uuid":"167261418","full_name":"bearmug/erlang-pure-migrations","owner":"bearmug","description":"Erlang ❤ pure database migrations","archived":false,"fork":false,"pushed_at":"2024-06-20T13:08:45.000Z","size":1811,"stargazers_count":22,"open_issues_count":7,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-26T01:10:02.235Z","etag":null,"topics":["database","erlang","migration","mysql","postgresql","version-control"],"latest_commit_sha":null,"homepage":"","language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bearmug.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-23T22:05:30.000Z","updated_at":"2025-03-19T09:05:41.000Z","dependencies_parsed_at":"2024-11-14T23:00:41.699Z","dependency_job_id":"7797514f-69cb-414d-9dac-c0c75d45c76b","html_url":"https://github.com/bearmug/erlang-pure-migrations","commit_stats":{"total_commits":23,"total_committers":1,"mean_commits":23.0,"dds":0.0,"last_synced_commit":"7e4e4065628d4729fee381c0b3c4bf80635039f0"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/bearmug/erlang-pure-migrations","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearmug%2Ferlang-pure-migrations","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearmug%2Ferlang-pure-migrations/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearmug%2Ferlang-pure-migrations/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearmug%2Ferlang-pure-migrations/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bearmug","download_url":"https://codeload.github.com/bearmug/erlang-pure-migrations/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bearmug%2Ferlang-pure-migrations/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262804337,"owners_count":23367108,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","erlang","migration","mysql","postgresql","version-control"],"created_at":"2025-07-01T05:01:09.030Z","updated_at":"2025-07-01T05:02:24.812Z","avatar_url":"https://github.com/bearmug.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Erlang ❤ pure database migrations\n![](./logo.png)\n\u003e PostgreSQL | MySQL version control engine. Applies effects deliberately.\n\n[![Build Status](https://travis-ci.org/bearmug/erlang-pure-migrations.svg?branch=master)](https://travis-ci.org/bearmug/erlang-pure-migrations)\n[![Coverage Status](https://coveralls.io/repos/github/bearmug/erlang-pure-migrations/badge.svg?branch=master)](https://coveralls.io/github/bearmug/erlang-pure-migrations?branch=master)\n[![Hex.pm](https://img.shields.io/hexpm/v/pure_migrations.svg)](https://hex.pm/packages/pure_migrations)\n\nMigrate your PostgreSQL or MySQL database from Erlang code with no effort.\nThis amazing toolkit has [one and only](https://en.wikipedia.org/wiki/Unix_philosophy)\npurpose - consistently upgrade database schema, using Erlang stack and\nplain SQL. Feel free to run it with any PostgreSQL/MySQL Erlang library (and see\nseveral ready-to-use examples below). As an extra - do this in\n\"no side-effects\" mode.\n\n# Table of contents\n- [Current limitations](#current-limitations)\n- [Quick start](#quick-start)\n  * [Compatibility table](#compatibility-table)\n  * [FAQ](#faq)\n    + [Is it possible to have integration against custom version of PostgreSQL or MySQL?](#is-it-possible-to-have-integration-against-custom-version-of-postgresql-or-mysql)\n    + [Why there are no integrations in production code?](#why-there-are-no-integrations-in-production-code)\n    + [What is the idea behind strict migration scripts numbering?](#what-is-the-idea-behind-strict-migration-scripts-numbering)\n  * [Live integrations](#live-integrations)\n    + [PostgreSQL and epgsql/epgsql](#postgresql-and-epgsqlepgsql)\n      - [Onboarding comments](#onboarding-comments)\n      - [Code sample](#code-sample)\n    + [PostgreSQL and semiocast/pgsql](#postgresql-and-semiocastpgsql)\n      - [Onboarding comments](#onboarding-comments-1)\n      - [Code sample](#code-sample-1)\n    + [PostgreSQL and processone/p1_pgsql](#postgresql-and-processonep1_pgsql)\n      - [Onboarding comments](#onboarding-comments-2)\n      - [Code sample](#code-sample-2)\n    + [MySQL and mysql-otp/mysql-otp](#mysql-and-mysql-otpmysql-otp)\n      - [Onboarding comments](#onboarding-comments-3)\n      - [Code sample](#code-sample-3)\n- [\"No-effects\" approach and tools used to achieve it](#no-effects-approach-and-tools-used-to-achieve-it)\n  * [Tool #1: effects externalization](#tool-1-effects-externalization)\n  * [Tool #2: make effects explicit](#tool-2-make-effects-explicit)\n- [Functional programming abstractions used](#functional-programming-abstractions-used)\n  * [Functions composition](#functions-composition)\n  * [Functor applications](#functor-applications)\n  * [Partial function applications](#partial-function-applications)\n- [License](#license) \n\n# Current limitations\n * **up** transactional migration available only. No **downgrade**\n    calls available. Either whole **up** migration completes OK\n    or failed and rolled back to the state before migration.\n * Validated MySQL implementation obviously featured with \n    [**implicit commit**](https://dev.mysql.com/doc/refman/5.7/en/implicit-commit.html)\n    behavior, which means that truly transactional MySQL upgrades limited \n    in scope. At the same time you may adjust MySQL transaction callback, \n    as it is proposed by [API](#quick-start).\n * migrations engine **deliberately isolated from any specific\n    database library**. This way engine user is free to choose from variety\n    of frameworks (see tested combinations [here](#compatibility-table)) \n    and so on.\n\n# Quick start\nJust call `pure_migrations:migrate/3` (see specification [here](src/engine.erl#L9)), providing:\n * `Path` to migration scripts folder (strictly and incrementally enumerated).\n * `FTx` transaction handler\n * `FQuery` database queries execution handler\n\nMigration logic is idempotent and could be executed multiple times\nagainst the same database with the same migration scripts set. Moreover,\nit is safe to migrate your database concurrently (as a part of nodes\nstartup in scalable environments and if you providing proper transaction\nhandler). Please see verified integrations and live code snippets below.\n\n## Compatibility table\nAll integrations validated against PostgreSQL 9.4/9.6\n\n| Database dialect | Library | Example |\n| -------------- | ------ | ------- |\n| postgres  | [epgsql/epgsql:4.2.1](https://github.com/epgsql/epgsql/releases/tag/4.2.1) | [epgsql test](test/epgsql_migrations_SUITE.erl)\n| postgres  | [semiocast/pgsql:v26.0.2](https://github.com/semiocast/pgsql/releases/tag/v26.0.2) | [spgsql test](test/spgsql_migrations_SUITE.erl)\n| postgres  | [processone/p1_pgsql:1.1.6](https://github.com/processone/p1_pgsql/releases/tag/1.1.6) | [p1pgsql test](test/p1pgsql_migrations_SUITE.erl)\n| mysql     | [mysql-otp/mysql-otp:1.4.0](https://github.com/mysql-otp/mysql-otp/releases/tag/1.4.0) | [otp_mysql test](test/otp_mysql_migrations_SUITE.erl)\n| postgres  | any library with basic sql functional | [generic test](test/pure_migrations_SUITE.erl)\n\n## FAQ\n### Is it possible to have integration against custom version of PostgreSQL or MySQL?\nSure! Please follow these simple steps below:\n* for **local build** just amend related [PostgreSQL](./Makefile#L5) \n  or [MySQL](./Makefile#L4) images references inside project Makefile\n  and just run `make local`.\n* for your [CI](https://travis-ci.org/) build experiments please follow related \n  Travis docs for [Postgres](https://docs.travis-ci.com/user/database-setup/#postgresql) \n  or [MySQL](https://docs.travis-ci.com/user/database-setup/#mysql) \n  instructions.\n### Why there are no integrations in production code?\nLibrary production code has no third-party dependencies at all. \nCode becomes extremely lightweight and decoupled from particular\nlibrary bottlenecks. User absolutely free to choose any \nimplementation (maybe one of [validated ones](#live-integrations)) \nand it's version as well.\n### What is the idea behind strict migration scripts numbering?\nThe approach could be expressed as 2 rules:\n1. Each script name has a number prefix\n2. Numbers should start from 0 and increment strictly by 1\n\nThis model gives much more clarity for migrations sequence. And there\nis no chance to interlace migrations accidentally. Which is the case\nif multiple migrations are being developed and merged to default branch\nsimultaneously.\n\n## Live integrations\n### PostgreSQL and [epgsql/epgsql](https://github.com/epgsql/epgsql)\n#### Onboarding comments\n+ most popular out of onboarded postgres integrations\n+ transactions with proper stack trace available out of the box\n+ reasonably structured query responses, provided with data and its schema\n- although binary strings could be very reasonable, sometimes code too\n  verbose because of this\n#### Code sample\n\u003cdetails\u003e\n  \u003csummary\u003eClick to expand\u003c/summary\u003e\n\n  ```erlang\n  Conn = ?config(conn, Opts),\n  MigrationCall =\n    pure_migrations:migrate(\n      \"scripts/folder/path\",\n      fun(F) -\u003e epgsql:with_transaction(Conn, fun(_) -\u003e F() end) end,\n      fun(Q) -\u003e\n        case epgsql:squery(Conn, Q) of\n          {ok, [\n            {column, \u003c\u003c\"version\"\u003e\u003e, _, _, _, _, _},\n            {column, \u003c\u003c\"filename\"\u003e\u003e, _, _, _, _, _}], Data} -\u003e\n              [{list_to_integer(binary_to_list(BinV)), binary_to_list(BinF)} || {BinV, BinF} \u003c- Data];\n          {ok, [{column, \u003c\u003c\"max\"\u003e\u003e, _, _, _, _, _}], [{null}]} -\u003e -1;\n          {ok, [{column, \u003c\u003c\"max\"\u003e\u003e, _, _, _, _, _}], [{N}]} -\u003e\n            list_to_integer(binary_to_list(N));\n          [{ok, _, _}, {ok, _}] -\u003e ok;\n          {ok, _, _} -\u003e ok;\n          {ok, _} -\u003e ok;\n          Default -\u003e Default\n        end\n      end),\n  ...\n  %% more preparation steps if needed\n  ...\n  %% migration call\n  ok = MigrationCall(),\n\n  ```\nAlso see examples from live epgsql integration tests\n[here](test/epgsql_migrations_SUITE.erl)\n\u003c/details\u003e\n\n### PostgreSQL and [semiocast/pgsql](https://github.com/semiocast/pgsql)\n#### Onboarding comments\n+ no need for extra parsing (strings, numbers, ...)\n- queries results structure has no metadata, like column types or names,\n  which could be sub-optimal sometimes\n- no transactions out of the box\n#### Code sample\n\u003cdetails\u003e\n  \u003csummary\u003eClick to expand\u003c/summary\u003e\n\n  ```erlang\n  Conn = ?config(conn, Opts),\n  MigrationCall =\n    pure_migrations:migrate(\n      \"scripts/folder/path\",\n      fun(F) -\u003e\n        pgsql_connection:simple_query(\"BEGIN\", Conn),\n        try F() of\n          Res -\u003e\n            pgsql_connection:simple_query(\"COMMIT\", Conn),\n            Res\n        catch\n           _:Problem -\u003e\n             pgsql_connection:simple_query(\"ROLLBACK\", Conn),\n             {rollback, Problem}\n        end\n      end,\n      fun(Q) -\u003e\n        case pgsql_connection:simple_query(Q, Conn) of\n          {{select, 0}, []} -\u003e [];\n          {{select, 1}, Data = [{_V, _F}|_]}  -\u003e\n            [{V, binary_to_list(BinF)} || {V, BinF} \u003c- Data];\n          {{select, 1}, [{null}]} -\u003e -1;\n          {{select, 1}, [{N}]} -\u003e N;\n          {{insert, 0, 1}, []} -\u003e ok;\n          {{create, table},[]} -\u003e ok;\n          {error, Details} -\u003e {error, Details};\n          _ -\u003e ok\n        end\n      end),\n  ...\n  %% more preparation steps if needed\n  ...\n  %% migration call\n  ok = MigrationCall(),\n\n  ```\nAlso see examples from live semiocast/pgsql integration tests\n[here](test/spgsql_migrations_SUITE.erl)\n\u003c/details\u003e\n\n### PostgreSQL and [processone/p1_pgsql](https://github.com/processone/p1_pgsql)\n#### Onboarding comments\n+ least popular lib,but at the same time - most succinct in terms of\n  integration code (see below)\n+ decent types balance gives opportunity to keep code clean\n- no transactions out of the box\n- error reporting different for postgres 9.4/9.6\n#### Code sample\n\u003cdetails\u003e\n  \u003csummary\u003eClick to expand\u003c/summary\u003e\n\n  ```erlang\n  Conn = ?config(conn, Opts),\n  MigrationCall =\n    pure_migrations:migrate(\n      \"scripts/folder/path\",\n      fun(F) -\u003e\n        pgsql:squery(Conn, \"BEGIN\"),\n        try F() of\n          Res -\u003e\n            pgsql:squery(Conn, \"COMMIT\"),\n            Res\n        catch\n           _:Problem -\u003e\n             pgsql:squery(Conn, \"ROLLBACK\"),\n             {rollback, Problem}\n        end\n      end,\n      fun(Q) -\u003e\n        case pgsql:squery(Conn, Q) of\n          {ok, [{error, Details}]} -\u003e {error, Details};\n          {ok, [{_, [\n                     {\"version\", text, _, _, _, _, _},\n                     {\"filename\", text, _, _, _, _, _}], Data}]} -\u003e\n              [{list_to_integer(V), F} || [V, F] \u003c- Data];\n          {ok, [{\"SELECT 1\", [{\"max\", text, _, _, _, _, _}], [[null]]}]} -\u003e -1;\n          {ok, [{\"SELECT 1\", [{\"max\", text, _, _, _, _, _}], [[N]]}]} -\u003e\n              list_to_integer(N);\n          {ok, _} -\u003e ok\n        end\n      end),\n  ...\n  %% more preparation steps if needed\n  ...\n  %% migration call\n  ok = MigrationCall(),\n\n  ```\nAlso see examples from live epgsql integration tests\n[here](test/p1pgsql_migrations_SUITE.erl)\n\u003c/details\u003e\n\n### MySQL and [mysql-otp/mysql-otp](https://github.com/mysql-otp/mysql-otp)\n#### Onboarding comments\n+ almost no result-set parsing required\n- [implicit commit](https://dev.mysql.com/doc/refman/5.7/en/implicit-commit.html)\n  specifics a kind an obstacle for simple and safe migration\n- mysql docker tooling should be operated carefully and ensured for \n  proper startup before any use\n#### Code sample\n\u003cdetails\u003e\n  \u003csummary\u003eClick to expand\u003c/summary\u003e\n\n  ```erlang\n  Conn = ?config(conn, Opts),\n  MigrationCall =\n    pure_migrations:migrate(\n      \"scripts/folder/path\",\n      fun(F) -\u003e\n        %% no full-scope tx API available here\n        %% alternatively use mysql:transaction/2, but please be aware about\n        %% mysql implicit transactions commit behavior\n        try F() of\n          Res -\u003e Res\n        catch\n          _:Problem -\u003e {rollback_unavailable, Problem}\n        end\n      end,\n      fun(Q) -\u003e\n        case mysql:query(Conn, Q) of\n          {error, Details} -\u003e {error, Details};\n          {ok,[\u003c\u003c\"version\"\u003e\u003e,\u003c\u003c\"filename\"\u003e\u003e],[]} -\u003e [];\n          {ok,[\u003c\u003c\"version\"\u003e\u003e,\u003c\u003c\"filename\"\u003e\u003e], Data} -\u003e\n              [{V, binary_to_list(F)} || [V, F] \u003c- Data];\n          {ok,[\u003c\u003c\"max(version)\"\u003e\u003e],[[null]]} -\u003e -1;\n          {ok,[\u003c\u003c\"max(version)\"\u003e\u003e],[[V]]} -\u003e V;\n          {ok, _} -\u003e ok;\n          ok -\u003e ok\n        end\n      end),\n  ...\n  %% more preparation steps if needed\n  ...\n  %% migration call\n  ok = MigrationCall(),\n\n  ```\nAlso see examples from live epgsql integration tests\n[here](test/otp_mysql_migrations_SUITE.erl)\n\u003c/details\u003e\n\n# \"No-effects\" approach and tools used to achieve it\nOh, **there is more!** Library implemented in the [way](https://en.wikipedia.org/wiki/Pure_function),\nthat all side-effects either externalized or deferred explicitly. Reasons\nare quite common:\n * bring side-effects as close to program edges as possible. Which may\n mean enhanced code reasoning, better bugs reproduceability, etc...\n * simplify module contracts testing\n * library users empowered to re-run idempotent code safely. Well, if\n tx/query handlers are real ones - execution is still idempotent (at\n application level) and formally pure. But purity maintained inside\n library code only. One call is to be issued anyway - migrations\n table creation, if this one does not exists.\n\n## Tool #1: effects externalization\nThere are 2 externalized kind of effects:\n * transaction management handler\n * database queries handler\nAlthough, those two can`t be pure in real application, it is fairly\nsimple to replace them with their pure versions if we would like to\n(for debug purposes, or testing, or something else).\n\n## Tool #2: make effects explicit\nOther effects (like file operations) are deferred in bulk with outcome\nlike:\n * pure referentially-transparent program actions composed only. Impact\n or any communication with external world postponed until later stages\n * library users decide when they ready to apply migration changes.\n Maybe for some reason they would like to prepare execution -\u003e\n prepare migrations folder content -\u003e run migrations.\n\n# Functional programming abstractions used\n## Functions composition\nThis trick is quite useful if someone would like to compose two functions\nwithout their actual execution (or without their application,\nalternatively speaking). This pretty standard routine may look like below\n(Scala or Kotlin+Arrow):\n```scala\nval divideByTwo = (number : Int) =\u003e number / 2;\nval addThree = (number: Int) =\u003e number + 3;\nval composed = addThree compose divideByTwo\n```\nSimplistic Erlang version:\n```erlang\ncompose(F1, F2) -\u003e fun() -\u003e F2(F1()) end.\n```\n\n## Functor applications\nThere area few places in library with clear need to compose function **A**\nand another function **B** inside deferred execution context. Specifics is\nthat **A** supplies list of objects, and **B** should be applied to each of\nthem. Sounds like some functor **B** to be applied to **A** output, when\nthis output is being wrapped into future execution context. Two cases\nof this appeared in library:\n * have functor running and produce nested list of contexts:\n```erlang\n%% Map/1 call here produces new context (defferred function call)\nmap(Generate, Map) -\u003e fun() -\u003e [Map(R) || R \u003c- Generate()] end.\n```\n * flatten (or fold) contexts (or function calls) list to a single one:\n```erlang\nflatten(Generate) -\u003e fun() -\u003e [ok = R() || R \u003c- Generate()], ok end.\n```\n## Partial function applications\nPartial application is very useful, in case if not all function arguments\nknown yet. Or maybe there is deliberate decision to pass some of arguments\nlater on. Again, in Scala it may look like:\n```scala\nval add = (a: Int, b: Int) =\u003e a + b\nval partiallyApplied = add(3, _)\n```\nLibrary code has very simplistic partial application, done for exact\narguments number (although it is easy to generalize it for arguments,\nrepresented as list):\n```erlang\nPartial = fun(V_F) -\u003e do_migration(Path, FQuery, V_F) end,\n```\n\n# License\nMIT, see [LICENSE](./LICENSE) for more details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbearmug%2Ferlang-pure-migrations","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbearmug%2Ferlang-pure-migrations","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbearmug%2Ferlang-pure-migrations/lists"}