{"id":32500996,"url":"https://github.com/k2bd/flux-migrations","last_synced_at":"2026-05-07T09:33:57.621Z","repository":{"id":256978919,"uuid":"797946917","full_name":"k2bd/flux-migrations","owner":"k2bd","description":"A database migration tool written for Python projects","archived":false,"fork":false,"pushed_at":"2025-01-19T13:13:28.000Z","size":339,"stargazers_count":1,"open_issues_count":13,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-27T16:37:31.532Z","etag":null,"topics":["database","database-design","database-management","database-migrations","database-schema","databases","dbml","postgres","postgresql","python","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/k2bd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-05-08T19:36:15.000Z","updated_at":"2025-01-19T13:13:29.000Z","dependencies_parsed_at":"2024-09-14T06:48:14.187Z","dependency_job_id":"50018adf-8d0d-4d22-92bc-be3971bc20ce","html_url":"https://github.com/k2bd/flux-migrations","commit_stats":null,"previous_names":["k2bd/flux-migrations"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/k2bd/flux-migrations","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k2bd%2Fflux-migrations","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k2bd%2Fflux-migrations/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k2bd%2Fflux-migrations/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k2bd%2Fflux-migrations/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/k2bd","download_url":"https://codeload.github.com/k2bd/flux-migrations/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k2bd%2Fflux-migrations/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32731541,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-07T02:14:30.463Z","status":"ssl_error","status_checked_at":"2026-05-07T02:14:29.405Z","response_time":62,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","database-design","database-management","database-migrations","database-schema","databases","dbml","postgres","postgresql","python","python3"],"created_at":"2025-10-27T16:24:56.813Z","updated_at":"2026-05-07T09:33:57.599Z","avatar_url":"https://github.com/k2bd.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flux Migrations\n\n[![codecov](https://codecov.io/gh/k2bd/flux-migrations/graph/badge.svg?token=PJF3cYLtZh)](https://codecov.io/gh/k2bd/flux-migrations)\n![PyPI - Python Version](https://img.shields.io/pypi/pyversions/flux-migrations)\n[![PyPI - Version](https://img.shields.io/pypi/v/flux-migrations)](https://pypi.org/project/flux-migrations/)\n\n\n`flux` is a database migration tool written in Python and built with Python projects in mind.\n\n## N.B. this project is in a pre-release state. It is not ready for use in any form. There may be major problems and breaking changes to all aspects of the tool while some decisions are being made and changed.\n\n## Adding `flux` to your project\n\n### CLI\n\n``flux`` can be installed for now from Github. For example:\n\n```\npoetry add \"git+https://github.com/k2bd/flux-migrations.git[postgres]\"\n```\n\nThe project will be properly maintained on PyPI when it's stable. The PyPI version may therefore not be up-to-date at this time.\n\n``flux`` commands can then be listed with ``flux --help``.\nAny subcommand can also be suffixed with ``--help`` for more information about it and any options and arguments.\n\nThe main commands are:\n\n- ``flux init {backend}`` - Create a ``flux.toml`` file in the current directory for your project with an installed ``{backend}``\n- ``flux new \"Migration short description\"`` - Create a new migration with the given short description\n- ``flux apply {database-uri}`` Apply unapplied migrations to the target ``{database-uri}``\n- ``flux rollback {database-uri}`` Rollback applied migrations from the target ``{database-uri}``\n\nFor example, migrations can be initialized and started with:\n\n```\nflux init postgres\n\nflux new \"Initial tables\"\n```\n\n## Writing migrations\n\nThere are two forms that migrations can take in ``flux`` - Python files and sql files.\nBoth forms define up and optional down migrations.\n\nIn each case, you can append ``--pre`` or ``--post`` to create pre-apply and post-apply migrations.\nThese will run before/after any batch of migrations are run (by default, they're also run before/after rollbacks, but this can be disabled)\n\n### Migrations as Python files\n\nBy default ``flux`` creates Python migration files when you run ``flux new \"My new migration\"``.\n\nA migration written as a Python file must contain at minimum a function ``apply() -\u003e str`` which returns a string representing one or multiple sql statements for the migration.\nIt can also define a function ``undo() -\u003e str`` that returns a down-migration.\n\nBecause these migration files are Python files, you can write reusable tools to help accelerate producing high quality migrations.\nBut because these functions must return a string, ``flux`` (along with a good testing and deployment strategy) can ensure that changes to these functions can't corrupt older migrations (see [below](#migration-directory-corruption-detection)).\nA common pattern to work around this, in my experience at least, is to have versioned migration helpers suffixed with `_v1`, `_v2` etc, and each new migration is expected to use the latest version of any given helper.\n\nThis means you get the power of modular code when writing migrations *and* the security that resolved migrations remain immutable (even if the file creating the migration imports code from mutable modules).\n\nFor example:\n\n```python\n# -- migration_helpers.py\n\ndef table_admin_permissions_v1(table_name: str) -\u003e str:\n    return f\"\"\"\n    grant select, insert, delete on table {table_name} to admin_user;\n    \"\"\"\n\n\ndef give_admin_permissions_v2(table_name: str) -\u003e str:\n    return f\"\"\"\n    {table_admin_permissions_v1(table_name)}\n    grant select on table {table_name} to read_only_admin_user;\n    \"\"\"\n```\n\nA migration that was created when only v1 of our helper was available:\n\n```python\n# -- 20200202_001_some-old-migration.py\n\nfrom some_package.migration_helpers import table_admin_permissions_v1\n\n\ndef apply():\n    f\"\"\"\n    create table users (\n        id uuid not null primary key default gen_random_uuid(),\n        name text not null\n    );\n    {table_admin_permissions_v1(\"users\")}\n    \"\"\"\n```\n\nAnd another created later with v2:\n\n```python\n# -- 20240422_002_shiny-new-migration.py\n\nfrom some_package.migration_helpers import table_admin_permissions_v2\n\n\ndef apply():\n    f\"\"\"\n    create table user_posts (\n        id uuid not null primary key default gen_random_uuid(),\n        user_id uuid not null references users (id),\n        posted_at timestamp not null default (now() at time zone 'utc'),\n        content text not null\n    );\n    {table_admin_permissions_v2(\"user_posts\")}\n    \"\"\"\n```\n\n(These examples only contain apply functions for brevity - real migrations should have undo steps!)\n\n### Migrations as sql files\n\nIt may be that you prefer just writing sql files for your migrations, and you just want ``flux`` for its flexibility or testing functionality.\nThat's cool too, just run ``flux new --sql \"My new migration\"``.\n\nUp-migration files are just files ending with ``.sql``. They can have down-migration counterparts ending with ``.undo.sql``.\n\nThese files just contain sql, but as above the hash of the up migration is stored for [detecting migration directory corruption](#migration-directory-corruption-detection).\n\n## Migration directory corruption detection\n\nThe hash of the up-migration is stored by ``flux`` to check for migration directory corruption.\nThat is, the content of past migrations are not allowed to change so the record of applied migrations is clear in all environments.\nIf ``flux`` sees that a previously-applied migration has changed content when validating migrations (as a standalone command or as part of e.g. ``apply``), it will raise an error.\n\n## Use as a library\n\n``flux`` can be used as a library in your Python project to manage migrations programmatically.\nThis can be particularly useful for testing.\n\n(TODO: Demo API)\n\n## Database backends\n\n``flux`` is a generic migration tool that can be adapted for use in many databases. It does this by having an abstract backend specification that can be implemented for any target DBMS. Backends can also have their own configuration options.\n\n### Inbuilt backends\n\n#### Postgres\n\n``flux`` comes optionally packaged with a Postgres backend if installed with the `postgres` extra. It maintains information about migrations in a configurable schema and table. Additionally, it uses an advisory lock while migrations are being applied with a configurable index. The available ``[backend]`` configs are:\n\n- ``migrations_schema``\n    - The schema in which to put the migration history table\n    - (default \"public\")\n- ``migrations_table``\n    - The table used for applied migration history\n    - (default \"_flux_migrations\")\n- ``migrations_lock_id``\n    - The ``pg_advisory_lock`` ID to use while applying migrations\n    - (default 3589 ('flux' on a phone keypad))\n\n### Adding a new backend\n\nBackends are loaded as plugins through Python's entry point system.\nThis means that you can add a new backend by simply installing a package that provides the backend as a plugin.\n\nTo create a new backend in your package, you need to subclass ``flux.MigrationBackend`` and implement its abstract methods.\nThen register that class under the ``flux.backend`` entry point group in your package setup.\n\nFor example, in ``pyproject.toml``:\n    \n```toml\n[project.entry-points.\"flux.backend\"]\ncooldb = \"my_package.my_module:CoolDbBackend\"\n```\n\nWhen the new package is installed in the same environment as ``flux``, the backend will be available to use with ``flux``.\nAn example ``flux.toml`` file that uses our new backend:\n\n```toml\n[flux]\nbackend = \"cooldb\"\nmigration_directory = \"migrations\"\n\n[backend]\ncoolness_level = 11\nanother_option = \"cool_value\"\n```\n\n## Why `flux`?\n\nI have used a number of migration frameworks for databases that sit behind Python projects.\nI've liked some features of different projects but the complete feature-set I'd like to use in my work has never been in one project.\n\nA non-exhaustive list of this feature-set includes\n- very flexible support for repeatable migration scripts\n- migration directory corruption detection\n- the ability to easily leverage Python to reuse code in migrations\n- a Python library to easily manage migrations programmatically for test writing (e.g. integration tests of the effects of individual migrations)\n\nSo, the motivation for this project was to\n- present a more complete feature-set you'd want to find in a migration framework for use with Python projects\n- use design patterns that make it easy to adapt for different kinds of projects, such as the plugin-based backend system\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk2bd%2Fflux-migrations","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fk2bd%2Fflux-migrations","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk2bd%2Fflux-migrations/lists"}