{"id":23003007,"url":"https://github.com/educationwarehouse/migrate","last_synced_at":"2025-10-30T10:44:27.563Z","repository":{"id":155620771,"uuid":"631314695","full_name":"educationwarehouse/migrate","owner":"educationwarehouse","description":"Migrate database changes","archived":false,"fork":false,"pushed_at":"2024-04-24T11:46:09.000Z","size":184,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-07-07T09:15:02.517Z","etag":null,"topics":["database","python"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/edwh-migrate/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/educationwarehouse.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-22T16:19:25.000Z","updated_at":"2024-04-24T11:46:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"f437495a-a4c5-454d-a61d-d9dbca575b0e","html_url":"https://github.com/educationwarehouse/migrate","commit_stats":null,"previous_names":[],"tags_count":36,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/educationwarehouse%2Fmigrate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/educationwarehouse%2Fmigrate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/educationwarehouse%2Fmigrate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/educationwarehouse%2Fmigrate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/educationwarehouse","download_url":"https://codeload.github.com/educationwarehouse/migrate/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246828706,"owners_count":20840516,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","python"],"created_at":"2024-12-15T07:13:12.585Z","updated_at":"2025-10-30T10:44:27.549Z","avatar_url":"https://github.com/educationwarehouse.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Educationwarehouse's Migrate\n\n[![PyPI - Version](https://img.shields.io/pypi/v/edwh-migrate.svg)](https://pypi.org/project/edwh-migrate)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/edwh-migrate.svg)](https://pypi.org/project/edwh-migrate)\n\n-----\n\n**Table of Contents**\n\n- [Installation](#installation)\n- [Documentation](#documentation)\n- [License](#license)\n\n## Installation\n\n```console\npip install edwh-migrate\n# or to include extra dependencies (psycopg2, redis):\npip install edwh-migrate[full]\n```\n\n## Documentation\n\n### Config: Environment variables\n\nThese variables can be set in the current environment or via `.env`:\n\n* `MIGRATE_URI` (required): regular `postgres://user:password@host:port/database` or `sqlite:///path/to/database` URI\n* `MIGRATIONS_FILE` (optional) path to a file containing migrations. By default, a file called `migrations.py` in the working directory is used.\n* `DATABASE_TO_RESTORE`: path to a (compressed) SQL file to restore. `.xz`,`.gz` and `.sql` are supported.\n* `MIGRATE_CAT_COMMAND`: for unsupported compression formats, this command decompresses the file and produces sql on the\n  stdout.\n* `SCHEMA_VERSION`: Used in case of schema versioning. Set by another process.\n* `REDIS_HOST`: If set, all keys of the redis database 0 will be removed.\n* `MIGRATE_TABLE`: name of the table where installed migrations are stored. Defaults to `ewh_implemented_features`. \n* `FLAG_LOCATION`: when using schema versioned lock files, this directory is used to store the flags. Defaults to `/flags`.\n* `CREATE_FLAG_LOCATION` (bool): should the directory above be created if it does not exist yet? Defaults to 0 (false). \n* `SCHEMA`: (for postgres) set the default namespace (`search_path`). Defaults to `public`.\n* `USE_TYPEDAL`: pass a TypeDAL instance to migrations instead of a regular pyDAL.\n\n### Config: pyproject.toml\n\nYou can also set your config variables via the `[tool.migrate]` key in `pyproject.toml`.\nFirst, these variables are loaded and then updated with variables from the environment.\nThis way, you can set static variables (the ones you want in git, e.g. the `migrate_table` name or path to the backup to\nrestore) in the toml, and keep private/dynamic vars in the environment (e.g. the database uri or schema version).\n\nExample:\n\n```toml\n[tool.migrate]\nmigrate_uri = \"\" # filled in by .env\ndatabase-to-restore = \"migrate/data/db_backup.sql\"\n# ...\n```\n\n### Creating a `migrations.py`\n\n```python\nfrom edwh_migrate import migration\n\n@migration\ndef feature_1(db):\n    print(\"feature_1\")\n    return True\n\n\n@migration(requires=[feature_1]) # optional `requires` ensures previous migration(s) are installed\ndef functionalname_date_sequencenr(db: pydal.DAL):\n    db.executesql(\"\"\"\n        CREATE TABLE ...\n    \"\"\")\n    db.commit()\n    return True\n\n```\n\n### Usage\n\nWhen your configuration is set up properly and you have a file containing your migrations, you can simply run:\n\n```bash\nmigrate\n# or, to use a different name than migrations.py:\nmigrate path/to/my/migrate_file.py\n```\n\n## Advanced Topics\n\n### Using `ViewMigrationManager` via Subclasses\n\n`ViewMigrationManager` is designed to manage the lifecycle of view migrations in a database using context management. It ensures that migrations are properly handled with dependencies between different migrations.\n\n#### Usage\n\n1. **Define Subclasses**: Create subclasses of `ViewMigrationManager` and implement the required methods `up` and `down`.\n\n    ```python\n    from edwh_migrate import ViewMigrationManager\n\n    class MyExampleView_V1(ViewMigrationManager):\n        # Define dependencies (optional)\n        uses = ()\n        # Specify a migration that must have run before this class may be used\n        since = \"previous_migration\"\n\n        def up(self):\n            # Logic to apply the migration\n            self.db.executesql(\n                '''\n                CREATE MATERIALIZED VIEW my_example_view AS\n                SELECT id, name FROM my_table;\n                '''\n            )\n\n        def down(self):\n            # Logic to reverse the migration\n            self.db.executesql(\n                '''\n                DROP MATERIALIZED VIEW IF EXISTS my_example_view;\n                '''\n            )\n\n    class AnotherExampleView(ViewMigrationManager):\n        # This class depends on MyExampleView_V1\n        uses = (MyExampleView_V1,)\n\n        def up(self):\n            # Logic to apply the migration\n            self.db.executesql(\n                '''\n                CREATE MATERIALIZED VIEW another_example_view AS\n                SELECT id, name FROM my_example_view;\n                '''\n            )\n\n        def down(self):\n            # Logic to reverse the migration\n            self.db.executesql(\n                '''\n                DROP MATERIALIZED VIEW IF EXISTS another_example_view;\n                '''\n            )\n    ```\n\n2. **Define the `previous_migration`**: Create a migration function that serves as the prerequisite for `MyExampleView_V1`.\n\n    ```python\n    from edwh_migrate import migration\n\n    @migration\n    def previous_migration(db):\n        db.executesql('''\n        CREATE TABLE my_table (\n            id SERIAL PRIMARY KEY,\n            name VARCHAR(255)\n        );\n        ''')\n        db.commit()\n        return True\n    ```\n\n3. **Use the Subclass in a Migration Function**: Utilize the subclass in a migration function to manage the view migration context.\n\n    ```python\n    from edwh_migrate import migration\n\n    @migration\n    def upgrade_some_source_table_that_my_example_view_depends_on(db):\n        with MyExampleView_V1(db):\n            db.executesql('''\n            ALTER TABLE my_table\n            ADD COLUMN new_column VARCHAR(255);\n            ''')\n        db.commit()\n        return True\n    ```\n\nIn the example above:\n- `MyExampleView_V1` is a subclass of `ViewMigrationManager` that manages the lifecycle of a materialized view named `my_example_view`.\n- `AnotherExampleView` is another subclass that depends on `MyExampleView_V1` and manages the lifecycle of another materialized view named `another_example_view`.\n- The `up` method in `MyExampleView_V1` contains the logic to create the materialized view `my_example_view`.\n- The `down` method in `MyExampleView_V1` contains the logic to drop the materialized view `my_example_view`.\n- The `up` method in `AnotherExampleView` contains the logic to create the materialized view `another_example_view` that references `my_example_view`.\n- The `down` method in `AnotherExampleView` contains the logic to drop the materialized view `another_example_view`.\n- The `since` attribute specifies that a particular migration (`previous_migration`) must have run before `MyExampleView_V1` may be used.\n- The `previous_migration` function creates the table `my_table` and serves as a prerequisite for `MyExampleView_V1`.\n- The `migration` decorator is used to define a migration function (`upgrade_some_source_table_that_my_example_view_depends_on`) that executes within the context of `MyExampleView_V1`.\n- The `with MyExampleView_V1(db)` block ensures that the `down` method is called before the block executes and the `up` method is called after the block completes.\n\nIn addition to 'since', the inverse 'until' can also be used.\n\n## License\n\n`edwh-migrate` is distributed under the terms of the [MIT](https://spdx.org/licenses/MIT.html) license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feducationwarehouse%2Fmigrate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feducationwarehouse%2Fmigrate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feducationwarehouse%2Fmigrate/lists"}