{"id":22398825,"url":"https://github.com/schubergphilis/data-migrator","last_synced_at":"2026-03-15T20:51:17.011Z","repository":{"id":19884402,"uuid":"88175998","full_name":"schubergphilis/data-migrator","owner":"schubergphilis","description":"A declarative data-migration package","archived":false,"fork":false,"pushed_at":"2024-10-29T22:51:53.000Z","size":280,"stargazers_count":16,"open_issues_count":17,"forks_count":4,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-10-30T00:47:48.738Z","etag":null,"topics":["anonymization","data-policy","data-structures","database","development-tools","development-utils","gdpr","kinesis","migration","python-library"],"latest_commit_sha":null,"homepage":"http://data-migrator.readthedocs.io/en/latest/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/schubergphilis.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-13T14:50:29.000Z","updated_at":"2024-07-19T09:11:44.000Z","dependencies_parsed_at":"2024-05-29T01:43:42.133Z","dependency_job_id":"2dfaa70c-5b56-4838-95a4-98ec16e849d2","html_url":"https://github.com/schubergphilis/data-migrator","commit_stats":{"total_commits":208,"total_committers":3,"mean_commits":69.33333333333333,"dds":0.4903846153846154,"last_synced_commit":"b5ced9abd6bc2c60e9c115951fee38c2fd08923f"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schubergphilis%2Fdata-migrator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schubergphilis%2Fdata-migrator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schubergphilis%2Fdata-migrator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/schubergphilis%2Fdata-migrator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/schubergphilis","download_url":"https://codeload.github.com/schubergphilis/data-migrator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228249352,"owners_count":17891461,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anonymization","data-policy","data-structures","database","development-tools","development-utils","gdpr","kinesis","migration","python-library"],"created_at":"2024-12-05T07:12:22.435Z","updated_at":"2026-03-15T20:51:11.975Z","avatar_url":"https://github.com/schubergphilis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Data-migrator (version 0.6.3.dev2) is a simple data-migration package for python lovers.\n\n.. image:: https://img.shields.io/pypi/pyversions/data-migrator.svg\n    :target: https://pypi.python.org/pypi/data-migrator\n\n.. image:: https://circleci.com/gh/schubergphilis/data-migrator.svg?style=shield\n    :target: https://circleci.com/gh/schubergphilis/data-migrator\n\n----\n\n.. image:: https://pyup.io/repos/github/schubergphilis/data-migrator/shield.svg\n     :target: https://pyup.io/repos/github/schubergphilis/data-migrator/\n     :alt: Updates\n\n.. image:: https://readthedocs.org/projects/data-migrator/badge/?version=latest\n    :target: http://data-migrator.readthedocs.io/en/latest/?badge=latest\n    :alt: Documentation Status\n\n.. image:: https://api.codacy.com/project/badge/Grade/bf6030e9e7e248979607802880336611\n    :target: https://www.codacy.com/app/schubergphilis/data-migrator?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=schubergphilis/data-migrator\u0026amp;utm_campaign=Badge_Grade\n\n.. image:: https://api.codacy.com/project/badge/Coverage/bf6030e9e7e248979607802880336611\n    :target: https://www.codacy.com/app/schubergphilis/data-migrator?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=schubergphilis/data-migrator\u0026amp;utm_campaign=Badge_Coverage\n\n.. image:: https://badge.fury.io/py/data-migrator.svg\n    :target: https://badge.fury.io/py/data-migrator\n\n\nData-migrator is a declarative DSL for table driven data transformations, set up as\nan open and extensive system. Use this to create data transformations for\nchanging databases as a result of changing code, initial loads to datalakes\n(it contains a Kinesis provider)and more.\n\nExample\n-------\n\nData-migrator assumes data is extracted and loaded with client access.\n\n.. code-block:: bash\n\n\t$ mysql source_db -E 'select id,a,b from table' -B  | python my_filter.py | mysql target_db\n\nIt than offers a wide range of primitives with default settings to build complex transformations\nfast, readable and extendable\n\n.. code-block:: python\n\n  from data_migrator import models, transform\n\n  class Result(models.Model):\n    id   = models.IntField(pos=0) # keep id\n    uuid = models.UUIDField()     # generate new uuid4 field\n    a    = models.StringField(pos=1, default='NO_NULL', max_length=5, nullable='NULL', replacement=lambda x:x.upper())\n    b    = models.StringField(pos=2, name='my_b')\n\n  if __name__ == \"__main__\":\n    transform.Transformer(models=[Result]).process()\n\nInstallation\n------------\n\nExecute the following command to install *data-migrator* with ``pip``::\n\n    pip install data-migrator\n\nSee the `Installation Instructions\n\u003chttp://data-migrator.readthedocs.io/en/latest/install.html\u003e`_ in Documentation for\nmore instructions on installing, upgrading, and uninstalling *data-migrator*.\n\nThe project is `maintained at GitHub \u003chttps://github.com/schubergphilis/data-migrator\u003e`_.\n\nSupport and contribute\n----------------------\nQuestions, comments, bug reports and especially tested patches may be\nsubmitted directly to the `issue tracker\n\u003chttps://github.com/schubergphilis/data-migrator/issues\u003e`_.\n\nEveryone interacting with this codebase, issue trackers,\nchat rooms, and mailing lists is expected to follow the\n`Code of Conduct \u003chttp://data-migrator.readthedocs.io/en/latest/code-of-conduct.html\u003e`_.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fschubergphilis%2Fdata-migrator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fschubergphilis%2Fdata-migrator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fschubergphilis%2Fdata-migrator/lists"}