{"id":13828553,"url":"https://github.com/DivineOmega/uxdm","last_synced_at":"2025-07-09T06:32:13.408Z","repository":{"id":56969707,"uuid":"130364449","full_name":"DivineOmega/uxdm","owner":"DivineOmega","description":"🔀 UXDM helps developers migrate data from one system or format to another.","archived":false,"fork":false,"pushed_at":"2020-06-01T08:35:02.000Z","size":475,"stargazers_count":167,"open_issues_count":7,"forks_count":9,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-11-16T18:03:11.256Z","etag":null,"topics":["array","csv","data-conversion","data-migration","data-migrator","json","markdown","pdo","php","php-library","uxdm","wordpress","xml"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DivineOmega.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-04-20T13:16:24.000Z","updated_at":"2024-06-25T15:48:19.000Z","dependencies_parsed_at":"2022-08-21T06:40:28.803Z","dependency_job_id":null,"html_url":"https://github.com/DivineOmega/uxdm","commit_stats":null,"previous_names":[],"tags_count":76,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DivineOmega%2Fuxdm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DivineOmega%2Fuxdm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DivineOmega%2Fuxdm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DivineOmega%2Fuxdm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DivineOmega","download_url":"https://codeload.github.com/DivineOmega/uxdm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225492420,"owners_count":17482869,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["array","csv","data-conversion","data-migration","data-migrator","json","markdown","pdo","php","php-library","uxdm","wordpress","xml"],"created_at":"2024-08-04T09:02:52.006Z","updated_at":"2025-07-09T06:32:13.401Z","avatar_url":"https://github.com/DivineOmega.png","language":"PHP","readme":"# 🔀 Universal Extensible Data Migrator (UXDM)\n\nUXDM helps developers migrate data from one system or format to another.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"assets/images/uxdm-data-migration-example.png\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://github.com/DivineOmega/uxdm/actions/workflows/phpunit.yml\"\u003e\u003cimg src=\"https://github.com/DivineOmega/uxdm/actions/workflows/phpunit.yml/badge.svg?branch=master\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://coveralls.io/github/DivineOmega/uxdm?branch=master\"\u003e\u003cimg src=\"https://coveralls.io/repos/github/DivineOmega/uxdm/badge.svg?branch=master\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://styleci.io/repos/130364449\"\u003e\u003cimg src=\"https://styleci.io/repos/130364449/shield?branch=master\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://packagist.org/packages/divineomega/uxdm/stats\"\u003e\u003cimg src=\"https://img.shields.io/packagist/dt/DivineOmega/uxdm.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n## Installation\n\nUXDM can be easily installed using Composer. Just run the following command from the root of your project.\n\n```\ncomposer require divineomega/uxdm\n```\n\nIf you have never used the Composer dependency manager before, head to the [Composer website](https://getcomposer.org/) for more information on how to get started.\n\n## Quick Start\n\n1. Create a new PHP file to contain your UXDM migration code. In this example, we'll call it `user-csv-import.php`. Remember to add `require 'vendor/autoload.php'` and relevant `use` statements, if necessary.\n\n2. Create your source and destination objects. This example uses a CSV source and PDO (database) destination.\n\n```php\n$csvSource = new CSVSource('users.csv');\n\n$pdoDestination = new PDODestination(new PDO('mysql:dbname=test-database;host=127.0.0.1', 'root', 'password'), 'users');\n```\n\n3. Create and configure a new UXDM migrator object.\n\n```php\n$migrator = new Migrator;\n$migrator-\u003esetSource($csvSource)\n         -\u003esetDestination($pdoDestination)\n         -\u003esetFieldsToMigrate(['id', 'email', 'name'])\n         -\u003esetKeyFields(['id'])\n         -\u003ewithProgressBar()\n         -\u003emigrate();\n```\n\n4. Run your newly created migration. In this example, we can just run `php user-csv-import.php` from the command line and will get a nice progress bar.\n\nSee the sections below for more information on the available source and destination objects, and more advanced usage examples.\n\n## Migrations\n\nEach UXDM migration requires a source object and at least one destination object. These determine where and how data is read and written. \nThe UXDM package works with a variety of source and destination objects, including the following.\n\n* PDO (PHP Database Object) Source \u0026 Destination\n* Eloquent (as used in Laravel) Source \u0026 Destination\n* Doctrine (as used in Symfony) Destination\n* CSV (Comma Separated Values) Source \u0026 Destination\n* Excel Source \u0026 Destination\n* Associative Array Source \u0026 Destination\n* JSON Files Source \u0026 Destination\n* XML Source \u0026 Destination\n* WordPress Post Source\n* WordPress User Source\n* Debug Output Destination\n\nSome of these are built-in to the core UXDM package, while others are available as separate packages.\n\nSource and destination objects can be used in any combination. Data can be migrated from a CSV and inserted into a database, just as easily as data can be migrated from a database to a CSV.\n\nYou can also use similar source and destination objects in the same migration. For example, a common use of UXDM is to use a PDO source and PDO destination to transfer data from one database to another. \n\nPlease see the [Sources \u0026 Destinations](/docs/uxdm-sources-and-destinations.md) page for more sources and destinations, and detailed documentation on their usage.\n\n## Examples\n\n### Database to database migration\n\nAn example of a basic database to database UXDM migration is shown below.\n\n```php\n$pdoSource = new PDOSource(new PDO('mysql:dbname=old-test;host=127.0.0.1', 'root', 'password123'), 'users');\n\n$pdoDestination = new PDODestination(new PDO('mysql:dbname=new-test;host=127.0.0.1', 'root', 'password456'), 'new_users');\n\n$migrator = new Migrator;\n$migrator-\u003esetSource($pdoSource)\n         -\u003esetDestination($pdoDestination)\n         -\u003esetFieldsToMigrate(['id', 'email', 'name'])\n         -\u003esetKeyFields(['id'])\n         -\u003ewithProgressBar()\n         -\u003emigrate();\n```\n\nThis migration will move the `id`, `email` and `name` fields from the `users` table in the `old-test` database to the `new_users` table in the `new-test` database, replacing any existing records with the same `id` (the key field).\n\n### Source data validation\n\nYou can use UXDM to validate the source data. If validation fails part way through a migration, the migration will \nhalt and a `ValidationException` will be thrown. However, if `-\u003evalidateBeforeMigrating()` is called, all data rows\nwill be preemptively validated before the migration begins.\n\nThe code below shows how to validate various fields.\n\n```php\n$pdoSource = new PDOSource(new PDO('mysql:dbname=old-test;host=127.0.0.1', 'root', 'password123'), 'users');\n\n$pdoDestination = new PDODestination(new PDO('mysql:dbname=new-test;host=127.0.0.1', 'root', 'password456'), 'new_users');\n\n$migrator = new Migrator;\n$migrator-\u003esetSource($pdoSource)\n         -\u003esetDestination($pdoDestination)\n         -\u003esetFieldsToMigrate(['id', 'email', 'name'])\n         -\u003esetValidationRules([\n            'id' =\u003e [new Required(), new IsNumeric()],\n            'email' =\u003e [new Required(), new IsString(), new IsEmail()],\n            'name' =\u003e [new Required(), new IsString()],\n         ])\n      // -\u003evalidateBeforeMigrating()\n         -\u003esetKeyFields(['id'])\n         -\u003ewithProgressBar()\n         -\u003emigrate();\n```\n\nThis migration will validate the source data matches the defined validation rules.\n\n* 'id' must be present, and numeric.\n* 'email' must be present, a string, and a correctly formatted email address.\n* 'name' must be present, and a string.\n\nUXDM uses the [Omega Validator](https://github.com/DivineOmega/omega-validator) package. \nSee its documentation for all available validation rules.\n\n### Mapping field names from source to destination\n\nThis example shows how UXDM can map field names from source to destination.\n\n```php\n$migrator = new Migrator;\n$migrator-\u003esetSource($pdoSource)\n         -\u003esetDestination($pdoDestination)\n         -\u003esetFieldsToMigrate(['id', 'email', 'name'])\n         -\u003esetKeyFields(['id'])\n         -\u003esetFieldMap(['name' =\u003e 'full_name'])\n         -\u003ewithProgressBar()\n         -\u003emigrate();\n```\n\nThis migration will move data from the source `name` field into the destination `full_name` field, while still moving the `id` and `email` fields normally.\n\n### Transforming data rows during migration\n\nSometimes the data you want to move from source to destination needs transforming. This can be \nchanging existing items of data, adding new data items, or removing items you do not need.\n\nUXDM allows you to create one or more transformer objects, and add them to the migration.\nSee the following examples of how to use transformers to manipulate your data. \n\n#### Changing existing data items\n\nThis example shows how you can transform existing data items during migration.\n\n```php\nclass NameCaseTransformer implements TransformerInterface\n{\n    public function transform(DataRow $dataRow): void\n    {\n        $nameDataItem = $dataRow-\u003egetDataItemByFieldName('name');\n        $nameDataItem-\u003evalue = ucwords(strtolower($nameDataItem-\u003evalue));\n    }\n}\n\n$migrator = new Migrator;\n$migrator-\u003esetSource($pdoSource)\n         -\u003esetDestination($pdoDestination)\n         -\u003esetFieldsToMigrate(['id', 'email', 'name'])\n         -\u003esetKeyFields(['id'])\n         -\u003eaddTransformer(new NameCaseTransformer())\n         -\u003ewithProgressBar()\n         -\u003emigrate();\n```\n\nThis migration will ensure that all name fields have consistent case.\n\n#### Adding data items\n\nThis example shows how you can add new data items while the migration is taking place.\n\n```php\nclass AddRandomNumberTransformer implements TransformerInterface\n{\n    public function transform(DataRow \u0026$dataRow): void\n    {\n        $dataRow-\u003eaddDataItem(new DataItem('random_number', rand(1,1000)));\n    }\n}\n\n$migrator = new Migrator;\n$migrator-\u003esetSource($pdoSource)\n         -\u003esetDestination($pdoDestination)\n         -\u003esetFieldsToMigrate(['id', 'email', 'name'])\n         -\u003esetKeyFields(['id'])\n         -\u003eaddTransformer(new AddRandomNumberTransformer())\n         -\u003ewithProgressBar()\n         -\u003emigrate();\n```\n\nThis migration will add a random number into a field called `random_number` for each row of data. \nThis will then be migrated to the destination database along with the other fields.\n\n#### Removing data items\n\nThis example demonstrates how data items can be removed from a data row. \nYou may wish to do this if you want to use its value, but not actually \nmigrate it to the destination.\n\n```php\nclass EmailToHashTransformer implements TransformerInterface\n{\n    public function transform(DataRow $dataRow): void\n    {\n        $emailDataItem = $dataRow-\u003egetDataItemByFieldName('email');\n        $dataRow-\u003eaddDataItem(new DataItem('email_hash', md5($emailDataItem-\u003evalue)));\n        $dataRow-\u003eremoveDataItem($emailDataItem);\n    }\n}\n\n$migrator = new Migrator;\n$migrator-\u003esetSource($pdoSource)\n         -\u003esetDestination($pdoDestination)\n         -\u003esetFieldsToMigrate(['id', 'email', 'name'])\n         -\u003esetKeyFields(['id'])\n         -\u003eaddTransformer(new EmailToHashTransformer())\n         -\u003ewithProgressBar()\n         -\u003emigrate();\n```\n\nThis migration gets the data from the `email` field in the source, creates a \nnew `email_hash` data item which contains an md5 of the email address, and then \nremoves the original `email` data item. This new `email_hash` will then be \nmigrated to the destination database along with the other fields, excluding \nthe removed `email` field.\n","funding_links":[],"categories":["PHP"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDivineOmega%2Fuxdm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FDivineOmega%2Fuxdm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDivineOmega%2Fuxdm/lists"}