{"id":13669353,"url":"https://github.com/mrjgreen/db-sync","last_synced_at":"2025-12-27T05:09:47.235Z","repository":{"id":14355607,"uuid":"17065273","full_name":"mrjgreen/db-sync","owner":"mrjgreen","description":"PHP library with command line tool for efficiently syncing tables between remote MySQL databases","archived":false,"fork":false,"pushed_at":"2018-08-05T16:40:22.000Z","size":2285,"stargazers_count":293,"open_issues_count":11,"forks_count":90,"subscribers_count":34,"default_branch":"v3","last_synced_at":"2025-10-18T07:05:08.516Z","etag":null,"topics":["database","mysql","mysql-backup","mysql-replication","sync-data"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mrjgreen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-02-21T18:15:04.000Z","updated_at":"2025-10-07T04:29:14.000Z","dependencies_parsed_at":"2022-08-30T02:01:54.530Z","dependency_job_id":null,"html_url":"https://github.com/mrjgreen/db-sync","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/mrjgreen/db-sync","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrjgreen%2Fdb-sync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrjgreen%2Fdb-sync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrjgreen%2Fdb-sync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrjgreen%2Fdb-sync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mrjgreen","download_url":"https://codeload.github.com/mrjgreen/db-sync/tar.gz/refs/heads/v3","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mrjgreen%2Fdb-sync/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28072753,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-27T02:00:05.897Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","mysql","mysql-backup","mysql-replication","sync-data"],"created_at":"2024-08-02T08:01:10.843Z","updated_at":"2025-12-27T05:09:47.212Z","avatar_url":"https://github.com/mrjgreen.png","language":"PHP","funding_links":[],"categories":["PHP"],"sub_categories":[],"readme":"DbSync\n=======\n\n[![Build Status](https://travis-ci.org/mrjgreen/db-sync.svg?branch=v3)](https://travis-ci.org/mrjgreen/db-sync)\n[![Coverage Status](https://coveralls.io/repos/mrjgreen/db-sync/badge.svg?branch=v3\u0026service=github)](https://coveralls.io/github/mrjgreen/db-sync?branch=v3)\n[![Latest Stable Version](https://poser.pugx.org/mrjgreen/db-sync/v/stable)](https://packagist.org/packages/mrjgreen/db-sync)\n[![License](https://poser.pugx.org/mrjgreen/db-sync/license)](https://packagist.org/packages/mrjgreen/db-sync)\n[![Total Downloads](https://poser.pugx.org/mrjgreen/db-sync/downloads)](https://packagist.org/packages/mrjgreen/db-sync)\n[![SensioLabsInsight](https://insight.sensiolabs.com/projects/34585f74-7436-47c0-8b40-41265ef5a1ba/mini.png)](https://insight.sensiolabs.com/projects/34585f74-7436-47c0-8b40-41265ef5a1ba)\n\n### WARNING - This package modifies database tables. Use with extreme caution and back up databases before running.\n\n#### Always perform a dry run (this is the default action) first before specifying the `--execute (-e)` option.\n\nWhat is it?\n-----------\n\nDbSync is a tool for efficiently comparing and synchronising two or more remote MySQL database tables.\n\nIn order to do this without comparing every byte of data, the tool preforms a checksum (MD5, SHA1, CRC32) \nover a range of rows on both the source and destination tables, and compares only the hash. If a block is found to have \nan inconsistency, the tool performs a checksum on each half of the block, recursively (down to a minimum \nblock transfer size), until it finds the inconsistency.\n\n\nNotes About Deletion\n--------------------\nDbSync will only delete rows from the destination that no longer exist on the source when the `--delete` option is specified. \nUse this option with extreme caution. Always perform a dry run first.\n\nIf you use DbSync to synchronise a table which has row deletions on the source without using the `--delete` option, \nDbSync will find inconsistencies in any block with a deleted row on every run but will not be able to remove the rows from the target.\n\n\nInstallation\n------------\n\nVia composer - run the following command in your project directory:\n\n    composer require mrjgreen/db-sync\n\nOr use the packaged archive directly\n\n    wget https://github.com/mrjgreen/db-sync/raw/v3/db-sync.phar -O db-sync.phar\n    chmod a+x db-sync.phar\n\nOptionally make the command available globally\n\n    sudo mv db-sync.phar /usr/bin/db-sync\n\n```\nUsage:\ndb-sync [options] \u003csource\u003e \u003ctarget\u003e \u003ctable\u003e\n\nSync a mysql database table from one host to another using an efficient checksum algorithm to find differences.\n\nArguments:\n  source                                     The source host ip to use.\n  target                                     The target host ip to use.\n  table                                      The fully qualified database table to sync.\n\nOptions:\n  -b, --block-size=BLOCK-SIZE                The maximum block to use for when comparing. [default: 1024]\n      --charset=CHARSET                      The charset to use for database connections. [default: \"utf8\"]\n  -c, --columns=COLUMNS                      Columns to sync - all columns not \"ignored\" will be included....\n  -C, --comparison=COMPARISON                Columns from the list of synced columns to use to create the...\n  -f, --config-file=CONFIG-FILE              A path to a config.ini file from which to read values. [default: \"dbsync.ini\"]\n      --delete                               Remove rows from the target table that do not exist in the source.\n  -e, --execute                              Perform the data write on non-matching blocks.\n  -h, --help                                 Show this usage information.\n  -H, --hash                                 Specify the hash algorithm used to generate the comparison hash. [default: \"md5\"]\n  -i, --ignore-columns=IGNORE-COLUMNS        Columns to ignore. Will not be copied or used to create the hash....\n  -I, --ignore-comparison=IGNORE-COMPARISON  Columns to ignore from the hash. Columns will still be copied....\n  -p, --password[=PASSWORD]                  The password for the specified user. Will be solicited on the tty if...\n  -u, --user=USER                            The name of the user to connect with. [default: \"USER\"]\n  -s, --transfer-size=TRANSFER-SIZE          The maximum copy size to use for when comparing. [default: 8]\n      --target.user=TARGET.USER              The name of the user to connect to the target host with if different...\n      --target.table=TARGET.TABLE            The name of the table on the target host if different to the source.\n      --target.password=TARGET.PASSWORD      The password for the target host if the target user is specified....\n      --where=WHERE                          A where clause to apply to the tables.\n  -v, --verbose                              Enable verbose output.\n  -q, --quiet                                Disable output, overrides \"verbose\" option.\n```\n\n### Examples\n\n*Note - All of these commands will perform a \"dry-run\" only. To execute the insert/update statement against the target\ndatabase, you must specify the --execute (-e) option*\n\n##### Example 1\n\nSync the table `web.customers` from one host to another (non-standard port on target):\n\n```\ndb-sync --user root --password mypass 127.0.0.1 111.222.3.44:13306 web.customers\n```\n\n##### Example 2\n\nSync the table `web.customers` from one host to another, deleting rows from the target that no longer exist on the source (using SHA1 hash for comparison):\n```\ndb-sync --user root --password mypass --hash sha1 --delete 127.0.0.1 111.222.3.44 web.customers\n```\n\n##### Example 3\n\nSync the table `web.customers` from one host to another using different credentials:\n\n```\ndb-sync --user root --password mypass --target.user admin --target.password password 127.0.0.1 111.222.3.44 web.customers:\n```\n\n##### Example 4\n\nSync only the `email` and `name` fields from the table `web.customers`:\n\n \u003e NB. The primary key will automatically be included in the column set\n\n```\ndb-sync --user root --password mypass 127.0.0.1 111.222.3.44 web.customers -c email -c name\n```\n\n##### Example 5\n\nSync every column except for the `updated_at` fields from the table `web.customers`:\n\n```\ndb-sync --user root --password mypass 127.0.0.1 111.222.3.44 web.customers -i updated_at\n```\n\n##### Example 6\n\nSync every column from the table `web.customers` but only use the `updated_at` fields when calculating the hash:\n\n \u003e Inconsistencies in other fields will not be detected. In the event of a hash inconsistency in fields which are \n included, the excluded fields will still be copied to the target host.\n\n```\ndb-sync --user root --password mypass 127.0.0.1 111.222.3.44 web.customers -C updated_at\n```\n\n##### Example 7\n\nSync every column from the table `web.customers` and use all fields except for the `notes` or `info` fields when calculating the hash:\n\n \u003e Inconsistencies in excluded fields will not be detected. In the event of a hash inconsistency in fields which are included, \n the excluded fields will still be copied to the target host.\n\n \u003e This is especially useful for tables with long text fields that don't change after initial insert, or which are associated\n with an `on update CURRENT_TIMESTAMP` field. For large tables this can offer a big performance boost.\n\n```\ndb-sync --user root --password mypass 127.0.0.1 111.222.3.44 web.customers -I notes -I info\n```\n\n##### Example 8\n\nSync the table `web.customers` to a table under a different name in a different database `web_backup.customers_2`:\n\n```\ndb-sync --user root --password mypass --target.table web_backup.customers_2 127.0.0.1 111.222.3.44 web.customers\n```\n\n##### Example 9\n\nSync the table `web.customers` only including active records and delete records in the target table, that are no longer active:\n\n```\ndb-sync --user root --password mypass 127.0.0.1 111.222.3.44:13306 web.customers --delete --where=\"active = 1\"\n```\n\nConfig File\n-----------\n\nTo avoid having to specify options repeatedly, and to avoid exposing your password on the tty you can specify a config file.\nBy default DbSync will look for a file named `dbsync.ini` in the current working directory.\n\nExample:\n\n```ini\nuser=root\npassword=mypass\ntarget.user=admin\ntarget.password=myadminpass\n```\n\nUse library within project (non-commandline)\n--------------------------------------------\n\nYou can include the library within your project and use the components directly:\n\n```PHP\nuse \\DbSync\\DbSync;\nuse \\DbSync\\Transfer\\Transfer;\nuse \\DbSync\\Hash\\ShaHash;\nuse \\DbSync\\Table;\nuse \\DbSync\\ColumnConfiguration;\n\n$sync = new DbSync(new Transfer(new ShaHash(), $blockSize, $transferSize));\n\n$sync-\u003esetLogger(new YourPsrLogger());\n\n$sync-\u003edryRun(false);\n\n$sync-\u003edelete(true);\n\n$sourceTable = new Table($sourceConnection, $sourceDb, $sourceTable);\n$targetTable = new Table($targetConnection, $targetDb, $targetTable);\n\n// if you only want specific columns \n$columnConfig = new ColumnConfiguration($syncColumns, $ignoreColumns);\n\n// if you only want to use specific columns for the comparison\n$compareConfig = new ColumnConfiguration($compareColumns, $ignoreCompareColumns);\n\n// optionally apply a where clause - this can be useful when sync-ing large tables, where\n// you can make use of a column to rule out large portions of the data\n// that you know haven't changed, such as columns with \"on update CURRENT_TIMESTAMP\" etc..\n$sourceTable-\u003esetWhereClause(new WhereClause(\"column_name = ?\", ['value']));\n$targetTable-\u003esetWhereClause(new WhereClause(\"column_name \u003e ?\", ['value']));\n\n$sync-\u003esync($sourceTable, $targetTable, $columnConfig, $compareConfig);\n```\n\nRoadmap\n-------\n\n * [x] 100% test coverage via full stack integration tests\n * [x] Allow option to delete data from target where not present on the source\n * [x] Use symfony console command for sync\n * [ ] Option to re-try with back-off on lock wait timeouts\n * [ ] Option to create missing tables on target\n * [ ] Option to skip duplicate key errors\n * [ ] Speed up initial sync of empty table - Maybe offer combination with other tool for full fast outfile based replacement\n\nRequirements\n------------\n\nPHP 5.4 or above\nPDO MySQL Extension\n\nLicense\n-------\n\nDbSync is licensed under the MIT License - see the LICENSE file for details\n\nAcknowledgments\n---------------\n\n- Inspiration for this project came from the Percona Tools `pt-table-sync`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrjgreen%2Fdb-sync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmrjgreen%2Fdb-sync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmrjgreen%2Fdb-sync/lists"}