{"id":14983691,"url":"https://github.com/jackd248/db-sync-tool","last_synced_at":"2025-04-09T15:06:09.405Z","repository":{"id":37709022,"uuid":"255380193","full_name":"jackd248/db-sync-tool","owner":"jackd248","description":"Python script to synchronize a database from an origin to a target system.","archived":false,"fork":false,"pushed_at":"2025-04-07T11:09:08.000Z","size":1173,"stargazers_count":28,"open_issues_count":5,"forks_count":6,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-09T15:06:04.410Z","etag":null,"topics":["database-sync","database-synchronization","python","python3","symfony3","symfony4","typo3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jackd248.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-13T16:22:51.000Z","updated_at":"2025-04-07T11:09:12.000Z","dependencies_parsed_at":"2024-09-24T15:25:24.336Z","dependency_job_id":"4d6c1b8d-20a3-48ee-848c-9eb19102536d","html_url":"https://github.com/jackd248/db-sync-tool","commit_stats":{"total_commits":306,"total_committers":4,"mean_commits":76.5,"dds":"0.016339869281045805","last_synced_commit":"6fa9b118685714f10cbf91f173fd27974a788615"},"previous_names":[],"tags_count":84,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jackd248%2Fdb-sync-tool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jackd248%2Fdb-sync-tool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jackd248%2Fdb-sync-tool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jackd248%2Fdb-sync-tool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jackd248","download_url":"https://codeload.github.com/jackd248/db-sync-tool/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248055284,"owners_count":21040157,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database-sync","database-synchronization","python","python3","symfony3","symfony4","typo3"],"created_at":"2024-09-24T14:07:47.266Z","updated_at":"2025-04-09T15:06:09.398Z","avatar_url":"https://github.com/jackd248.png","language":"Python","readme":"# db sync tool\n\n![PyPI - Python Version](https://img.shields.io/pypi/pyversions/db_sync_tool-kmi)\n![PyPI](https://img.shields.io/pypi/v/db_sync_tool-kmi)\n![Pepy Total Downloads](https://img.shields.io/pepy/dt/db-sync-tool-kmi)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/db-sync-tool-kmi)\n[![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/jackd248/db-sync-tool/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/jackd248/db-sync-tool/?branch=master)\n[![Build Status](https://scrutinizer-ci.com/g/jackd248/db-sync-tool/badges/build.png?b=master)](https://scrutinizer-ci.com/g/jackd248/db-sync-tool/build-status/master)\n\nPython script to synchronize a database from an origin to a target system with automatic database credential extraction depending on the selected framework.\n\n## Features\n\n- __Database sync__ from and to a remote system\n  - [MySQL](https://www.mysql.com/) (\u003e= 5.5)\n  - [MariaDB](https://mariadb.org/) (\u003e= 10.0)\n- __Proxy mode__ between two remote systems\n- Several [synchronisation modes](docs/MODE.md)\n- Automatic database __credential extraction__ using a supported framework\n    - [TYPO3](https://typo3.org/) (\u003e= v7.6)\n    - [Symfony](https://symfony.com/) (\u003e= v2.8)\n    - [Drupal](https://www.drupal.org/) (\u003e= v8.0)\n    - [Wordpress](https://wordpress.org) (\u003e= v5.0)\n    - [Laravel](https://laravel.com/) (\u003e= v4.0)\n- Easily dump creation (database __backup__)\n- __Cleanup__ feature for backups\n- Extended __logging__ capabilities\n- Many more possibilities for [customization](docs/CONFIG.md)\n\n## Installation\n\n### Prerequisite\n\nThe script needs [python](https://python.org/) __3.5__ or higher. It is necessary for some additional functionalities to have [pip](https://pypi.org/project/pip/) installed on your local machine. \n\n\u003ca name=\"install-pip\"\u003e\u003c/a\u003e\n### pip\nThe library can be installed from [PyPI](https://pypi.org/project/db-sync-tool-kmi/):\n```bash\n$ pip3 install db-sync-tool-kmi\n```\n\n\u003ca name=\"install-composer\"\u003e\u003c/a\u003e\n### composer\nWhile using the script within the PHP framework context, the script is available via [packagist.org](https://packagist.org/packages/kmi/db-sync-tool) using composer:\n\n```bash\n$ composer require kmi/db-sync-tool\n```\n\nAdditionally install the python requirements via the following pip command:\n\n````bash\n$ pip3 install -e vendor/kmi/db-sync-tool/\n````\n\n## Quickstart\n\nDetailed instructions for:\n\n- [Manual database sync](docs/quickstart/START.md)\n- [TYPO3 database sync](docs/quickstart/TYPO3.md)\n- [Symfony database sync](docs/quickstart/SYMFONY.md)\n- [Drupal database sync](docs/quickstart/DRUPAL.md)\n- [Wordpress database sync](docs/quickstart/WORDPRESS.md)\n\nIf you want to have an inside in more configuration examples, see the [test scenarios](tests/scenario). \n\n## Usage\n\n### Command line\n\nRun the python script via command line.\n\nInstalled via [pip](#install-pip):\n```bash\n$ db_sync_tool\n```\n\nInstalled via [composer](#install-composer):\n```bash\n$ python3 vendor/kmi/db-sync-tool/db_sync_tool\n```\n\n![Example receiver](docs/images/db-sync-tool-example-receiver.gif)\n\n\u003ca name=\"shell-arguments\"\u003e\u003c/a\u003e\n#### Shell arguments\n\n```bash\nusage: db_sync_tool [-h] [-f CONFIG_FILE] [-v] [-y] [-m] [-dr] [-i IMPORT_FILE] [-dn DUMP_NAME] [-kd KEEP_DUMP] [-o HOST_FILE] [-l LOG_FILE] [-cd] [-ta TABLES] [-r] [-t TYPE] [-tp TARGET_PATH]\n                    [-tn TARGET_NAME] [-th TARGET_HOST] [-tu TARGET_USER] [-tpw TARGET_PASSWORD] [-tk TARGET_KEY] [-tpo TARGET_PORT] [-tdd TARGET_DUMP_DIR] [-tkd TARGET_KEEP_DUMPS] [-tdn TARGET_DB_NAME]\n                    [-tdh TARGET_DB_HOST] [-tdu TARGET_DB_USER] [-tdpw TARGET_DB_PASSWORD] [-tdpo TARGET_DB_PORT] [-tad TARGET_AFTER_DUMP] [-op ORIGIN_PATH] [-on ORIGIN_NAME] [-oh ORIGIN_HOST]\n                    [-ou ORIGIN_USER] [-opw ORIGIN_PASSWORD] [-ok ORIGIN_KEY] [-opo ORIGIN_PORT] [-odd ORIGIN_DUMP_DIR] [-okd ORIGIN_KEEP_DUMPS] [-odn ORIGIN_DB_NAME] [-odh ORIGIN_DB_HOST]\n                    [-odu ORIGIN_DB_USER] [-odpw ORIGIN_DB_PASSWORD] [-odpo ORIGIN_DB_PORT] [-fpw] [-ur] [-uro USE_RSYNC_OPTIONS]\n                    [origin] [target]\n\nA tool for automatic database synchronization from and to host systems.\n\npositional arguments:\n  origin                Origin database defined in host file\n  target                Target database defined in host file\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -f CONFIG_FILE, --config-file CONFIG_FILE\n                        Path to configuration file\n  -v, --verbose         Enable extended console output\n  -y, --yes             Skipping user confirmation for database import\n  -m, --mute            Mute console output\n  -dr, --dry-run        Testing process without running database export, transfer or import.\n  -i IMPORT_FILE, --import-file IMPORT_FILE\n                        Import database from a specific file dump\n  -dn DUMP_NAME, --dump-name DUMP_NAME\n                        Set a specific dump file name (default is \"_[dbname]_[date]\")\n  -kd KEEP_DUMP, --keep-dump KEEP_DUMP\n                        Skipping target import of the database dump and saving the available dump file in the given directory\n  -o HOST_FILE, --host-file HOST_FILE\n                        Using an additional hosts file for merging hosts information with the configuration file\n  -l LOG_FILE, --log-file LOG_FILE\n                        File path for creating a additional log file\n  -cd, --clear-database\n                        Dropping all tables before importing a new sync to get a clean database.\n  -ta TABLES, --tables TABLES\n                        Defining specific tables to export, e.g. --tables=table1,table2\n  -r, --reverse         Reverse origin and target hosts\n  -t TYPE, --type TYPE  Defining the framework type [TYPO3, Symfony, Drupal, Wordpress]\n  -tp TARGET_PATH, --target-path TARGET_PATH\n                        File path to target database credential file depending on the framework type\n  -tn TARGET_NAME, --target-name TARGET_NAME\n                        Providing a name for the target system\n  -th TARGET_HOST, --target-host TARGET_HOST\n                        SSH host to target system\n  -tu TARGET_USER, --target-user TARGET_USER\n                        SSH user for target system\n  -tpw TARGET_PASSWORD, --target-password TARGET_PASSWORD\n                        SSH password for target system\n  -tk TARGET_KEY, --target-key TARGET_KEY\n                        File path to SSH key for target system\n  -tpo TARGET_PORT, --target-port TARGET_PORT\n                        SSH port for target system\n  -tdd TARGET_DUMP_DIR, --target-dump-dir TARGET_DUMP_DIR\n                        Directory path for database dump file on target system\n  -tkd TARGET_KEEP_DUMPS, --target-keep-dumps TARGET_KEEP_DUMPS\n                        Keep dump file count for target system\n  -tdn TARGET_DB_NAME, --target-db-name TARGET_DB_NAME\n                        Database name for target system\n  -tdh TARGET_DB_HOST, --target-db-host TARGET_DB_HOST\n                        Database host for target system\n  -tdu TARGET_DB_USER, --target-db-user TARGET_DB_USER\n                        Database user for target system\n  -tdpw TARGET_DB_PASSWORD, --target-db-password TARGET_DB_PASSWORD\n                        Database password for target system\n  -tdpo TARGET_DB_PORT, --target-db-port TARGET_DB_PORT\n                        Database port for target system\n  -tad TARGET_AFTER_DUMP, --target-after-dump TARGET_AFTER_DUMP\n                        Additional dump file to insert after the regular database import\n  -op ORIGIN_PATH, --origin-path ORIGIN_PATH\n                        File path to origin database credential file depending on the framework type\n  -on ORIGIN_NAME, --origin-name ORIGIN_NAME\n                        Providing a name for the origin system\n  -oh ORIGIN_HOST, --origin-host ORIGIN_HOST\n                        SSH host to origin system\n  -ou ORIGIN_USER, --origin-user ORIGIN_USER\n                        SSH user for origin system\n  -opw ORIGIN_PASSWORD, --origin-password ORIGIN_PASSWORD\n                        SSH password for origin system\n  -ok ORIGIN_KEY, --origin-key ORIGIN_KEY\n                        File path to SSH key for origin system\n  -opo ORIGIN_PORT, --origin-port ORIGIN_PORT\n                        SSH port for origin system\n  -odd ORIGIN_DUMP_DIR, --origin-dump-dir ORIGIN_DUMP_DIR\n                        Directory path for database dump file on origin system\n  -okd ORIGIN_KEEP_DUMPS, --origin-keep-dumps ORIGIN_KEEP_DUMPS\n                        Keep dump file count for origin system\n  -odn ORIGIN_DB_NAME, --origin-db-name ORIGIN_DB_NAME\n                        Database name for origin system\n  -odh ORIGIN_DB_HOST, --origin-db-host ORIGIN_DB_HOST\n                        Database host for origin system\n  -odu ORIGIN_DB_USER, --origin-db-user ORIGIN_DB_USER\n                        Database user for origin system\n  -odpw ORIGIN_DB_PASSWORD, --origin-db-password ORIGIN_DB_PASSWORD\n                        Database password for origin system\n  -odpo ORIGIN_DB_PORT, --origin-db-port ORIGIN_DB_PORT\n                        Database port for origin system\n  -fpw, --force-password\n                        Force password user query\n  -ur, --use-rsync      Use rsync as transfer method\n  -uro USE_RSYNC_OPTIONS, --use-rsync-options USE_RSYNC_OPTIONS\n                        Additional rsync options\n  -w WHERE, --where WHERE\n                        Additional where clause for mysql dump to sync only selected rows, e.g. --where=\"deleted=0\"\n  -amo OPTIONS, --additional-mysqldump-options OPTIONS\n                        Additional mysqldump options for creating the database dump, e.g. --additional-mysqldump-options=\"--where=\"deleted=0\"\n```\n\nIf you haven't declared a path to a SSH key, during the script execution you are requested to enter the SSH password for the given user in the shell argument or the `config.json` to enable a SSH connection for the remote system. \n\n### Import\n\nYou can import the python package and use them inside your project:\n\n```python\nfrom db_sync_tool import sync\n\nif __name__ == \"__main__\":\n    sync.Sync(config={}, args*)\n```\n\n## Configuration\n\nYou can configure the script with [shell arguments](#shell-arguments) or using a separate configuration file.\n\n### Configuration File\n\nThe `config.json` contains important information about the origin and the target system. In dependence on the given configuration the [synchronisation mode](docs/MODE.md) is implicitly selected.\n\nExample structure of a `config.yml` for a Symfony system in receiver mode (`path` defines the location of the Symfony database configuration file):\n```yaml\ntype: Symfony\norigin:\n    host: 192.87.33.123\n    user: ssh_demo_user\n    path: /var/www/html/project/shared/.env\ntarget:\n    path: /var/www/html/app/.env\n```\n\nIt is possible to adjust the `config.yml` [configuration](docs/CONFIG.md).\n\n## File sync\n\nThere is an addon script available to sync files to. Use the [file-sync-tool](https://github.com/jackd248/file-sync-tool) to easily transfer files between origin and target system. \n\n## Release Guide\n\nA detailed guide is available to release a new version. See [here](docs/RELEASE.md).\n\n## Tests\n\nA docker container set up is available for testing purpose. See [here](tests/README.md).\n\n## Support\n\nIf you like the project, feel free to support the development.\n\n\u003ca href=\"https://www.buymeacoffee.com/konradmichalik\" target=\"_blank\"\u003e\u003cimg src=\"https://cdn.buymeacoffee.com/buttons/default-green.png\" alt=\"Buy Me A Coffee\" height=\"41\" width=\"174\"\u003e\u003c/a\u003e\n","funding_links":["https://www.buymeacoffee.com/konradmichalik"],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjackd248%2Fdb-sync-tool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjackd248%2Fdb-sync-tool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjackd248%2Fdb-sync-tool/lists"}