{"id":19268594,"url":"https://github.com/moolen/vitess_playground","last_synced_at":"2026-06-18T17:32:22.627Z","repository":{"id":146316805,"uuid":"153770095","full_name":"moolen/vitess_playground","owner":"moolen","description":null,"archived":false,"fork":false,"pushed_at":"2018-10-19T11:07:38.000Z","size":7,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-23T19:46:26.325Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moolen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-10-19T11:04:47.000Z","updated_at":"2019-04-08T18:08:25.000Z","dependencies_parsed_at":"2024-06-13T04:30:15.110Z","dependency_job_id":null,"html_url":"https://github.com/moolen/vitess_playground","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/moolen/vitess_playground","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moolen%2Fvitess_playground","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moolen%2Fvitess_playground/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moolen%2Fvitess_playground/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moolen%2Fvitess_playground/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moolen","download_url":"https://codeload.github.com/moolen/vitess_playground/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moolen%2Fvitess_playground/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34501473,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-18T02:00:06.871Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T20:16:46.555Z","updated_at":"2026-06-18T17:32:22.613Z","avatar_url":"https://github.com/moolen.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# vitess migration\n\nAs of now, there is no vitess documentation on how to do a hot-migration. This is a PoC that does that. We use innobackupex to create a backup. This is not (yet) officially supported by vitess.\n\n## Concept\n\n* On the donor host: Create a backup using `innobackupex`.\n* stream/copy that backup to the vitess master\n* restore the backup (shutdown mysqld)\n* apply vitess `init_db.sql` dump\n* start replication\n\nWe do the same for the `vitess-master` to `vitess-replica` migration.\n\nOnce we're done with that vitess will take over the management and we should be able to use all of vitess' functionallity. (only Backups are tested at the moment).\n\n## Set up\n\nThe following step-by-step guide is derived from the official the [vitess guide](https://github.com/vitessio/vitess/tree/master/examples/compose)\n\n```\n\n$ docker-compose up -d\n\n# once consul \u0026 vtctld is ready\n$ ./lvtctl.sh ApplySchema -sql \"$(cat create_test_table.sql)\" test_keyspace\n\n# start vtgate\n$ docker-compose up -d\n\n$ ./migrate.sh from 1\n\n# test if mysql came up properly and data is present\n$ docker-compose exec vttablet1 mysql -h 127.0.0.1 -uroot vt_test_keyspace -e \"select * from messages\"\n\u003e +------+-----------------+---------+\n\u003e | page | time_created_ns | message |\n\u003e +------+-----------------+---------+\n\u003e |    1 |  20181019085053 | baz     |\n\u003e +------+-----------------+---------+\n\n# this should yield \"Slave IO/SQL Running: Yes\"\n$ ./replicate.sh 1 from root root\n\n# insert to \"from\" database\n$ docker-compose exec from mysql -h 127.0.0.1 -uroot -proot vt_test_keyspace -e \"insert into messages VALUES (2, NOW(), 'foo')\"\n\n# check again if stuff replicated properly\n$ docker-compose exec vttablet1 mysql -h 127.0.0.1 -uroot vt_test_keyspace -e \"select * from messages\"\n\u003e +------+-----------------+---------+\n\u003e | page | time_created_ns | message |\n\u003e +------+-----------------+---------+\n\u003e |    1 |  20181019085053 | baz     |\n\u003e +------+-----------------+---------+\n\u003e |    2 |  20181019094214 | foo     |\n\u003e +------+-----------------+---------+\n\n\n\n# create replicas\n$ ./migrate.sh vttablet1 2\n$ ./migrate.sh vttablet1 3\n\n# start replication with root user / nopasswd\n$ ./replicate.sh 2 vttablet1 root ''\n$ ./replicate.sh 3 vttablet1 root ''\n\n# use vitess client script to test sharding (run it twice)\n$ ./client.sh\n$ ./client.sh\n\n$ docker-compose restart vttablet2 vttablet3\n# once the vttablet2/3 containers come up\n# vttablet will take care about the replication \n# (it uses the vt_repl user)\n\n# run client again to check repl (after a few seconds)\n./client.sh\n\n# backup data:\n# vtctl tells vtctld to take backups\n# the respective vttablet will actually do the backup\n\n$ docker-compose exec vttablet1 bash -c \"vtctl \\$TOPOLOGY_FLAGS Backup test-2\"\n$ docker-compose exec vttablet1 bash -c \"vtctl \\$TOPOLOGY_FLAGS Backup test-3\"\n\n$ docker-compose exec vttablet2 bash -c \"ls -la /vt/vtdataroot/backups/test_keyspace/0\"\n$ docker-compose exec vttablet3 bash -c \"ls -la /vt/vtdataroot/backups/test_keyspace/0\"\n\n```\n\n\n## lessons learned\n\n* Vitess uses `--binlog-format=statement` see [here](https://vitess.io/user-guide/vitess-replication/)\n* Vitess can *NOT* create a backup from a master tablet.\n* Vitess Backups require to stop the `mysqld` process!\n* Vitess uses a different directory layout for the mysql data. See `cat /vt/vtdataroot/vt_000000000${TABLET_ID}/my.cnf` for reference. Especially `tablespaces, innodb and bin/relay logs` are in a different location\n* vitess does not support innobackupex/xtrabackup yet, see [vitess/#3957](https://github.com/vitessio/vitess/issues/3957) but with this approach it will work (downside: master won't be auto-replicated after container restart)\n* vttablet takes care about the replication once the mysqld comes up. This is *ONLY* true for replicas. masters need manual action for that\n\n\n## todo\n\n* bake binaries into Vitess images (innobackupex, socat, nc..)\n* stream backup instead of copying it over using socat:\n  * donor: `nohup socat tcp-l:4565,reuseaddr,fork system:\"innobackupex --stream=xbstream --ftwrl-wait-threshold=40 --ftwrl-wait-query-type=all --ftwrl-wait-timeout=180 --kill-long-queries-timeout=20 --kill-long-query-type=all $DATADIR\"`\n  * receiver: `nc -i 10 $TARGET_HOST $TARGET_PORT | pv -b | xbstream -x -C $DATADIR`\n* get rid of `sleep N`\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoolen%2Fvitess_playground","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoolen%2Fvitess_playground","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoolen%2Fvitess_playground/lists"}