{"id":27088963,"url":"https://github.com/devops-works/mygopherhose","last_synced_at":"2025-04-06T06:36:50.296Z","repository":{"id":64303068,"uuid":"420409135","full_name":"devops-works/mygopherhose","owner":"devops-works","description":"Faster import for mysqldumps","archived":false,"fork":false,"pushed_at":"2021-10-23T13:37:09.000Z","size":13,"stargazers_count":3,"open_issues_count":3,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-06-20T05:19:43.057Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devops-works.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-23T12:41:17.000Z","updated_at":"2022-02-18T23:56:35.000Z","dependencies_parsed_at":"2023-01-15T09:45:36.613Z","dependency_job_id":null,"html_url":"https://github.com/devops-works/mygopherhose","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devops-works%2Fmygopherhose","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devops-works%2Fmygopherhose/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devops-works%2Fmygopherhose/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devops-works%2Fmygopherhose/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devops-works","download_url":"https://codeload.github.com/devops-works/mygopherhose/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247445686,"owners_count":20939953,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-06T06:36:47.719Z","updated_at":"2025-04-06T06:36:50.291Z","avatar_url":"https://github.com/devops-works.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mygopherhose\n\nParallel importer for mysqldumps.\n\n## What\n\nmygopherhose uses a dump produced by `mysqldump` and imports it trying to\nparallelize INSERT statements.\n\nFrom [benchmarks](#gcp-benchmarks), it seems to perform 3x faster on small\ninstances (should be even better on high end machines).\n\n## Caveats\n\n- tables are not locked\n- does not support SETting things\n- does not support stored procedures\n\n**Use at your own risks in production.**\n\n## Usage\n\n```\nmygopherhose [-h host] -u user -p [password] [-P port] [-d dbname] [-b bufsize] dumpfile\n        -h defaults to 127.0.0.1\n        -P defaults to 3306\n        -b defaults to 10485760 bytes\n        -d can be omitted is dump contains `USE DATABASE foo;` stanza\n        -p if parameter is empty, password will be asked interactively\n```\n\n- 🐑 is displayed at each `CREATE TABLE` operation.\n- ⌛ is diplayed when all transactions for the current table have been sent to\n  the buffered channel for processing\n- a dot is displayed for each `INSERT` statement applied (those statements\n  almost always contains several rows, see\n  [`net_buffer_length`](https://dev.mysql.com/doc/refman/5.7/en/mysqldump.html#option_mysqldump_net-buffer-length))\n\n## GCP benchmarks\n\n- Client VM: n2-standard-2\n- CloudSQL: db-n1-standard-8 / 260GB\n- Dump: \n  - 20GB\n  - 95 tables\n  - 68781981 rows\n  - data size 23.04G\n  - index size 2.48 G\n\n| description         |   duration | db CPU |\n| ------------------- | ---------: | -----: |\n| mysql cli piped     | 30m24.209s |   ~11% |\n| mysql -e source     | 30m30.161s |   ~11% |\n| mygopherhose -c 10  | 13m02.016s | 54-87% |\n| mygopherhose -c 20  | 12m51.301s | 60-90% |\n| mygopherhose -c 40  | 12m51.799s | 60-93% |\n| mygopherhose -c 100 | 13m09.446s | 69-97% |\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevops-works%2Fmygopherhose","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevops-works%2Fmygopherhose","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevops-works%2Fmygopherhose/lists"}