{"id":16812905,"url":"https://github.com/sjmudd/iodku","last_synced_at":"2025-03-17T11:21:22.703Z","repository":{"id":57747911,"uuid":"465203897","full_name":"sjmudd/iodku","owner":"sjmudd","description":"Testing MySQL Insert Latency, intended for Group Replication","archived":false,"fork":false,"pushed_at":"2022-09-06T00:15:15.000Z","size":20,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-23T20:48:17.024Z","etag":null,"topics":["go","golang","inserts","latency","performance"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sjmudd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-03-02T07:38:44.000Z","updated_at":"2022-09-05T22:14:18.000Z","dependencies_parsed_at":"2022-09-09T05:10:56.895Z","dependency_job_id":null,"html_url":"https://github.com/sjmudd/iodku","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjmudd%2Fiodku","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjmudd%2Fiodku/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjmudd%2Fiodku/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjmudd%2Fiodku/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sjmudd","download_url":"https://codeload.github.com/sjmudd/iodku/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244022650,"owners_count":20385134,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","inserts","latency","performance"],"created_at":"2024-10-13T10:23:56.185Z","updated_at":"2025-03-17T11:21:22.678Z","avatar_url":"https://github.com/sjmudd.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# iodku\n\nTesting inserts to a table and the time it takes.\n\nRelated to: https://bugs.mysql.com/bug.php?id=106526\n\n## Table definition\n\nRequired table definition\n\n```\nCREATE TABLE `heartbeat` (\n  `id` int NOT NULL,\n  `master_ts` timestamp(6) NULL DEFAULT NULL,\n  `master_csec` bigint DEFAULT NULL,\n  `update_by` varchar(100) DEFAULT NULL,\n  `master_id` int DEFAULT NULL,\n  PRIMARY KEY (`id`)\n) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci\n```\n\n## Environment variables:\n\nYou can use `MYSQL_DSN=\u003cuser\u003e:\u003cpass\u003e@tcp(\u003chost\u003e:\u003cport\u003e)/\u003cdb\u003e` as the `MYSQL_DSN`\nto configure how `iodku` will connect to the server.\n\n## Command line options\n\n* `--insert-interval=XXX` is used to determine the insert interval. Default is `1s`. See: https://github.com/go-sql-driver/mysql#dsn-data-source-name for the exact format.\n* `--count=xx` number of inserts to perform, default is -1 (insert forever)\n* `--summary` provide summary statistics of the inserts which were performed.\n\n## Sample output\n\nNormal sample output might be:\n\n```\n$ MYSQL_DSN='user_test:user_pass@tcp(myhost:3306)/iodku' ./iodku --insert-interval=1s --count=10 --summary\n2022/09/06 01:19:18.999876 OK: took: 22.752079ms\n2022/09/06 01:19:20.020215 OK: took: 19.926164ms\n2022/09/06 01:19:21.040269 OK: took: 19.734472ms\n2022/09/06 01:19:22.060740 OK: took: 20.249455ms\n2022/09/06 01:19:23.080779 OK: took: 19.707931ms\n2022/09/06 01:19:24.100869 OK: took: 19.885405ms\n2022/09/06 01:19:25.109734 OK: took: 8.583185ms\n2022/09/06 01:19:26.118434 OK: took: 8.578377ms\n2022/09/06 01:19:27.127168 OK: took: 8.646657ms\n2022/09/06 01:19:28.141451 OK: took: 14.213081ms\n2022/09/06 01:19:29.141781 Summary:\n2022/09/06 01:19:29.141821 - interval:   10.164657709s\n2022/09/06 01:19:29.141826 - attempts:   10\n2022/09/06 01:19:29.141833 - successful: 10, 100.00%\n2022/09/06 01:19:29.141837 - min:        8.578377ms\n2022/09/06 01:19:29.141841 - average:    16.22768ms\n2022/09/06 01:19:29.141844 - max:        22.752079ms\n$\n```\n\nA sample output with errors might look like this:\n```\n$ MYSQL_DSN='user_test:user_pass@tcp(myhost:3306)/iodku' ./iodku --insert-interval=1s --count=10 --summary --max-wait=19ms\n2022/09/06 01:51:11.398199 OK: took: 9.171335ms\n2022/09/06 01:51:12.407491 OK: took: 8.341306ms\n2022/09/06 01:51:13.421910 OK: took: 14.060208ms\n2022/09/06 01:51:14.436470 OK: took: 14.200061ms\n2022/09/06 01:51:15.456453 Error after 19.323708ms: context deadline exceeded\n2022/09/06 01:51:16.476552 Error after 19.960234ms: context deadline exceeded\n2022/09/06 01:51:17.497449 Error after 19.847565ms: context deadline exceeded\n2022/09/06 01:51:18.517580 Error after 20.011119ms: context deadline exceeded\n2022/09/06 01:51:19.537625 Error after 19.952122ms: context deadline exceeded\n2022/09/06 01:51:20.558140 Error after 20.01018ms: context deadline exceeded\n2022/09/06 01:51:21.559110 Summary:\n2022/09/06 01:51:21.559172 - interval:   10.170091554s\n2022/09/06 01:51:21.559188 - attempts:   10\n2022/09/06 01:51:21.559197 - successful: 4,  40.00%\n2022/09/06 01:51:21.559201 - min:        8.341306ms\n2022/09/06 01:51:21.559206 - average:    16.487783ms\n2022/09/06 01:51:21.559209 - max:        20.011119ms\n$\n```\n\n## TODO List\n\nBelow is a list of some additional changes which could be made to make the usage more helpful.\n\n* add an optional \"ping time checker\" starting at the same time as the insert\n  * this allows us to verify network latency/jitter to the server without doing any transactional work.\n  * use separate go routines starting in parallel and using `sync.WaitGroup` to wait for both to finish before continuing.\n* add an option to have multiple parallel inserters\n* add an option to be able to provide a pre-configured sized payload when performing the insert\n* for group replication, collect GR group members and status and if the primary fails change to use the failed over primary instead of the failed primary being talked to.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsjmudd%2Fiodku","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsjmudd%2Fiodku","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsjmudd%2Fiodku/lists"}