{"id":18489749,"url":"https://github.com/abicky/embulk-output-jdbc-issue-241","last_synced_at":"2025-05-13T23:34:49.827Z","repository":{"id":139218194,"uuid":"181419850","full_name":"abicky/embulk-output-jdbc-issue-241","owner":"abicky","description":null,"archived":false,"fork":false,"pushed_at":"2019-04-15T06:14:23.000Z","size":18,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-12T06:37:56.413Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/abicky.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-15T05:45:20.000Z","updated_at":"2019-04-15T06:14:25.000Z","dependencies_parsed_at":"2024-07-09T16:46:14.063Z","dependency_job_id":null,"html_url":"https://github.com/abicky/embulk-output-jdbc-issue-241","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abicky%2Fembulk-output-jdbc-issue-241","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abicky%2Fembulk-output-jdbc-issue-241/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abicky%2Fembulk-output-jdbc-issue-241/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abicky%2Fembulk-output-jdbc-issue-241/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/abicky","download_url":"https://codeload.github.com/abicky/embulk-output-jdbc-issue-241/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254043217,"owners_count":22004912,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T12:57:44.425Z","updated_at":"2025-05-13T23:34:49.752Z","avatar_url":"https://github.com/abicky.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# embulk-output-jdbc-issue-241\n\nThis repository includes files to reproduce https://github.com/embulk/embulk-output-jdbc/issues/241.\nCSV files are created by the following command:\n\n```\nfor i in {000..007}; do shuf -i 1-1000 -n 1000 \u003e csv/$i.csv; echo $i \u003e\u003e csv/$i.csv; done\n```\n\n## Steps to reproduce\n\n```\ngit clone https://github.com/abicky/embulk-output-jdbc-issue-241.git\ncd embulk-output-jdbc-issue-241\ndocker-compose up\n```\n\n## Expected Behavior\n\n``SELECT * FROM `test`.`users` WHERE `name` LIKE \"00_\";`` outputs a text like below:\n\n```\nname\n000\n001\n002\n003\n004\n005\n006\n007\n```\n\n## Actual Behavior\n\n``SELECT * FROM `test`.`users` WHERE `name` LIKE \"00_\";`` outputs a text like below:\n\n```\nname\n001\n002\n003\n006\n007\n```\n\n## Example Log\n\n```\n-- snip --\nembulk_1  | 2019-04-15 05:48:19.104 +0000 [INFO] (0020:task-0005): Prepared SQL: INSERT INTO `users` (`name`) VALUES (?) ON DUPLICATE KEY UPDATE `name` = VALUES(`name`)\nembulk_1  | 2019-04-15 05:48:19.107 +0000 [INFO] (0019:task-0004): Prepared SQL: INSERT INTO `users` (`name`) VALUES (?) ON DUPLICATE KEY UPDATE `name` = VALUES(`name`)\nembulk_1  | 2019-04-15 05:48:19.108 +0000 [INFO] (0021:task-0006): Prepared SQL: INSERT INTO `users` (`name`) VALUES (?) ON DUPLICATE KEY UPDATE `name` = VALUES(`name`)\nembulk_1  | 2019-04-15 05:48:19.108 +0000 [INFO] (0022:task-0007): Prepared SQL: INSERT INTO `users` (`name`) VALUES (?) ON DUPLICATE KEY UPDATE `name` = VALUES(`name`)\nembulk_1  | 2019-04-15 05:48:19.108 +0000 [INFO] (0018:task-0003): Prepared SQL: INSERT INTO `users` (`name`) VALUES (?) ON DUPLICATE KEY UPDATE `name` = VALUES(`name`)\nembulk_1  | 2019-04-15 05:48:19.109 +0000 [INFO] (0015:task-0000): Prepared SQL: INSERT INTO `users` (`name`) VALUES (?) ON DUPLICATE KEY UPDATE `name` = VALUES(`name`)\nembulk_1  | 2019-04-15 05:48:19.118 +0000 [INFO] (0016:task-0001): Prepared SQL: INSERT INTO `users` (`name`) VALUES (?) ON DUPLICATE KEY UPDATE `name` = VALUES(`name`)\nembulk_1  | 2019-04-15 05:48:19.126 +0000 [INFO] (0017:task-0002): Prepared SQL: INSERT INTO `users` (`name`) VALUES (?) ON DUPLICATE KEY UPDATE `name` = VALUES(`name`)\nembulk_1  | 2019-04-15 05:48:19.505 +0000 [INFO] (0022:task-0007): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:19.540 +0000 [INFO] (0018:task-0003): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:19.545 +0000 [INFO] (0021:task-0006): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:19.545 +0000 [INFO] (0019:task-0004): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:19.549 +0000 [INFO] (0017:task-0002): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:19.550 +0000 [INFO] (0016:task-0001): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:19.551 +0000 [INFO] (0020:task-0005): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:19.567 +0000 [WARN] (0020:task-0005): Operation failed (1213:40001), retrying 1/1 after 1 seconds. Message: Deadlock found when trying to get lock; try restarting transaction\nembulk_1  | 2019-04-15 05:48:19.568 +0000 [INFO] (0022:task-0007): \u003e 0.06 seconds (loaded 1,001 rows in total)\nembulk_1  | 2019-04-15 05:48:19.570 +0000 [INFO] (0017:task-0002): \u003e 0.02 seconds (loaded 1,001 rows in total)\nembulk_1  | 2019-04-15 05:48:19.570 +0000 [INFO] (0016:task-0001): \u003e 0.02 seconds (loaded 1,001 rows in total)\nembulk_1  | 2019-04-15 05:48:19.572 +0000 [INFO] (0015:task-0000): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:19.586 +0000 [WARN] (0021:task-0006): Operation failed (1213:40001), retrying 1/1 after 1 seconds. Message: Deadlock found when trying to get lock; try restarting transaction\nembulk_1  | 2019-04-15 05:48:19.587 +0000 [WARN] (0015:task-0000): Operation failed (1213:40001), retrying 1/1 after 1 seconds. Message: Deadlock found when trying to get lock; try restarting transaction\nembulk_1  | 2019-04-15 05:48:19.592 +0000 [INFO] (0018:task-0003): \u003e 0.05 seconds (loaded 1,001 rows in total)\nembulk_1  | 2019-04-15 05:48:19.592 +0000 [INFO] (0019:task-0004): \u003e 0.05 seconds (loaded 1,001 rows in total)\nmysql_1   | 2019-04-15T05:48:19.596374Z 2 [Note] Got timeout reading communication packets\nembulk_1  | 2019-04-15 05:48:20.568 +0000 [INFO] (0020:task-0005): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:20.571 +0000 [INFO] (0020:task-0005): \u003e 0.00 seconds (loaded 1,001 rows in total)\nembulk_1  | 2019-04-15 05:48:20.587 +0000 [INFO] (0021:task-0006): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:20.588 +0000 [INFO] (0015:task-0000): Loading 1,001 rows\nembulk_1  | 2019-04-15 05:48:20.589 +0000 [INFO] (0021:task-0006): \u003e 0.00 seconds (loaded 1,001 rows in total)\nembulk_1  | 2019-04-15 05:48:20.590 +0000 [INFO] (0015:task-0000): \u003e 0.00 seconds (loaded 1,001 rows in total)\nembulk_1  | 2019-04-15 05:48:20.591 +0000 [INFO] (0001:transaction): {done:  8 / 8, running: 0}\nembulk_1  | 2019-04-15 05:48:20.591 +0000 [INFO] (0001:transaction): {done:  8 / 8, running: 0}\nembulk_1  | 2019-04-15 05:48:20.592 +0000 [INFO] (0001:transaction): {done:  8 / 8, running: 0}\nembulk_1  | 2019-04-15 05:48:20.592 +0000 [INFO] (0001:transaction): {done:  8 / 8, running: 0}\nembulk_1  | 2019-04-15 05:48:20.592 +0000 [INFO] (0001:transaction): {done:  8 / 8, running: 0}\nembulk_1  | 2019-04-15 05:48:20.593 +0000 [INFO] (0001:transaction): {done:  8 / 8, running: 0}\nembulk_1  | 2019-04-15 05:48:20.593 +0000 [INFO] (0001:transaction): {done:  8 / 8, running: 0}\nembulk_1  | 2019-04-15 05:48:20.594 +0000 [INFO] (0001:transaction): {done:  8 / 8, running: 0}\nembulk_1  | 2019-04-15 05:48:20.604 +0000 [INFO] (main): Committed.\nembulk_1  | 2019-04-15 05:48:20.604 +0000 [INFO] (main): Next config diff: {\"in\":{\"last_path\":\"./csv/007.csv\"},\"out\":{}}\nembulk_1  | name\nembulk_1  | 001\nembulk_1  | 002\nembulk_1  | 003\nembulk_1  | 006\nembulk_1  | 007\nembulk_1  | 2019/04/15 05:48:20 Command finished successfully.\n```\n\n## Why are some recoreds not inserted?\n\n`com.mysql.jdbc.PreparedStatement#executeBatchInternal` always clear its `batchedArgs`, so `batchedArgs` is empty in second try.\n\ncf. https://github.com/mysql/mysql-connector-j/blob/5.1.44/src/com/mysql/jdbc/PreparedStatement.java#L1263-L1267\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabicky%2Fembulk-output-jdbc-issue-241","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabicky%2Fembulk-output-jdbc-issue-241","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabicky%2Fembulk-output-jdbc-issue-241/lists"}