{"id":22870897,"url":"https://github.com/percona-lab/mysql_random_data_load","last_synced_at":"2025-04-06T18:14:58.665Z","repository":{"id":25424614,"uuid":"103967260","full_name":"Percona-Lab/mysql_random_data_load","owner":"Percona-Lab","description":"MySQL random data loader","archived":false,"fork":false,"pushed_at":"2022-12-24T04:52:52.000Z","size":2993,"stargazers_count":293,"open_issues_count":22,"forks_count":70,"subscribers_count":49,"default_branch":"master","last_synced_at":"2025-03-30T15:12:29.065Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Percona-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-09-18T17:00:33.000Z","updated_at":"2025-01-14T03:55:59.000Z","dependencies_parsed_at":"2023-01-14T07:00:11.986Z","dependency_job_id":null,"html_url":"https://github.com/Percona-Lab/mysql_random_data_load","commit_stats":null,"previous_names":["percona-lab/random_data_load"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Percona-Lab%2Fmysql_random_data_load","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Percona-Lab%2Fmysql_random_data_load/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Percona-Lab%2Fmysql_random_data_load/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Percona-Lab%2Fmysql_random_data_load/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Percona-Lab","download_url":"https://codeload.github.com/Percona-Lab/mysql_random_data_load/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247526764,"owners_count":20953143,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-13T13:16:32.508Z","updated_at":"2025-04-06T18:14:58.642Z","avatar_url":"https://github.com/Percona-Lab.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Random data generator for MySQL\r\n[![Build Status](https://travis-ci.org/Percona-Lab/mysql_random_data_load.svg?branch=master)](https://travis-ci.org/Percona-Lab/mysql_random_data_load)\r\n\r\nMany times in my job I need to generate random data for a specific table in order to reproduce an issue.  \r\nAfter writing many random generators for every table, I decided to write a random data generator, able to get the table structure and generate random data for it.  \r\nPlase take into consideration that this is the first version and it doesn't support all field types yet!  \r\n\r\n**NOTICE**  \r\nThis is an early stage project.  \r\n\r\n## Supported fields:\r\n\r\n|Field type|Generated values|\r\n|----------|----------------|\r\n|tinyint|0 ~ 0xFF|\r\n|smallint|0 ~ 0XFFFF|\r\n|mediumint|0 ~ 0xFFFFFF|\r\n|int - integer|0 ~ 0xFFFFFFFF|\r\n|bigint|0 ~ 0xFFFFFFFFFFFFFFFF|\r\n|float|0 ~ 1e8|\r\n|decimal(m,n)|0 ~ 10^(m-n)|\r\n|double|0 ~ 1000|\r\n|char(n)|up to n random chars|\r\n|varchar(n)|up to n random chars|\r\n|date|NOW() - 1 year ~ NOW()|\r\n|datetime|NOW() - 1 year ~ NOW()|\r\n|timestamp|NOW() - 1 year ~ NOW()|\r\n|time|00:00:00 ~ 23:59:59|\r\n|year|Current year - 1 ~ current year|\r\n|tinyblob|up to 100 chars random paragraph|\r\n|tinytext|up to 100 chars random paragraph|\r\n|blob|up to 100 chars random paragraph|\r\n|text|up to 100 chars random paragraph|\r\n|mediumblob|up to 100 chars random paragraph|\r\n|mediumtext|up to 100 chars random paragraph|\r\n|longblob|up to 100 chars random paragraph|\r\n|longtext|up to 100 chars random paragraph|\r\n|varbinary|up to 100 chars random paragraph|\r\n|enum|A random item from the valid items list|\r\n|set|A random item from the valid items list|\r\n\r\n### How strings are generated\r\n\r\n- If field size \u003c 10 the program generates a random \"first name\"\r\n- If the field size \u003e 10 and \u003c 30 the program generates a random \"full name\"\r\n- If the field size \u003e 30 the program generates a \"lorem ipsum\" paragraph having up to 100 chars.\r\n \r\nThe program can detect if a field accepts NULLs and if it does, it will generate NULLs ramdomly (~ 10 % of the values).\r\n\r\n## Usage\r\n`mysql_random_data_load \u003cdatabase\u003e \u003ctable\u003e \u003cnumber of rows\u003e [options...]`\r\n\r\n## Options\r\n|Option|Description|\r\n|------|-----------|\r\n|--bulk-size|Number of rows per INSERT statement (Default: 1000)|\r\n|--debug|Show some debug information|\r\n|--fk-samples-factor|Percentage used to get random samples for foreign keys fields. Default 0.3|\r\n|--host|Host name/ip|\r\n|--max-fk-samples|Maximum number of samples for fields having foreign keys constarints. Default: 100|\r\n|--max-retries|Maximum number of rows to retry in case of errors. See duplicated keys. Deafult: 100|\r\n|--no-progressbar|Skip showing the progress bar. Default: false|\r\n|--password|Password|\r\n|--port|Port number|\r\n|--Print|Print queries to the standard output instead of inserting them into the db|\r\n|--user|Username|\r\n|--version|Show version and exit|\r\n\r\n## Foreign keys support\r\nIf a field has Foreign Keys constraints, `random-data-load` will get up to `--max-fk-samples` random samples from the referenced tables in order to insert valid values for the field.  \r\nThe number of samples to get follows this rules:  \r\n**1.** Get the aproximate number of rows in the referenced table using the `rows` field in:  \r\n```\r\nEXPLAIN SELECT COUNT(*) FROM \u003creferenced schema\u003e.\u003creferenced table\u003e\r\n```\r\n**1.1** If the number of rows is less than `max-fk-samples`, all rows are retrieved from the referenced table using this query: \r\n```\r\nSELECT \u003creferenced field\u003e FROM \u003creferenced schema\u003e.\u003creferenced table\u003e\r\n```\r\n**1.2** If the number of rows is greater than `max-fk-samples`, samples are retrieved from the referenced table using this query:  \r\n```\r\nSELECT \u003creferenced field\u003e FROM \u003creferenced schema\u003e.\u003creferenced table\u003e WHERE RAND() \u003c= \u003cfk-samples-factor\u003e LIMIT \u003cmax-fk-samples\u003e\r\n```\r\n\r\n### Example\r\n```\r\nCREATE DATABASE IF NOT EXISTS test;\r\n\r\nCREATE TABLE `test`.`t3` (\r\n  `id` int(11) NOT NULL AUTO_INCREMENT,\r\n  `tcol01` tinyint(4) DEFAULT NULL,\r\n  `tcol02` smallint(6) DEFAULT NULL,\r\n  `tcol03` mediumint(9) DEFAULT NULL,\r\n  `tcol04` int(11) DEFAULT NULL,\r\n  `tcol05` bigint(20) DEFAULT NULL,\r\n  `tcol06` float DEFAULT NULL,\r\n  `tcol07` double DEFAULT NULL,\r\n  `tcol08` decimal(10,2) DEFAULT NULL,\r\n  `tcol09` date DEFAULT NULL,\r\n  `tcol10` datetime DEFAULT NULL,\r\n  `tcol11` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,\r\n  `tcol12` time DEFAULT NULL,\r\n  `tcol13` year(4) DEFAULT NULL,\r\n  `tcol14` varchar(100) DEFAULT NULL,\r\n  `tcol15` char(2) DEFAULT NULL,\r\n  `tcol16` blob,\r\n  `tcol17` text,\r\n  `tcol18` mediumtext,\r\n  `tcol19` mediumblob,\r\n  `tcol20` longblob,\r\n  `tcol21` longtext,\r\n  `tcol22` mediumtext,\r\n  `tcol23` varchar(3) DEFAULT NULL,\r\n  `tcol24` varbinary(10) DEFAULT NULL,\r\n  `tcol25` enum('a','b','c') DEFAULT NULL,\r\n  `tcol26` set('red','green','blue') DEFAULT NULL,\r\n  `tcol27` float(5,3) DEFAULT NULL,\r\n  `tcol28` double(4,2) DEFAULT NULL,\r\n  PRIMARY KEY (`id`)\r\n) ENGINE=InnoDB;\r\n```\r\nTo generate 100K random rows, just run:\r\n```\r\nmysql_random_data_load test t3 100000 --user=root --password=root\r\n```\r\n```\r\nmysql\u003e select * from t3 limit 1\\G\r\n*************************** 1. row ***************************\r\n    id: 1\r\ntcol01: 10\r\ntcol02: 173\r\ntcol03: 1700\r\ntcol04: 13498\r\ntcol05: 33239373\r\ntcol06: 44846.4\r\ntcol07: 5300.23\r\ntcol08: 11360967.75\r\ntcol09: 2017-09-04\r\ntcol10: 2016-11-02 23:11:25\r\ntcol11: 2017-03-03 08:11:40\r\ntcol12: 03:19:39\r\ntcol13: 2017\r\ntcol14: repellat maxime nostrum provident maiores ut quo voluptas.\r\ntcol15: Th\r\ntcol16: Walter\r\ntcol17: quo repellat accusamus quidem odi\r\ntcol18: esse laboriosam nobis libero aut dolores e\r\ntcol19: Carlos Willia\r\ntcol20: et nostrum iusto ipsa sunt recusa\r\ntcol21: a accusantium laboriosam voluptas facilis.\r\ntcol22: laudantium quo unde molestiae consequatur magnam.\r\ntcol23: Pet\r\ntcol24: Richard\r\ntcol25: c\r\ntcol26: green\r\ntcol27: 47.430\r\ntcol28: 6.12\r\n1 row in set (0.00 sec)\r\n```\r\n\r\n## How to download the precompiled binaries\r\n\r\nThere are binaries available for each version for Linux and Darwin. You can find compiled binaries for each version in the releases tab:\r\n\r\nhttps://github.com/Percona-Lab/mysql_random_data_load/releases\r\n\r\n## To do\r\n- [ ] Add suport for all data types.\r\n- [X] Add supporrt for foreign keys.\r\n- [ ] Support config files to override default values/ranges.\r\n- [ ] Support custom functions via LUA plugins.\r\n\r\n## Version history\r\n\r\n#### 0.1.10\r\n- Fixed argument validations\r\n- Fixed ~/.my.cnf loading\r\n\r\n#### 0.1.10\r\n- Fixed connection parameters for MySQL 5.7 (set driver's AllowNativePasswords: true)\r\n\r\n#### 0.1.9\r\n- Added support for bunary and varbinary columns\r\n- By default, read connection params from ${HOME}/.my.cnf\r\n\r\n#### 0.1.8 \r\n- Fixed error for triggers created with MySQL 5.6\r\n- Added Travis-CI\r\n- Code clean up\r\n\r\n#### 0.1.7 \r\n- Support for MySQL 8.0\r\n- Added --print parameter \r\n- Added --version parameter\r\n- Removed qps parameter\r\n\r\n#### 0.1.6 \r\n- Improved generation speed (up to 50% faster)\r\n- Improved support for TokuDB (Thanks Agustin Gallego)\r\n- Code refactored\r\n- Improved debug logging\r\n- Added Query Per Seconds support (experimental)\r\n\r\n#### 0.1.5 \r\n- Fixed handling of NULL collation for index parser\r\n\r\n#### 0.1.4\r\n- Fixed handling of time columns\r\n- Improved support of GENERATED columns\r\n\r\n#### 0.1.3\r\n- Fixed handling of nulls\r\n\r\n#### 0.1.2\r\n- New table parser able to retrieve all the information for fields, indexes and foreign keys constraints.\r\n- Support for foreign keys constraints\r\n- Added some tests\r\n\r\n#### 0.1.1\r\n- Fixed random data generation\r\n\r\n#### 0.1.0\r\n- Initial version\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpercona-lab%2Fmysql_random_data_load","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpercona-lab%2Fmysql_random_data_load","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpercona-lab%2Fmysql_random_data_load/lists"}