{"id":18521731,"url":"https://github.com/transferwise/pipelinewise-tap-mysql","last_synced_at":"2025-04-13T09:42:28.009Z","repository":{"id":40236619,"uuid":"189061824","full_name":"transferwise/pipelinewise-tap-mysql","owner":"transferwise","description":"Singer.io Tap for MySQL - PipelineWise compatible","archived":false,"fork":false,"pushed_at":"2024-04-04T13:42:54.000Z","size":514,"stargazers_count":17,"open_issues_count":23,"forks_count":56,"subscribers_count":57,"default_branch":"master","last_synced_at":"2024-04-14T05:18:23.931Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://transferwise.github.io/pipelinewise","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/transferwise.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-28T16:08:21.000Z","updated_at":"2024-05-09T10:45:21.665Z","dependencies_parsed_at":"2024-05-09T10:45:08.999Z","dependency_job_id":"876f7cde-7d3c-4fd3-88b0-16e224415750","html_url":"https://github.com/transferwise/pipelinewise-tap-mysql","commit_stats":{"total_commits":536,"total_committers":26,"mean_commits":"20.615384615384617","dds":0.582089552238806,"last_synced_commit":"250b2f314284b0d29891ef2ab9e497a2c0937ed0"},"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transferwise%2Fpipelinewise-tap-mysql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transferwise%2Fpipelinewise-tap-mysql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transferwise%2Fpipelinewise-tap-mysql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/transferwise%2Fpipelinewise-tap-mysql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/transferwise","download_url":"https://codeload.github.com/transferwise/pipelinewise-tap-mysql/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248693583,"owners_count":21146826,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T17:27:15.137Z","updated_at":"2025-04-13T09:42:27.980Z","avatar_url":"https://github.com/transferwise.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Notice\nTo better serve Wise business and customer needs, the PipelineWise codebase needs to shrink.\nWe have made the difficult decision that, going forward many components of PipelineWise will be removed or incorporated in the main repo.\nThe last version before this decision is [v0.64.1](https://github.com/transferwise/pipelinewise/tree/v0.64.1)\n\nWe thank all in the open-source community, that over the past 6 years, have helped to make PipelineWise a robust product for heterogeneous replication of many many Terabytes, daily\n\n# pipelinewise-tap-mysql\n\n[![PyPI version](https://badge.fury.io/py/pipelinewise-tap-mysql.svg)](https://badge.fury.io/py/pipelinewise-tap-mysql)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pipelinewise-tap-mysql.svg)](https://pypi.org/project/pipelinewise-tap-mysql/)\n[![License: MIT](https://img.shields.io/badge/License-GPLv3-yellow.svg)](https://opensource.org/licenses/GPL-3.0)\n\n[Singer](https://www.singer.io/) tap that extracts data from a [MySQL](https://www.mysql.com/) database and produces JSON-formatted data following the [Singer spec](https://github.com/singer-io/getting-started/blob/master/docs/SPEC.md).\n\nThis is a [PipelineWise](https://transferwise.github.io/pipelinewise) compatible tap connector.\n\n## How to use it\n\nThe recommended method of running this tap is to use it from [PipelineWise](https://transferwise.github.io/pipelinewise). When running it from PipelineWise you don't need to configure this tap with JSON files and most of things are automated. Please check the related documentation at [Tap MySQL](https://transferwise.github.io/pipelinewise/connectors/taps/mysql.html)\n\nIf you want to run this [Singer Tap](https://singer.io) independently please read further.\n\n## Usage\n\nThis section dives into basic usage of `tap-mysql` by walking through extracting\ndata from a table. It assumes that you can connect to and read from a MySQL\ndatabase.\n\n### Install\n\nFirst, make sure Python 3 is installed on your system or follow these\ninstallation instructions for [Mac](http://docs.python-guide.org/en/latest/starting/install3/osx/) or\n[Ubuntu](https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-ubuntu-16-04).\n\nIt's recommended to use a virtualenv:\n\n```bash\n  python3 -m venv venv\n  pip install pipelinewise-tap-mysql\n```\n\nor\n\n```bash\n  python3 -m venv venv\n  . venv/bin/activate\n  pip install --upgrade pip\n  pip install .\n```\n\n### Have a source database\n\nThere's some important business data siloed in this MySQL database -- we need to\nextract it. Here's the table we'd like to sync:\n\n```\nmysql\u003e select * from example_db.animals;\n+----|----------|----------------------+\n| id | name     | likes_getting_petted |\n+----|----------|----------------------+\n|  1 | aardvark |                    0 |\n|  2 | bear     |                    0 |\n|  3 | cow      |                    1 |\n+----|----------|----------------------+\n3 rows in set (0.00 sec)\n```\n\n### Create the configuration file\n\nCreate a config file containing the database connection credentials, see [sample](config.json.sample).\n\nList of config parameters:\n\n| Parameter         | type                          | required | default                                                                                                                                                           | description                                                                                                               |\n|-------------------|-------------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|\n| host              | string                        | yes      | -                                                                                                                                                                 | mysql/mariadb host                                                                                                        |\n| port              | int                           | yes      | -                                                                                                                                                                 | mysql/mariadb port                                                                                                        |\n| user              | string                        | yes      | -                                                                                                                                                                 | db username                                                                                                               |\n| password          | string                        | yes      | -                                                                                                                                                                 | db password                                                                                                               |\n| cursorclass       | string                        | No       | `pymysql.cursors.SSCursor`                                                                                                                                        | set cursorclass used by PyMYSQL                                                                                           |\n| database          | string                        | No       | -                                                                                                                                                                 | Database to use, None to not use a particular one. Used by PyMYSQL                                                        |\n| server_id         | int                           | False    | Randomly generated int                                                                                                                                            | Used as the slave id when this tap is connecting to the server                                                            |\n| filter_dbs        | string                        | False    | -                                                                                                                                                                 | Comma separated list of schemas to extract tables only from particular schemas and to improve data extraction performance |\n| use_gtid          | bool                          | False    | False                                                    \u003cbr/\u003e                                                                                                         | Flag to enable log based replication using GTID               |\n| engine            | string ('mysql' or 'mariadb') | False    | 'mysql'                                                                                                                                                           | Indicate which flavor the server is, used for LOG_BASED with GTID                                                         |\n| ssl               | string (\"true\")               | No       | False                                                                                                                                                             | Enable SSL connection                                                                                                     |\n| ssl_ca            | string                        | No       | -                                                                                                                                                                 | for self-signed SSL                                                                                                       |\n| ssl_cert          | string                        | No       | -                                                                                                                                                                 | for self-signed SSL                                                                                                       |\n| ssl_key           | string                        | No       | -                                                                                                                                                                 | for self-signed SSL                                                                                                       |\n| internal_hostname | string | No       | -                                                                                                                                                                 | Override match hostname for google cloud                                                                                  |\n| session_sqls      | List of strings               | No       | ```['SET @@session.time_zone=\"+0:00\"', 'SET @@session.wait_timeout=28800', 'SET @@session.net_read_timeout=3600', 'SET @@session.innodb_lock_wait_timeout=3600']``` | Set session variables dynamically.                                                                                        |\n\n\n### Discovery mode\n\nThe tap can be invoked in discovery mode to find the available tables and\ncolumns in the database:\n\n```bash\n$ tap-mysql --config config.json --discover\n\n```\n\nA discovered catalog is output, with a JSON-schema description of each table. A\nsource table directly corresponds to a Singer stream.\n\n```json\n{\n  \"streams\": [\n    {\n      \"tap_stream_id\": \"example_db-animals\",\n      \"table_name\": \"animals\",\n      \"schema\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"name\": {\n            \"inclusion\": \"available\",\n            \"type\": [\n              \"null\",\n              \"string\"\n            ],\n            \"maxLength\": 255\n          },\n          \"id\": {\n            \"inclusion\": \"automatic\",\n            \"minimum\": -2147483648,\n            \"maximum\": 2147483647,\n            \"type\": [\n              \"null\",\n              \"integer\"\n            ]\n          },\n          \"likes_getting_petted\": {\n            \"inclusion\": \"available\",\n            \"type\": [\n              \"null\",\n              \"boolean\"\n            ]\n          }\n        }\n      },\n      \"metadata\": [\n        {\n          \"breadcrumb\": [],\n          \"metadata\": {\n            \"row-count\": 3,\n            \"table-key-properties\": [\n              \"id\"\n            ],\n            \"database-name\": \"example_db\",\n            \"selected-by-default\": false,\n            \"is-view\": false,\n          }\n        },\n        {\n          \"breadcrumb\": [\n            \"properties\",\n            \"id\"\n          ],\n          \"metadata\": {\n            \"sql-datatype\": \"int(11)\",\n            \"selected-by-default\": true\n          }\n        },\n        {\n          \"breadcrumb\": [\n            \"properties\",\n            \"name\"\n          ],\n          \"metadata\": {\n            \"sql-datatype\": \"varchar(255)\",\n            \"selected-by-default\": true\n          }\n        },\n        {\n          \"breadcrumb\": [\n            \"properties\",\n            \"likes_getting_petted\"\n          ],\n          \"metadata\": {\n            \"sql-datatype\": \"tinyint(1)\",\n            \"selected-by-default\": true\n          }\n        }\n      ],\n      \"stream\": \"animals\"\n    }\n  ]\n}\n\n```\n\n### Field selection\n\nIn sync mode, `tap-mysql` consumes the catalog and looks for tables and fields\nhave been marked as _selected_ in their associated metadata entries.\n\nRedirect output from the tap's discovery mode to a file so that it can be\nmodified:\n\n```bash\n$ tap-mysql -c config.json --discover \u003e properties.json\n```\n\nThen edit `properties.json` to make selections. In this example we want the\n`animals` table. The stream's metadata entry (associated with `\"breadcrumb\": []`)\ngets a top-level `selected` flag, as does its columns' metadata entries. Additionally,\nwe will mark the `animals` table to replicate using a `FULL_TABLE` strategy. For more,\ninformation, see [Replication methods and state file](#replication-methods-and-state-file).\n\n```json\n[\n  {\n    \"breadcrumb\": [],\n    \"metadata\": {\n      \"row-count\": 3,\n      \"table-key-properties\": [\n        \"id\"\n      ],\n      \"database-name\": \"example_db\",\n      \"selected-by-default\": false,\n      \"is-view\": false,\n      \"selected\": true,\n      \"replication-method\": \"FULL_TABLE\"\n    }\n  },\n  {\n    \"breadcrumb\": [\n      \"properties\",\n      \"id\"\n    ],\n    \"metadata\": {\n      \"sql-datatype\": \"int(11)\",\n      \"selected-by-default\": true,\n      \"selected\": true\n    }\n  },\n  {\n    \"breadcrumb\": [\n      \"properties\",\n      \"name\"\n    ],\n    \"metadata\": {\n      \"sql-datatype\": \"varchar(255)\",\n      \"selected-by-default\": true,\n      \"selected\": true\n    }\n  },\n  {\n    \"breadcrumb\": [\n      \"properties\",\n      \"likes_getting_petted\"\n    ],\n    \"metadata\": {\n      \"sql-datatype\": \"tinyint(1)\",\n      \"selected-by-default\": true,\n      \"selected\": true\n    }\n  }\n]\n```\n\n### Sync mode\n\nWith a properties catalog that describes field and table selections, the tap can be invoked in sync mode:\n\n```bash\n$ tap-mysql -c config.json --properties properties.json\n```\n\nMessages are written to standard output following the Singer specification. The\nresultant stream of JSON data can be consumed by a Singer target.\n\n```json\n{\"value\": {\"currently_syncing\": \"example_db-animals\"}, \"type\": \"STATE\"}\n\n{\"key_properties\": [\"id\"], \"stream\": \"animals\", \"schema\": {\"properties\": {\"name\": {\"inclusion\": \"available\", \"maxLength\": 255, \"type\": [\"null\", \"string\"]}, \"likes_getting_petted\": {\"inclusion\": \"available\", \"type\": [\"null\", \"boolean\"]}, \"id\": {\"inclusion\": \"automatic\", \"minimum\": -2147483648, \"type\": [\"null\", \"integer\"], \"maximum\": 2147483647}}, \"type\": \"object\"}, \"type\": \"SCHEMA\"}\n\n{\"stream\": \"animals\", \"version\": 1509133344771, \"type\": \"ACTIVATE_VERSION\"}\n\n{\"record\": {\"name\": \"aardvark\", \"likes_getting_petted\": false, \"id\": 1}, \"stream\": \"animals\", \"version\": 1509133344771, \"type\": \"RECORD\"}\n\n{\"record\": {\"name\": \"bear\", \"likes_getting_petted\": false, \"id\": 2}, \"stream\": \"animals\", \"version\": 1509133344771, \"type\": \"RECORD\"}\n\n{\"record\": {\"name\": \"cow\", \"likes_getting_petted\": true, \"id\": 3}, \"stream\": \"animals\", \"version\": 1509133344771, \"type\": \"RECORD\"}\n\n{\"stream\": \"animals\", \"version\": 1509133344771, \"type\": \"ACTIVATE_VERSION\"}\n\n{\"value\": {\"currently_syncing\": \"example_db-animals\", \"bookmarks\": {\"example_db-animals\": {\"initial_full_table_complete\": true}}}, \"type\": \"STATE\"}\n\n{\"value\": {\"currently_syncing\": null, \"bookmarks\": {\"example_db-animals\": {\"initial_full_table_complete\": true}}}, \"type\": \"STATE\"}\n```\n\n## Replication methods and state file\n\nIn the above example, we invoked `tap-mysql` without providing a _state_ file and without specifying a replication\nmethod. The ways to replicate a given table are `FULL_TABLE`, `LOG_BASED` and `INCREMENTAL`.\n\n### LOG_BASED\n\nLOG_BASED replication makes use of the server's binary logs (binlogs), this method can work with primary\nservers, the tap acts as a replica and requests the primary to stream log events,the tap then consumes events\npertaining to row changes (inserts, updates, deletes), binlog file rotate and gtid events.\n\nLog_based method always requires an initial sync to get a snapshot of the table and current binlog coordinates/gtid\nposition.\n\nThe tap support two ways of consuming log events: using binlog coordinates or GTID, the default behavior is using\nbinlog coordinates, when turning the `use_gtid` flag, you have to specify the engine flavor (mariadb/mysql) due to\nhow different are the GTID implementations in these two engines.\n\nWhen enabling the `use_gtid` flag and the engine is MariaDB, the tap will dynamically infer the GTID pos from\nexisting binlog coordinate in the state, if the engine is mysql, it will fail.\n\n#### State when using binlog coordinates\n```json\n{\n  \"bookmarks\": {\n    \"example_db-table1\": {\"log_file\": \"mysql-binlog.0003\", \"log_pos\": 3244},\n    \"example_db-table2\": {\"log_file\": \"mysql-binlog.0001\", \"log_pos\": 42},\n    \"example_db-table3\": {\"log_file\": \"mysql-binlog.0003\", \"log_pos\": 100}\n  }\n}\n```\n\n#### State when using GTID\n```json\n{\n  \"bookmarks\": {\n    \"example_db-table1\": {\"log_file\": \"mysql-binlog.0003\", \"log_pos\": 3244, \"gtid\": \"0:364864374:599\"},\n    \"example_db-table2\": {\"log_file\": \"mysql-binlog.0001\", \"log_pos\": 42, \"gtid\": \"0:364864374:375\"},\n    \"example_db-table3\": {\"log_file\": \"mysql-binlog.0003\", \"log_pos\": 100, \"gtid\": \"0:364864374:399\"}\n  }\n}\n```\n\n### Full Table\n\nFull-table replication extracts all data from the source table each time the tap is invoked.\n\n### Incremental\n\nIncremental replication works in conjunction with a state file to only extract\nnew records each time the tap is invoked. This requires a replication key to be\nspecified in the table's metadata as well.\n\n#### Example\n\nLet's sync the `animals` table again, but this time using incremental\nreplication. The replication method and replication key are set in the\ntable's metadata entry in properties file:\n\n```json\n{\n  \"streams\": [\n    {\n      \"tap_stream_id\": \"example_db-animals\",\n      \"table_name\": \"animals\",\n      \"schema\": { ... },\n      \"metadata\": [\n        {\n          \"breadcrumb\": [],\n          \"metadata\": {\n            \"row-count\": 3,\n            \"table-key-properties\": [\n              \"id\"\n            ],\n            \"database-name\": \"example_db\",\n            \"selected-by-default\": false,\n            \"is-view\": false,\n            \"replication-method\": \"INCREMENTAL\",\n            \"replication-key\": \"id\"\n          }\n        },\n        ...\n      ],\n      \"stream\": \"animals\"\n    }\n  ]\n}\n```\n\nWe have no meaningful state so far, so just invoke the tap in sync mode again\nwithout a state file:\n\n```bash\n$ tap-mysql -c config.json --properties properties.json\n```\n\nThe output messages look very similar to when the table was replicated using the\ndefault `FULL_TABLE` replication method. One important difference is that the\n`STATE` messages now contain a `replication_key_value` -- a bookmark or\nhigh-water mark -- for data that was extracted:\n\n```json\n{\"type\": \"STATE\", \"value\": {\"currently_syncing\": \"example_db-animals\"}}\n\n{\"stream\": \"animals\", \"type\": \"SCHEMA\", \"schema\": {\"type\": \"object\", \"properties\": {\"id\": {\"type\": [\"null\", \"integer\"], \"minimum\": -2147483648, \"maximum\": 2147483647, \"inclusion\": \"automatic\"}, \"name\": {\"type\": [\"null\", \"string\"], \"inclusion\": \"available\", \"maxLength\": 255}, \"likes_getting_petted\": {\"type\": [\"null\", \"boolean\"], \"inclusion\": \"available\"}}}, \"key_properties\": [\"id\"]}\n\n{\"stream\": \"animals\", \"type\": \"ACTIVATE_VERSION\", \"version\": 1509135204169}\n\n{\"stream\": \"animals\", \"type\": \"RECORD\", \"version\": 1509135204169, \"record\": {\"id\": 1, \"name\": \"aardvark\", \"likes_getting_petted\": false}}\n\n{\"stream\": \"animals\", \"type\": \"RECORD\", \"version\": 1509135204169, \"record\": {\"id\": 2, \"name\": \"bear\", \"likes_getting_petted\": false}}\n\n{\"stream\": \"animals\", \"type\": \"RECORD\", \"version\": 1509135204169, \"record\": {\"id\": 3, \"name\": \"cow\", \"likes_getting_petted\": true}}\n\n{\"type\": \"STATE\", \"value\": {\"bookmarks\": {\"example_db-animals\": {\"version\": 1509135204169, \"replication_key_value\": 3, \"replication_key\": \"id\"}}, \"currently_syncing\": \"example_db-animals\"}}\n\n{\"type\": \"STATE\", \"value\": {\"bookmarks\": {\"example_db-animals\": {\"version\": 1509135204169, \"replication_key_value\": 3, \"replication_key\": \"id\"}}, \"currently_syncing\": null}}\n```\n\nNote that the final `STATE` message has a `replication_key_value` of `3`,\nreflecting that the extraction ended on a record that had an `id` of `3`.\nSubsequent invocations of the tap will pick up from this bookmark.\n\nNormally, the target will echo the last `STATE` after it's finished processing\ndata. For this example, let's manually write a `state.json` file using the\n`STATE` message:\n\n```json\n{\n  \"bookmarks\": {\n    \"example_db-animals\": {\n      \"version\": 1509135204169,\n      \"replication_key_value\": 3,\n      \"replication_key\": \"id\"\n    }\n  },\n  \"currently_syncing\": null\n}\n```\n\nLet's add some more animals to our farm:\n\n```\nmysql\u003e insert into animals (name, likes_getting_petted) values ('dog', true), ('elephant', true), ('frog', false);\n```\n\n```bash\n$ tap-mysql -c config.json --properties properties.json --state state.json\n```\n\nThis invocation extracts any data since (and including) the\n`replication_key_value`:\n\n```json\n{\"type\": \"STATE\", \"value\": {\"bookmarks\": {\"example_db-animals\": {\"replication_key\": \"id\", \"version\": 1509135204169, \"replication_key_value\": 3}}, \"currently_syncing\": \"example_db-animals\"}}\n\n{\"key_properties\": [\"id\"], \"schema\": {\"properties\": {\"name\": {\"maxLength\": 255, \"inclusion\": \"available\", \"type\": [\"null\", \"string\"]}, \"id\": {\"maximum\": 2147483647, \"minimum\": -2147483648, \"inclusion\": \"automatic\", \"type\": [\"null\", \"integer\"]}, \"likes_getting_petted\": {\"inclusion\": \"available\", \"type\": [\"null\", \"boolean\"]}}, \"type\": \"object\"}, \"type\": \"SCHEMA\", \"stream\": \"animals\"}\n\n{\"type\": \"ACTIVATE_VERSION\", \"version\": 1509135204169, \"stream\": \"animals\"}\n\n{\"record\": {\"name\": \"cow\", \"id\": 3, \"likes_getting_petted\": true}, \"type\": \"RECORD\", \"version\": 1509135204169, \"stream\": \"animals\"}\n{\"record\": {\"name\": \"dog\", \"id\": 4, \"likes_getting_petted\": true}, \"type\": \"RECORD\", \"version\": 1509135204169, \"stream\": \"animals\"}\n{\"record\": {\"name\": \"elephant\", \"id\": 5, \"likes_getting_petted\": true}, \"type\": \"RECORD\", \"version\": 1509135204169, \"stream\": \"animals\"}\n{\"record\": {\"name\": \"frog\", \"id\": 6, \"likes_getting_petted\": false}, \"type\": \"RECORD\", \"version\": 1509135204169, \"stream\": \"animals\"}\n\n{\"type\": \"STATE\", \"value\": {\"bookmarks\": {\"example_db-animals\": {\"replication_key\": \"id\", \"version\": 1509135204169, \"replication_key_value\": 6}}, \"currently_syncing\": \"example_db-animals\"}}\n\n{\"type\": \"STATE\", \"value\": {\"bookmarks\": {\"example_db-animals\": {\"replication_key\": \"id\", \"version\": 1509135204169, \"replication_key_value\": 6}}, \"currently_syncing\": null}}\n```\n\n## To run tests:\n\n1. You'll need to have a running MySQL or MariaDB server to run the tests. Run the following SQL commands as a privileged user to create the required objects:\n```\nCREATE USER \u003cmysql-user\u003e IDENTIFIED BY '\u003cmysql-password\u003e';\nCREATE DATABASE tap_mysql_test;\nGRANT ALL PRIVILEGES ON tap_mysql_test.* TO \u003cmysql-user\u003e;\n```\n\n**Note**: The user and password can be anything but the database name needs to be `tap_mysql_test`.\n\n2. Define the environment variables that are required to run the tests:\n```\n  export TAP_MYSQL_HOST=\u003cmysql-host\u003e\n  export TAP_MYSQL_PORT=\u003cmysql-port\u003e\n  export TAP_MYSQL_USER=\u003cmysql-user\u003e\n  export TAP_MYSQL_PASSWORD=\u003cmysql-password\u003e\n  export TAP_MYSQL_ENGINE=\u003cengine\u003e\n```\n\n3. Install python test dependencies in a virtual env\n\n```bash\npython3 -m venv venv\n. venv/bin/activate\npip install --upgrade pip\npip install .[test]\n```\n\n4. To run tests:\n```bash\nnosetests -c .noserc tests\n```\n\n### To run pylint:\n\n1. Install python dependencies and run python linter\n```\n  python3 -m venv venv\n  . venv/bin/activate\n  pip install --upgrade pip\n  pip install .[test]\n  pylint --rcfile .pylintrc tap_mysql\n```\n\n---\n\nBased on Stitch documentation\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftransferwise%2Fpipelinewise-tap-mysql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftransferwise%2Fpipelinewise-tap-mysql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftransferwise%2Fpipelinewise-tap-mysql/lists"}