{"id":13806766,"url":"https://github.com/guoyu511/vertx-mysql-binlog-client","last_synced_at":"2026-01-11T17:57:57.955Z","repository":{"id":57733368,"uuid":"94186558","full_name":"guoyu511/vertx-mysql-binlog-client","owner":"guoyu511","description":"A Vert.x client allowing applications tapping into MySQL replication stream.","archived":false,"fork":false,"pushed_at":"2017-10-29T03:00:19.000Z","size":64,"stargazers_count":16,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-08-04T01:06:46.703Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/guoyu511.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-13T07:59:07.000Z","updated_at":"2024-05-30T22:29:41.000Z","dependencies_parsed_at":"2022-09-26T22:11:22.323Z","dependency_job_id":null,"html_url":"https://github.com/guoyu511/vertx-mysql-binlog-client","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guoyu511%2Fvertx-mysql-binlog-client","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guoyu511%2Fvertx-mysql-binlog-client/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guoyu511%2Fvertx-mysql-binlog-client/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/guoyu511%2Fvertx-mysql-binlog-client/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/guoyu511","download_url":"https://codeload.github.com/guoyu511/vertx-mysql-binlog-client/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225260765,"owners_count":17446103,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T01:01:15.812Z","updated_at":"2026-01-11T17:57:57.902Z","avatar_url":"https://github.com/guoyu511.png","language":"Java","funding_links":[],"categories":["Database Clients"],"sub_categories":[],"readme":"# MySQL Binlog client for Vert.x\n\n[![Travis CI](https://travis-ci.org/guoyu511/vertx-mysql-binlog-client.svg?branch=master)](https://travis-ci.org/guoyu511/vertx-mysql-binlog-client)\n\nA Vert.x client for tapping into MySQL replication stream. Based on Vert.x 3.4.1.\n\nIt uses [MySQL Binary Log connector](https://github.com/shyiko/mysql-binlog-connector-java) to interact with the MySQL which implemented the MySQL binlog protocol by java.\n\n## How to use\n\n### Configure your MySQL master\n\nBe sure the binlog is enabled on the MySQL master and it is in **ROW** format, otherwise the client cannot receive any row events.\n\nTo enable the binary log, start the server with the [--log-bin=base_name](https://dev.mysql.com/doc/refman/5.7/en/replication-options-binary-log.html#option_mysqld_log-bin) option. For example:\n\n```\nmysqld --log-bin=mysql-bin --binlog-format=ROW\n```\n\nTo specify the format globally for all clients, set the global value of the [binlog_format](https://dev.mysql.com/doc/refman/5.7/en/replication-options-binary-log.html#sysvar_binlog_format) system variable:\n\n```\nSET GLOBAL binlog_format = 'ROW';\n```\n\n### Using Binlog Client\n\nGet the latest JAR(s) from here. Alternatively you can include following Maven dependency (available through Maven Central):\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003eio.github.guoyu511\u003c/groupId\u003e\n    \u003cartifactId\u003evertx-mysql-binlog\u003c/artifactId\u003e\n    \u003cversion\u003e0.2.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n### Tapping into replication stream\n\nIn order to connect to MySQL as a slave, you need a `BinlogClient` instance first.\n\nYou can create a client specifying a `BinlogClientOptions`:\n\n```java\nBinlogClient binlogClient = BinlogClient.create(vertx, binlogClientOptions);\n```\n\nThe `BinlogClientOptions` containing the following values:\n\n* `host` the MySQL server host name. Default is `localhost`\n* `port` the MySQL server port. Default is `3306`\n* `username` the username. Default is `root`\n* `password` the user's passowrd. Default is null means NO Password.\n* `filename` the binlog file where to starts. Default is the current file as master\n* `position` the binlog position where to starts. Default is the current position as master\n* `keepAlive` enable \"keep alive\" feature on this client. Default is true\n* `keepAliveInterval` \"keep alive\" interval in milliseconds. Default is 1 minutes\n* `heartbeatInterval` heartbeat interval in milliseconds. Default is 0 means no heartbeat\n\nBe sure that the user **must** has the `REPLICATION CLIENT` privilege for the given schema.\n\nYou can then connect to the MySQL master with the method `connect`.\n\nIt happens asynchronously and the client may not be connected until some time after the call has returned:\n\n```java\nbinlogClient.connect();\n```\n\nYou can also supplying a handler which will be called after the connection established (or failed).\n\n```\nbinlogClient.connect((ar) -\u003e {\n  ar.succeeded() // true if connection established\n});\n```\n\nAfter connected to the MySQL master as a slave, the client can handle events now.\n\n### Handle Row Events\n\nThere were several types of event defined in MySQL binlog protocol. For this client, it only concerned about events related to data modification such as `write`, `update` and  `delete`. All the events are presented as `JsonObject`. \n\nYou can set a handler to the client to handle those types of events.\n\n```java\nbinlogClient.handler((event) -\u003e {\n  String type = event.getString(\"type\");\n});\n```\n\nFor a data modification event (write / update / delete) the `JsonObject` will be looks like that:\n\n```json\n{\n  \"type\" : \"write\",\n  \"schema\" : \"test_db\",\n  \"table\" : \"test_table\",\n  \"row\" : {\n    \"id\" : 1000,\n    \"name\" : \"guoyu\"\n  }\n}\n```\n\nThe row event containing the following values:\n\n* `type` the event type, should be one of `write`, `update`, `delete`\n* `schema` the database which the data changed in\n* `table` the table name which the row changed in\n* `row` the row data in Json, columns as key / value pair.\n\n**Column mapping**\n\nThe origin `ROW Events` sent by MySQL master contains the column index but not the column name.\n\nTherefore, the BinlogClient use a `MySQLClient` instance to query the column index and names from `information_schema` database when received a `ROW Events` of a table for first time. Then it cache the column mapping and build the event object with them.\n\nWhen there is any `DROP TABLE`, `ALTER TABLE` or `CREATE TABLE` event recevied, the mapping cached will be cleared and the BinlogClient will requery the column mapping for the subsequent `ROW Events`.\n\n\n### Using as ReadStream\n\nThe BinlogClient implemented `ReadStream\u003cJsonObject\u003e` interface, that means all the methods provided by the `ReadStream` are available. \n\nFor example, use `pause ` to pause reading (that will stop to read from the underlying InputStream) .\n\n```java\nbinlogClient.pause();\n```\n\nuse `resume` to continue:\n\n```java\nbinlogClient.resume();\n```\n\nEven using `Pump` to pump the events to another `WriteStream`:\n\n```java\nPump.pump(binlogClient, targetStream).start();\n```\n\nOr pump the stream to event bus message producer:\n\n```java\nPump.pump(binlogClient, eventBus.sender('binlog.event')).start();\n\n// handle the event by event bus\neventBus.consumer('binlog.event', (msg) -\u003e {\n  JsonObject json = msg.body();\n});\n```\n\n### Using with RxJava\n\nIt also provided a Rx-ified version of the binlog client.\n\nTo using the Rx-ified api, create binlog client instance using the `BinlogClient` interface under the `io.vertx.rxjava.ext.binlog.mysql` package with the RX version of vertx.\n\n```java\nimport io.vertx.rxjava.ext.binlog.mysql.BinlogClient;\n\nBinlogClient rxBinlogClient = BinlogClient.create(rxVertx, options);\n\n```\n\nOr wrap a existing client:\n\n```java\nBinlogClient rxClient = BinlogClient.newInstance(client);\n```\n\nThen you can use the client as a `Observable`, for example, handle all `update` events:\n\n```\nrxClient.toObservable()\n  .filter((event) -\u003e \"update\".equals(event.getString(\"type\")))\n  .subscribe((event) -\u003e {\n    //do sth with this event\n  });\n```\n\n\n### Binlog Filename and position\n\nSome time you need to know the filename and the position where the replication stream at. \n\nFor example, you may want to save the filename and position when a event comes.\n\nYou can retrieve them use `filename` and `position` :\n\n```java\nbinlogClient.handle((event) -\u003e {\n  //some event coming\n  String filename = binlogClient.filename();\n  long position = binlogClient.position();\n  //save them in any way for future use\n});\n\n```\n\nNext time when you create your client, you can pass the filename and position to the `BinlogClientOptions` to let the client to connect at the specified position. \n\n```java\nbinlogClientOptions.setFilename(filename);\nbinlogClientOptions.setPosition(position);\nBinlogClient binlogClient = BinlogClient.create(vertx, binlogClientOptions);\n```\n\nThis ensures that you will not lose any events.\n\n### Closing the client\n\nYou can hold on to the client for a long time (e.g. the life-time of your verticle).\n\nOnce you have finished with it, you should close it:\n\n```java\nbinlogClient.close();\n// or close with a callback handler\nbinlogClient.close((ar) -\u003e {});\n```\n\n\n## Running the Tests\n\nYou can run tests with a specified MySQL instance:\n\n```\n% mvn test -Dbinlog.host=[host] -Dbinlog.port=[port] -Dbinlog.user=[user] -Dbinlog.password=[password] -Dbinlog.schema=[schema]\n```\n\nThe user must has `ALL ` privileges for the given schema.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguoyu511%2Fvertx-mysql-binlog-client","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fguoyu511%2Fvertx-mysql-binlog-client","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fguoyu511%2Fvertx-mysql-binlog-client/lists"}