{"id":13800857,"url":"https://github.com/laserdisc-io/mysql-binlog-stream","last_synced_at":"2025-06-14T19:32:55.918Z","repository":{"id":40345997,"uuid":"264344417","full_name":"laserdisc-io/mysql-binlog-stream","owner":"laserdisc-io","description":null,"archived":false,"fork":false,"pushed_at":"2024-07-29T17:53:16.000Z","size":408,"stargazers_count":14,"open_issues_count":12,"forks_count":5,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-08-04T00:05:34.811Z","etag":null,"topics":["hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/laserdisc-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-16T02:40:53.000Z","updated_at":"2024-08-04T00:05:40.950Z","dependencies_parsed_at":"2023-02-09T03:45:30.642Z","dependency_job_id":"ed0a9333-bcc3-48d5-8431-5d05050946dc","html_url":"https://github.com/laserdisc-io/mysql-binlog-stream","commit_stats":null,"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laserdisc-io%2Fmysql-binlog-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laserdisc-io%2Fmysql-binlog-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laserdisc-io%2Fmysql-binlog-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/laserdisc-io%2Fmysql-binlog-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/laserdisc-io","download_url":"https://codeload.github.com/laserdisc-io/mysql-binlog-stream/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224996484,"owners_count":17404487,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest"],"created_at":"2024-08-04T00:01:17.001Z","updated_at":"2024-11-17T04:15:08.326Z","avatar_url":"https://github.com/laserdisc-io.png","language":"Scala","funding_links":[],"categories":["Table of Contents"],"sub_categories":["Database"],"readme":"### Motivation\nStream in real-time the MySQL database events, such as CRUD and DDL. That may be useful for building Data Warehouse, integration with monolith application on the database level, etc.\n\n### Example \n```scala\nMysqlBinlogStream\n              .rawEvents[IO](binlogClient)\n              .through(streamEvents[IO](transactionState))\n              .evalTap(msg =\u003e logger.info(s\"received $msg\"))\n```\nRefer to [examples](mysql-binlog-stream-examples) for complete snippet\n\n### How to build and launch example\n\n- build docker image \n```shell script\nsbt \"project mysql-binlog-stream-examples\" clean  \"docker:publishLocal\"\n```\n- run docker compose with MySql and example service\n```shell script\ncd mysql-binlog-stream-examples\ndocker-compose -f docker-compose-mysql-it-mac.yaml up\n```\n- open new console\n- in new console \n```shell script\nmysql -u 'root' --host=0.0.0.0 --port=3307 --password=''\n``` \n- try to insert new row in `SKU` or `VARIANT` table\n```mysql\nuse test;\ninsert into sku(id, sku) values(3, '123');\n```\n- in the `docker-compose` console you should see \n```shell script\nexample_1  | 02:35:58.734 [ioapp-compute-2] INFO application - received EventMessage(sku,1589596558000,create,8908ecfb63e4-bin.000007,415,true,{\nexample_1  |   \"id\" : 3\nexample_1  | },{\nexample_1  |   \"before\" : null,\nexample_1  |   \"after\" : {\nexample_1  |     \"id\" : 3,\nexample_1  |     \"sku\" : \"123\"\nexample_1  |   }\nexample_1  | })\n```\n\ntry to update some row and see what happens\n\n#### Event examples\n\nCreate:\n```json\n{\n  \"table\" : \"sku\",\n  \"timestamp\" : 1554754028000,\n  \"action\" : \"create\",\n  \"fileName\" : \"acd03b3a873f-bin.000003\",\n  \"offset\" : 2177,\n  \"endOfTransaction\" : false,\n  \"pk\" : {\n    \"id\" : 5\n  },\n  \"row\" : {\n    \"before\" : null,\n    \"after\" : {\n      \"id\" : 5,\n      \"sku\" : \"sku5\"\n    }\n  }\n}\n```\n\nUpdate:\n\n```json\n{\n  \"table\" : \"sku\",\n  \"timestamp\" : 1554754028000,\n  \"action\" : \"create\",\n  \"fileName\" : \"acd03b3a873f-bin.000003\",\n  \"offset\" : 2177,\n  \"endOfTransaction\" : false,\n  \"pk\" : {\n    \"id\" : 5\n  },\n  \"row\" : {\n    \"before\" : {\n      \"id\" : 5,\n      \"sku\" : \"sku5\"\n    },\n    \"after\" : {\n      \"id\" : 5,\n      \"sku\" : \"new sku 5\"\n    }\n  }\n}\n```\n\nDelete:\n```json\n{\n  \"table\" : \"sku\",\n  \"timestamp\" : 1554754028000,\n  \"action\" : \"create\",\n  \"fileName\" : \"acd03b3a873f-bin.000003\",\n  \"offset\" : 2177,\n  \"endOfTransaction\" : false,\n  \"pk\" : {\n    \"id\" : 5\n  },\n  \"row\" : {\n    \"before\" : {\n      \"id\" : 5,\n      \"sku\" : \"new sku 5\"\n    },\n    \"after\" : null\n  }\n}\n```\n\nTruncate:\n```json\n{\n  \"table\" : \"sku\",\n  \"timestamp\" : 1554754028000,\n  \"action\" : \"truncate\",\n  \"fileName\" : \"acd03b3a873f-bin.000003\",\n  \"offset\" : 2497,\n  \"endOfTransaction\" : true,\n  \"pk\" : null,\n  \"row\" : null\n}\n```\n\n### Tech Stack\n- Scala\n- FS2 Functional Streams for Scala\n- circe - json streaming encoder/decoder\n- doobie - database integration layer \n\n## Support\n\n![YourKit Image](https://www.yourkit.com/images/yklogo.png \"YourKit\")\n\nThis project is supported by YourKit with monitoring and profiling Tools. YourKit supports open source with innovative and intelligent tools for monitoring and profiling Java and .NET applications. YourKit is the creator of [YourKit Java Profiler](https://www.yourkit.com/java/profiler/), [YourKit .NET Profiler](https://www.yourkit.com/.net/profiler/), and [YourKit YouMonitor](https://www.yourkit.com/youmonitor/).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaserdisc-io%2Fmysql-binlog-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flaserdisc-io%2Fmysql-binlog-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flaserdisc-io%2Fmysql-binlog-stream/lists"}