{"id":19956102,"url":"https://github.com/krowinski/php-mysql-replication","last_synced_at":"2025-04-13T06:17:29.825Z","repository":{"id":4116263,"uuid":"51588283","full_name":"krowinski/php-mysql-replication","owner":"krowinski","description":"Pure PHP Implementation of MySQL replication protocol. This allow you to receive event like insert, update, delete with their data and raw SQL queries.","archived":false,"fork":false,"pushed_at":"2024-12-23T13:34:11.000Z","size":613,"stargazers_count":330,"open_issues_count":25,"forks_count":100,"subscribers_count":22,"default_branch":"master","last_synced_at":"2025-04-13T04:57:05.479Z","etag":null,"topics":["binlog","composer-packages","events","gtid","mysql","mysql-binlog","mysql-replication","mysql-replication-protocol","php","php-library","php-mysql-replication","replication","replication-events"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/krowinski.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-02-12T13:13:44.000Z","updated_at":"2025-03-03T09:54:26.000Z","dependencies_parsed_at":"2024-01-29T22:24:35.091Z","dependency_job_id":"3b817efa-db59-413f-9e00-3613dc5260aa","html_url":"https://github.com/krowinski/php-mysql-replication","commit_stats":{"total_commits":280,"total_committers":13,"mean_commits":21.53846153846154,"dds":"0.19285714285714284","last_synced_commit":"41073226e6c35c793499ba928ae02d81f1a47b8a"},"previous_names":[],"tags_count":46,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krowinski%2Fphp-mysql-replication","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krowinski%2Fphp-mysql-replication/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krowinski%2Fphp-mysql-replication/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krowinski%2Fphp-mysql-replication/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/krowinski","download_url":"https://codeload.github.com/krowinski/php-mysql-replication/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248670437,"owners_count":21142905,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binlog","composer-packages","events","gtid","mysql","mysql-binlog","mysql-replication","mysql-replication-protocol","php","php-library","php-mysql-replication","replication","replication-events"],"created_at":"2024-11-13T01:30:03.636Z","updated_at":"2025-04-13T06:17:29.805Z","avatar_url":"https://github.com/krowinski.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"php-mysql-replication\n=========\n[![Latest Stable Version](https://poser.pugx.org/krowinski/php-mysql-replication/v/stable)](https://packagist.org/packages/krowinski/php-mysql-replication) [![Total Downloads](https://poser.pugx.org/krowinski/php-mysql-replication/downloads)](https://packagist.org/packages/krowinski/php-mysql-replication) [![Latest Unstable Version](https://poser.pugx.org/krowinski/php-mysql-replication/v/unstable)](https://packagist.org/packages/krowinski/php-mysql-replication)\n[![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/krowinski/php-mysql-replication/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/krowinski/php-mysql-replication/?branch=master)\n[![Code Coverage](https://scrutinizer-ci.com/g/krowinski/php-mysql-replication/badges/coverage.png?b=master)](https://scrutinizer-ci.com/g/krowinski/php-mysql-replication/?branch=master)\n\nPure PHP Implementation of MySQL replication protocol. This allows you to receive event like insert, update, delete with their data and raw SQL queries.\n\nBased on a great work of creators：https://github.com/noplay/python-mysql-replication and https://github.com/fengxiangyun/mysql-replication\n\nInstallation\n=========\n\nIn you project\n\n```sh\ncomposer require krowinski/php-mysql-replication\n```\n\nor standalone \n\n```sh\ngit clone https://github.com/krowinski/php-mysql-replication.git\n\ncomposer install -o\n```\n\nCompatibility (based on integration tests) \n=========\nPHP\n\n- php 8.2\n- php 8.3\n\nMYSQL\n - mysql 5.5\n - mysql 5.6\n - mysql 5.7\n - mysql 8.0 (mysql_native_password and caching_sha2_password supported)\n - mariadb 5.5\n - mariadb 10.0\n - mariadb 10.1\n - probably percona versions as is based on native mysql\n\nMySQL server settings\n=========\n\nIn your MySQL server configuration file you need to enable replication:\n\n    [mysqld]\n    server-id        = 1\n    log_bin          = /var/log/mysql/mysql-bin.log\n    expire_logs_days = 10\n    max_binlog_size  = 100M\n    binlog-format    = row #Very important if you want to receive write, update and delete row events\n\n\nMysql replication events explained\n    https://dev.mysql.com/doc/internals/en/event-meanings.html\n\n\nMysql user privileges:\n```\nGRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'user'@'host';\n\nGRANT SELECT ON `dbName`.* TO 'user'@'host';\n```\n\nConfiguration\n=========\n\nUse ConfigBuilder or ConfigFactory to create configuration.\nAvailable options:\n\n'user' - your mysql user (mandatory)\n\n'ip' or 'host' - your mysql host/ip (mandatory)\n\n'password' - your mysql password (mandatory)\n\n'port' - your mysql host port (default 3306)\n\n'charset' - db connection charset (default utf8)\n\n'gtid' - GTID marker(s) to start from (format 9b1c8d18-2a76-11e5-a26b-000c2976f3f3:1-177592)\n\n'mariaDbGtid' - MariaDB GTID marker(s) to start from (format 1-1-3,0-1-88)\n\n'slaveId' - script slave id for identification (default: 666) (SHOW SLAVE HOSTS)\n\n'binLogFileName' - bin log file name to start from\n\n'binLogPosition' - bin log position to start from \n\n'eventsOnly' - array  to listen on events (full list in [ConstEventType.php](https://github.com/krowinski/php-mysql-replication/blob/master/src/MySQLReplication/Definitions/ConstEventType.php) file)\n\n'eventsIgnore' - array to ignore events (full list in [ConstEventType.php](https://github.com/krowinski/php-mysql-replication/blob/master/src/MySQLReplication/Definitions/ConstEventType.php) file) \n\n'tablesOnly' - array to only listen on given tables (default all tables) \n\n'databasesOnly' - array to only listen on given databases (default all databases) \n \n'tableCacheSize' - some data are collected from information schema, this data is cached.\n\n'custom' - if some params must be set in extended/implemented own classes\n\n'heartbeatPeriod' - sets the interval in seconds between replication heartbeats. Whenever the master's binary log is updated with an event, the waiting period for the next heartbeat is reset. interval is a decimal value having the range 0 to 4294967 seconds and a resolution in milliseconds; the smallest nonzero value is 0.001. Heartbeats are sent by the master only if there are no unsent events in the binary log file for a period longer than interval.\n\n'saveUuid' - sets slave uuid for identification (default: 0015d2b6-8a06-4e5e-8c07-206ef3fbd274)\n\nSimilar projects\n=========\nRuby: https://github.com/y310/kodama\n\nJava: https://github.com/shyiko/mysql-binlog-connector-java\n\nGO: https://github.com/siddontang/go-mysql\n\nPython: https://github.com/noplay/python-mysql-replication\n\n.NET: https://github.com/rusuly/MySqlCdc\n\nExamples\n=========\n\nAll examples are available in the [examples directory](https://github.com/krowinski/php-mysql-replication/tree/master/example)\n\nThis example will dump all replication events to the console:\n\nRemember to change config for your user, host and password.\n\nUser should have replication privileges [ REPLICATION CLIENT, SELECT]\n\n```sh\nphp example/dump_events.php\n```\n\nFor test SQL events:\n\n```sql\nCREATE DATABASE php_mysql_replication;\nuse php_mysql_replication;\nCREATE TABLE test4 (id int NOT NULL AUTO_INCREMENT, data VARCHAR(255), data2 VARCHAR(255), PRIMARY KEY(id));\nINSERT INTO test4 (data,data2) VALUES (\"Hello\", \"World\");\nUPDATE test4 SET data = \"World\", data2=\"Hello\" WHERE id = 1;\nDELETE FROM test4 WHERE id = 1;\n```\n\nOutput will be similar to this (depends on configuration for example GTID off/on):\n\n    === Event format description ===\n    Date: 2017-07-06T13:31:11+00:00\n    Log position: 0\n    Event size: 116\n    Memory usage 2.4 MB\n    \n    === Event gtid ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803092\n    Event size: 48\n    Commit: true\n    GTID NEXT: 3403c535-624f-11e7-9940-0800275713ee:13675\n    Memory usage 2.42 MB\n    \n    === Event query ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803237\n    Event size: 145\n    Database: php_mysql_replication\n    Execution time: 0\n    Query: CREATE DATABASE php_mysql_replication\n    Memory usage 2.45 MB\n    \n    === Event gtid ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803285\n    Event size: 48\n    Commit: true\n    GTID NEXT: 3403c535-624f-11e7-9940-0800275713ee:13676\n    Memory usage 2.45 MB\n    \n    === Event query ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803500\n    Event size: 215\n    Database: php_mysql_replication\n    Execution time: 0\n    Query: CREATE TABLE test4 (id int NOT NULL AUTO_INCREMENT, data VARCHAR(255), data2 VARCHAR(255), PRIMARY KEY(id))\n    Memory usage 2.45 MB\n    \n    === Event gtid ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803548\n    Event size: 48\n    Commit: true\n    GTID NEXT: 3403c535-624f-11e7-9940-0800275713ee:13677\n    Memory usage 2.45 MB\n    \n    === Event query ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803637\n    Event size: 89\n    Database: php_mysql_replication\n    Execution time: 0\n    Query: BEGIN\n    Memory usage 2.45 MB\n    \n    === Event tableMap ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803708\n    Event size: 71\n    Table: test4\n    Database: php_mysql_replication\n    Table Id: 866\n    Columns amount: 3\n    Memory usage 2.71 MB\n    \n    === Event write ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803762\n    Event size: 54\n    Table: test4\n    Affected columns: 3\n    Changed rows: 1\n    Values: Array\n    (\n        [0] =\u003e Array\n            (\n                [id] =\u003e 1\n                [data] =\u003e Hello\n                [data2] =\u003e World\n            )\n    \n    )\n    \n    Memory usage 2.74 MB\n    \n    === Event xid ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803793\n    Event size: 31\n    Transaction ID: 662802\n    Memory usage 2.75 MB\n    \n    === Event gtid ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803841\n    Event size: 48\n    Commit: true\n    GTID NEXT: 3403c535-624f-11e7-9940-0800275713ee:13678\n    Memory usage 2.75 MB\n    \n    === Event query ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57803930\n    Event size: 89\n    Database: php_mysql_replication\n    Execution time: 0\n    Query: BEGIN\n    Memory usage 2.76 MB\n    \n    === Event tableMap ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57804001\n    Event size: 71\n    Table: test4\n    Database: php_mysql_replication\n    Table Id: 866\n    Columns amount: 3\n    Memory usage 2.75 MB\n    \n    === Event update ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57804075\n    Event size: 74\n    Table: test4\n    Affected columns: 3\n    Changed rows: 1\n    Values: Array\n    (\n        [0] =\u003e Array\n            (\n                [before] =\u003e Array\n                    (\n                        [id] =\u003e 1\n                        [data] =\u003e Hello\n                        [data2] =\u003e World\n                    )\n    \n                [after] =\u003e Array\n                    (\n                        [id] =\u003e 1\n                        [data] =\u003e World\n                        [data2] =\u003e Hello\n                    )\n    \n            )\n    \n    )\n    \n    Memory usage 2.76 MB\n    \n    === Event xid ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57804106\n    Event size: 31\n    Transaction ID: 662803\n    Memory usage 2.76 MB\n    \n    === Event gtid ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57804154\n    Event size: 48\n    Commit: true\n    GTID NEXT: 3403c535-624f-11e7-9940-0800275713ee:13679\n    Memory usage 2.76 MB\n    \n    === Event query ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57804243\n    Event size: 89\n    Database: php_mysql_replication\n    Execution time: 0\n    Query: BEGIN\n    Memory usage 2.76 MB\n    \n    === Event tableMap ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57804314\n    Event size: 71\n    Table: test4\n    Database: php_mysql_replication\n    Table Id: 866\n    Columns amount: 3\n    Memory usage 2.76 MB\n    \n    === Event delete ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57804368\n    Event size: 54\n    Table: test4\n    Affected columns: 3\n    Changed rows: 1\n    Values: Array\n    (\n        [0] =\u003e Array\n            (\n                [id] =\u003e 1\n                [data] =\u003e World\n                [data2] =\u003e Hello\n            )\n    \n    )\n    \n    Memory usage 2.77 MB\n    \n    === Event xid ===\n    Date: 2017-07-06T15:23:44+00:00\n    Log position: 57804399\n    Event size: 31\n    Transaction ID: 662804\n    Memory usage 2.77 MB\n\n\n\nBenchmarks\n=========\n\nTested on VM\n\n    Debian 8.7\n    PHP 5.6.30\n    Percona 5.6.35\n\n```sh\ninxi\n```\n\n    CPU(s)~4 Single core Intel Core i5-2500Ks (-SMP-) clocked at 5901 Mhz Kernel~3.16.0-4-amd64 x86_64 Up~1 day Mem~1340.3/1996.9MB HDD~41.9GB(27.7% used) Procs~122 Client~Shell inxi~2.1.28\n\n```sh\nphp example/benchmark.php\n```\n    Start insert data\n    7442 event by seconds (1000 total)\n    7679 event by seconds (2000 total)\n    7914 event by seconds (3000 total)\n    7904 event by seconds (4000 total)\n    7965 event by seconds (5000 total)\n    8006 event by seconds (6000 total)\n    8048 event by seconds (7000 total)\n    8038 event by seconds (8000 total)\n    8040 event by seconds (9000 total)\n    8055 event by seconds (10000 total)\n    8058 event by seconds (11000 total)\n    8071 event by seconds (12000 total)\n\nFAQ\n=========\n\n1. ### Why and when need php-mysql-replication ?\n\nWell first of all MYSQL don't give you async calls. You usually need to program this in your application (by event dispatching and adding to some queue system\nand if your db have many point of entry like web, backend other microservices its not always cheap to add processing to all of them. But using mysql replication\nprotocol you can listen on write events and process then asynchronously (the best combo it's to add item to some queue system like rabbitmq, redis or kafka).\nAlso in invalidate cache, search engine replication, real time analytics and audits.\n\n2. ### It's awesome ! but what is the catch ?\n\nWell first of all you need to know that a lot of events may come through, like if you update 1 000 000 records in table \"bar\" and you need this one insert from\nyour table \"foo\" Then all must be processed by script, and you need to wait for your data. This is normal and this how it's work. You can speed up\nusing [config options](https://github.com/krowinski/php-mysql-replication#configuration).\nAlso, if script crashes you need to save from time to time position form binlog (or gtid) to start from this position when you run this script again to avoid\nduplicates.\n\n3. ### I need to process 1 000 000 records and its taking forever!!\n Like I mention in 1 point use queue system like rabbitmq, redis or kafka, they will give you ability to process data in multiple scripts.\n\n4. ### I have a problem ? you script is missing something ! I have found a bug !\n Create an [issue](https://github.com/krowinski/php-mysql-replication/issues) I will try to work on it in my free time :)\n\n5. ### How much its give overhead to MYSQL server ?\n\nIt work like any other MYSQL in slave mode and its giving same overhead.\n \n6. ### Socket timeouts error\n\nTo fix this best is to increase db configurations ```net_read_timeout``` and ```net_write_timeout``` to 3600.  (tx Bijimon)\n \n7. ### Partial updates fix\n\nSet in my.conf ```binlog_row_image=full``` to fix receiving only partial updates.\n\n8. ### No replication events when connected to replica server   \nSet in my.conf ```log_slave_updates=on``` to fix this (#71)(#66)\n\n9. ### \"Big\" updates / inserts\nDefault MYSQL setting generates one big blob of stream this require more RAM/CPU you can change this for smaller stream using\nvariable ```binlog_row_event_max_size``` [https://dev.mysql.com/doc/refman/8.0/en/replication-options-binary-log.html#sysvar_binlog_row_event_max_size]  to\nsplit into smaller chunks \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrowinski%2Fphp-mysql-replication","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkrowinski%2Fphp-mysql-replication","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrowinski%2Fphp-mysql-replication/lists"}