{"id":15062171,"url":"https://github.com/userqq/mysql-binlog","last_synced_at":"2025-04-10T09:55:02.474Z","repository":{"id":65770573,"uuid":"599275282","full_name":"userqq/mysql-binlog","owner":"userqq","description":"MySQL/MariaDB change data capture","archived":false,"fork":false,"pushed_at":"2024-04-08T16:18:22.000Z","size":117,"stargazers_count":8,"open_issues_count":6,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T23:15:59.092Z","etag":null,"topics":["binlog","binlog2json","cdc","changedatacapture","events","mariadb","mariadb-binlog-events","mariadb-replication","mariadb-replication-listener","mariadb-replication-protocol","maridb-binlog","mysql","mysql-binlog","mysql-binlog-events","mysql-replication","mysql-replication-listener","mysql-replication-protocol","php","php-binlog","replication"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/userqq.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-08T20:05:57.000Z","updated_at":"2024-12-28T02:11:54.000Z","dependencies_parsed_at":"2024-04-08T17:32:03.373Z","dependency_job_id":"2bd37d86-eb2d-40be-b371-114799ee8039","html_url":"https://github.com/userqq/mysql-binlog","commit_stats":{"total_commits":9,"total_committers":1,"mean_commits":9.0,"dds":0.0,"last_synced_commit":"2357f16b4b6d0fb0acf877c46f7551d2d0c56dad"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/userqq%2Fmysql-binlog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/userqq%2Fmysql-binlog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/userqq%2Fmysql-binlog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/userqq%2Fmysql-binlog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/userqq","download_url":"https://codeload.github.com/userqq/mysql-binlog/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248198845,"owners_count":21063626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binlog","binlog2json","cdc","changedatacapture","events","mariadb","mariadb-binlog-events","mariadb-replication","mariadb-replication-listener","mariadb-replication-protocol","maridb-binlog","mysql","mysql-binlog","mysql-binlog-events","mysql-replication","mysql-replication-listener","mysql-replication-protocol","php","php-binlog","replication"],"created_at":"2024-09-24T23:31:33.685Z","updated_at":"2025-04-10T09:55:02.455Z","avatar_url":"https://github.com/userqq.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pure php MySQL/MariaDB replica client.\n\nThis package enables you to react to changes in your database without creating additional load on it, such as triggers, and performs faster than similar solutions. It impersonates a replica and uses binlog from MySQL/MariaDB to track changes in the database. When data changes, a database writes an event to binlog, which could be converted to JSON. This JSON can be sent to any destination, such as Apache Kafka, Kinesis, RabbitMQ or directly to other databases such as MongoDB, ClickHouse, Elastic to ensure fast and efficient data processing. With this package, you can have a copy of your database with a delay of minutes or even seconds, no matter how large your database is.\n\nIt can be useful for companies that want to keep a copy of their data in other systems to ensure more reliable storage or to process data with other tools. also it be used to send data to message queues, making it easier to process and analyze data in real-time. Therefore, companies can take advantage of the benefits of message queueing systems, such as load balancing, scalability, and fault-tolerance, while still maintaining a reliable copy of their data in other systems.\n\n## Installation\nThis package can be installed as a [Composer](https://getcomposer.org/) dependency.\n\n```bash\ncomposer require userqq/mysql-binlog \n```\n\n## Сonfiguration\n### Database configuration\nIn order to parse data from the binlog without additional actions and without excessive requests to the information_schema, the database must be properly configured. This allows for the re-reading of the data types and column names without additional requests, and thus the data stored in the binlog can be re-read even without the database, while still providing all the details about the data.\n\nThis library requires the `binlog_row_metadata` option, which is available in **MySQL \u003e= 8.0.1** or in **MariaDB \u003e= 10.5.0**. The `binlog_row_metadata` option must be set to `FULL` to ensure that all necessary metadata is included in the binlog for proper parsing.\n\nJust add the following config from example below to your database configuration, then restart the database and prune old binary logs:\n```mysql\nFLUSH BINARY LOGS; \nPURGE BINARY LOGS BEFORE NOW();\n```\nThis will flush the current binary log to disk and then remove all binary logs before the current time.\n\nUsually you can find config in `/etc/mysql` directory.\n```\n[mysqld]\nserver-id           = 1\nlog_bin             = /var/log/mysql/mysql-bin.log\nexpire_logs_days    = 10\nmax_binlog_size     = 100M\nbinlog-format       = row\nlog_slave_updates   = on\nbinlog_row_image    = full\nbinlog_row_metadata = full\nbinlog_do_db        = mydatabase\nnet_read_timeout    = 3600\nnet_write_timeout   = 3600\n```\n\n### Binlog reader configuration\nTo simplify configuration library exposes `UserQQ\\MySQL\\Binlog\\Config` class:\n```php\nuse UserQQ\\MySQL\\Binlog\\Config;\n\n$config = (new Config())\n    -\u003ewithUser('root')\n    -\u003ewithPassword('toor');\n```\n\nHere is a list of available configuration options:\n```php\nwithUser(string $user): static                   // Database user with replication privileges\n```\n\n```php\nwithPassword(string $password): static           // Database user's password\n```\n\n```php\nwithHost(string $host): static                   // Database host\n```\n\n```php\nwithPort(int $port): static                      // Database host\n```\n\n```php\nwithBinlogFile(?string $binlogFile): static      // Binlog file to start from\n```\n\n```php\nwithBinlogPosition(?int $binlogPosition): static // Position in binlog file to start from\n```\n\nPlease refer to [`Config.php`](https://github.com/userqq/mysql-binlog/blob/main/src/Config.php) for more details\n\n## Run the application\nThen just create an instance of `UserQQ\\MySQL\\Binlog\\EventsIterator` and iterate over it.\n```php\n\u003c?php declare(strict_types=1);\n\nrequire __DIR__ . '/vendor/autoload.php';\n\nuse UserQQ\\MySQL\\Binlog\\Config;\nuse UserQQ\\MySQL\\Binlog\\EventsIterator;\n\n$config = (new Config())\n    -\u003ewithUser('root')\n    -\u003ewithPassword('toor');\n\n$eventsIterator = new EventsIterator($config);\n\nforeach ($eventsIterator as $position =\u003e $event) {\n    echo json_encode($position) . PHP_EOL;\n    echo json_encode($event, JSON_PRETTY_PRINT) . PHP_EOL;\n    echo PHP_EOL;\n}\n```\nRun inserts, updates, and deletes on your database and see how it reacts!\n\nSee the [`examples`](https://github.com/userqq/mysql-binlog/tree/main/examples) folder for more examples.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuserqq%2Fmysql-binlog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuserqq%2Fmysql-binlog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuserqq%2Fmysql-binlog/lists"}