{"id":18408368,"url":"https://github.com/pingcap/tidb-binlog","last_synced_at":"2025-04-13T00:39:53.903Z","repository":{"id":37431120,"uuid":"67385952","full_name":"pingcap/tidb-binlog","owner":"pingcap","description":"A tool used to collect and merge tidb's binlog for real-time data backup and synchronization.","archived":false,"fork":false,"pushed_at":"2025-04-07T09:34:01.000Z","size":15053,"stargazers_count":295,"open_issues_count":93,"forks_count":130,"subscribers_count":44,"default_branch":"master","last_synced_at":"2025-04-13T00:39:11.029Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pingcap.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-09-05T03:44:12.000Z","updated_at":"2025-04-04T04:34:22.000Z","dependencies_parsed_at":"2023-11-28T07:24:55.755Z","dependency_job_id":"cf30ae3d-3adb-448b-8665-6f2091e1c39b","html_url":"https://github.com/pingcap/tidb-binlog","commit_stats":{"total_commits":664,"total_committers":63,"mean_commits":10.53968253968254,"dds":0.7831325301204819,"last_synced_commit":"552cffbb46230d7b4a41995379423ae9d2351a47"},"previous_names":[],"tags_count":204,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Ftidb-binlog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Ftidb-binlog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Ftidb-binlog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pingcap%2Ftidb-binlog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pingcap","download_url":"https://codeload.github.com/pingcap/tidb-binlog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248650414,"owners_count":21139671,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T03:18:50.992Z","updated_at":"2025-04-13T00:39:53.884Z","avatar_url":"https://github.com/pingcap.png","language":"Go","readme":"# TiDB-Binlog\n\n[![Build Status](https://internal.pingcap.net/idc-jenkins/buildStatus/icon?job=build_tidb_binlog_master)](https://internal.pingcap.net/idc-jenkins/job/build_tidb_binlog_master/)\n[![Coverage Status](https://coveralls.io/repos/github/pingcap/tidb-binlog/badge.svg?branch=master\u0026t=9Zn2om)](https://coveralls.io/github/pingcap/tidb-binlog?branch=master)\n[![Go Report Card](https://goreportcard.com/badge/github.com/pingcap/tidb-binlog)](https://goreportcard.com/report/github.com/pingcap/tidb-binlog)\n\n## TiDB-Binlog introduction\n\nTiDB-Binlog is a tool used to collect [TiDB's](https://github.com/pingcap/tidb) binary logs with the following features:\n\n- Data replication\n    \n    Synchronize data from the TiDB cluster to heterogeneous databases.\n\n- Real-time backup and recovery\n    \n    Backup the TiDB cluster into the Dump file and it can be used for recovery.\n\n- Multiple output format\n    \n    Support MySQL, Dump file, etc.\n    \n- History replay\n    \n    Replay from any history point.\n\n## Documentation\n+ [Detailed documentation](https://pingcap.com/docs/dev/reference/tidb-binlog-overview/)\n+ [简体中文](https://pingcap.com/docs-cn/dev/reference/tools/tidb-binlog/overview/)\n\n## Architecture\n\n![architecture](./docs/architecture.png)\n\n## Service list\n\n[Pump](./cmd/pump)\n\nPump is a daemon that receives real-time binlogs from tidb-server and writes in sequential disk files synchronously.\n\n[Drainer](./cmd/drainer)\n\nDrainer collects binlogs from each Pump in the cluster, transforms binlogs to various dialects of SQL, and applies to the downstream database or filesystem.\n\n## How to build\n\nTo check the code style and build binaries, you can simply run:\n\n```\nmake build   # build all components\n```\n\nIf you only want to build binaries, you can run:\n\n```\nmake pump  # build pump\n\nmake drainer  # build drainer\n```\n\nWhen TiDB-Binlog is built successfully, you can find the binary in the `bin` directory. \n\n## Run Test\n\nRun all tests, including unit test and integration test\n\n```\nmake test\n```\nSee [tests](./tests/README.md) for how to execute and add integration tests.\n\n## Deployment\n\nThe recommended startup sequence: PD -\u003e TiKV -\u003e [Pump](./cmd/pump) -\u003e TiDB -\u003e [Drainer](./cmd/drainer)\n\nThe best way to install TiDB-Binlog is via [TiDB-Binlog-Ansible](https://www.pingcap.com/docs-cn/tools/tidb-binlog-cluster/)\n\n## Tutorial\n\nHere's a [tutorial](https://www.pingcap.com/docs/dev/how-to/get-started/tidb-binlog/) to experiment with TiDB-Binlog (not for production use).\n\n## Config File\n* Pump config file: [pump.toml](./cmd/pump/pump.toml) \n* Drainer config file: [drainer.toml](./cmd/drainer/drainer.toml) \n\n## Contributing\nContributions are welcomed and greatly appreciated. See [CONTRIBUTING.md](./CONTRIBUTING.md)\nfor details on submitting patches and the contribution workflow.\n\n## License\nTiDB-Binlog is under the Apache 2.0 license. See the [LICENSE](./LICENSE) file for details.\n\n\u003c!-- VERSION_PLACEHOLDER: v8.3.0-alpha --\u003e","funding_links":[],"categories":["Databases"],"sub_categories":["TiDB"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpingcap%2Ftidb-binlog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpingcap%2Ftidb-binlog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpingcap%2Ftidb-binlog/lists"}