{"id":15002571,"url":"https://github.com/cqllum/db2ch","last_synced_at":"2025-09-12T17:42:37.228Z","repository":{"id":242245314,"uuid":"808087310","full_name":"cqllum/db2ch","owner":"cqllum","description":"⚡Replicate your Database to Clickhouse in real time without Debezium, Kafka, or Clickhouse Materialized Engines  ","archived":false,"fork":false,"pushed_at":"2024-06-01T21:28:01.000Z","size":43,"stargazers_count":5,"open_issues_count":6,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-05T22:42:12.593Z","etag":null,"topics":["cdc","clickhouse","go","golang","mssql","mysql","postgresql","realtime","realtime-framework","replication"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cqllum.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-05-30T11:03:58.000Z","updated_at":"2025-03-11T09:06:33.000Z","dependencies_parsed_at":"2025-04-23T15:16:53.566Z","dependency_job_id":"bf69210f-75ff-42a6-aeb3-b6b35f326688","html_url":"https://github.com/cqllum/db2ch","commit_stats":null,"previous_names":["cqllum/db2ch"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cqllum/db2ch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cqllum%2Fdb2ch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cqllum%2Fdb2ch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cqllum%2Fdb2ch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cqllum%2Fdb2ch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cqllum","download_url":"https://codeload.github.com/cqllum/db2ch/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cqllum%2Fdb2ch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266267290,"owners_count":23902334,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cdc","clickhouse","go","golang","mssql","mysql","postgresql","realtime","realtime-framework","replication"],"created_at":"2024-09-24T18:51:11.171Z","updated_at":"2025-07-21T08:33:07.173Z","avatar_url":"https://github.com/cqllum.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"![db2ch](https://i.imgur.com/Vq4PBVd.png)\n\n### VERSION 0.0 - Not available for Production - POC Development\n\n## ⚠️Warning\n`db2ch` is still under heavy development. We reccommend that you not try to use it for production use until development has progressed more and bugs are less likely to occur.\n\n# db2ch\n\n`db2ch` is an open-source framework designed to simplify Change Data Capture (CDC) to ClickHouse without the need for complex setups like Debezium, Kafka, or ClickHouse Materialized Views. This project supports real-time data replication from PostgreSQL, MySQL, and MSSQL to ClickHouse.\n\n\n## Features\n\n- **Real-time Replication**: Efficiently stream data changes from PostgreSQL, MySQL, and MSSQL to ClickHouse in real-time.\n- **Simplicity**: No need for additional tools like Debezium, Kafka, or ClickHouse Materialized Engines.\n- **High Throughput**: Built with Go, ensuring high performance and low latency.\n- **Control and Monitor**: HTTP APIs for controlling and monitoring replication processes.\n- **Automatic Table Creation**: `db2ch` will automatically map source data types to Clickhouse data types and create tables.\n- **Primary Key Based Backfill Capability**: `db2ch` also includes a capability to perform a backfil to bring across historic data.\n- **Column Control**: `db2ch` replication allows you to control which columns you bring across.\n\n## Project Structure\n```\ndb2ch/\n├── main.go\n├── config/\n│ ├── config.go\n│ ├── config.json\n├── api/\n│ ├── handler.go\n│ └── routes.go\n├── replication/\n│ ├── replication.go\n│ ├── clickhouse.go\n│ ├── mysql.go\n│ └── mssql.go\n└── go.mod\n```\n### Folder Descriptions\n- `config/` - The purpose of this directory is define configuration details (connection details, data mapping, http port, etc).\n- `api/` - Contains the `routes` for HTTP API endpoints and a handler for core api functionality.\n- `replication/` - The framework behind listening to data and producing data.\n\n## Getting Started\n\n### Prerequisites\n\n- Go 1.16+ installed\n- PostgreSQL, MySQL, and MSSQL databases set up\n- ClickHouse set up\n\n### Installation\n\n1. Clone the repository:\n\n    ```sh\n    git clone https://github.com/cqllum/db2ch.git\n    cd db2ch\n    ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcqllum%2Fdb2ch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcqllum%2Fdb2ch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcqllum%2Fdb2ch/lists"}