https://github.com/thunchanokbow/cinema-data-streaming-with-kafka
change data capture (CDC) to ingest binlogs from the source database, process them, and write them to the target database correctly, completely, and instantly in real time. using docker-compose,Kafka connectors (debezium, jdbc sink), compute engine, cloud sql, database mysql, confluent.
https://github.com/thunchanokbow/cinema-data-streaming-with-kafka
cdc cloudsql computeengine debezium docker-compose jdbc kafka ksqldb mysql sql
Last synced: 4 months ago
JSON representation
change data capture (CDC) to ingest binlogs from the source database, process them, and write them to the target database correctly, completely, and instantly in real time. using docker-compose,Kafka connectors (debezium, jdbc sink), compute engine, cloud sql, database mysql, confluent.
- Host: GitHub
- URL: https://github.com/thunchanokbow/cinema-data-streaming-with-kafka
- Owner: thunchanokbow
- Created: 2024-01-16T12:00:16.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-03-06T06:47:17.000Z (over 1 year ago)
- Last Synced: 2025-01-09T09:58:59.281Z (5 months ago)
- Topics: cdc, cloudsql, computeengine, debezium, docker-compose, jdbc, kafka, ksqldb, mysql, sql
- Language: HTML
- Homepage:
- Size: 101 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Cinema Data Streaming with Kafka
Build a data synchronization system between databases with **Kafka** that implements **change data capture (CDC)** to ingest binlogs from the source database, process them, and write them to the target database correctly, completely, and instantly in real time.
## Project Overview
## Contents
[Cloud SQL for MySQL database](sections/01-cloud-sql-for-mysql-database.md).
- [Hosting MySQL on Cloud SQL](sections/01-cloud-sql-for-mysql-database.md#Hosting-MySQL-on-Cloud-SQL).
- [Connect via DBeaver](sections/01-cloud-sql-for-mysql-database.md#Connect-via-DBeaver).
[Compute Engine](sections/02-compute-engine.md).
- [Create VM Instances](sections/02-compute-engine.md#Create-VM-Instances).
- [Install Git and Docker on a new VM](sections/02-compute-engine.md#Install-Git-and-Docker-on-a-new-VM).
- [Upload YAML files to VM ](sections/02-compute-engine.md#Upload-YAML-files-to-VM).
- [Get Confluent Control Center](sections/02-compute-engine.md#Get-Confluent-Control-Center).
[Connect Debezium connector to Cloud SQL for MySQL](sections/03-connect-debezium-connector-to-cloud-sql-for-mysql.md#Connect-Debezium-connector-to-Cloud-SQL-for-MySQL).
[Connect JDBC sink connector to CloudSQL for MySQL](sections/03-connect-debezium-connector-to-cloud-sql-for-mysql.md#Connect-JDBC-sink-connector-to-CloudSQL-for-MySQL).
- [How to fix error message "No suitable driver found"](sections/03-connect-debezium-connector-to-cloud-sql-for-mysql.md#Connect-JDBC-sink-connector-to-CloudSQL-for-MySQL)