https://github.com/toanlcgift/dbsync

Sync postgresql/mysql to postgresql/mysql using debezium and kafka connect
https://github.com/toanlcgift/dbsync

apache-kafka change-data-capture docker event-sourcing highly-available k8s kafka kubernetes microservices ubuntu2204

Last synced: 6 months ago
JSON representation

Sync postgresql/mysql to postgresql/mysql using debezium and kafka connect

• Host: GitHub
• URL: https://github.com/toanlcgift/dbsync
• Owner: toanlcgift
• Created: 2023-08-22T08:53:47.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-06-30T06:26:58.000Z (over 1 year ago)
• Last Synced: 2025-02-03T09:58:06.433Z (8 months ago)
• Topics: apache-kafka, change-data-capture, docker, event-sourcing, highly-available, k8s, kafka, kubernetes, microservices, ubuntu2204
• Homepage:
• Size: 230 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# DBSync

Sync postgresql to postgresql using debezium and kafka connect, HA Kubernetes cluster

## Deploy Kafka on K8s on-premise

![image](https://github.com/toanlcgift/DBSync/assets/12400049/83d68616-427d-412c-abc5-99ec4892be21)

### Environment

K8s Node               | OS                              | RAM      | Total Processor Cores      | local IP | DNS |

-----------------------|-------------------------------------------|-----------------------------|-------------------------|-------------------------|-------------------------|

master             | `Ubuntu 22.04 x86_64 LTS` | 4GB | 2 | 192.168.123.123 | localk8s.com |

node1             | `Ubuntu 22.04 x86_64 LTS` | 4GB | 2 | 192.168.123.124 |  localnodek8s.com |

### Install kubelet, kubeadm, kubectl

``` bash

sudo apt -y install curl apt-transport-https

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt update

sudo apt -y install vim git curl wget kubelet kubeadm kubectl

sudo apt-mark hold kubelet kubeadm kubectl

sudo modprobe overlay

sudo modprobe br_netfilter

sudo tee /etc/sysctl.d/kubernetes.conf<