https://github.com/dbgjerez/workshop-cdc
Worshop Change Data Capture with AMQ Streams and Camel
https://github.com/dbgjerez/workshop-cdc
amq-streams camel cdc change-data-capture kafka kafka-connect openshift quarkus reactjs
Last synced: about 1 year ago
JSON representation
Worshop Change Data Capture with AMQ Streams and Camel
- Host: GitHub
- URL: https://github.com/dbgjerez/workshop-cdc
- Owner: dbgjerez
- Created: 2023-10-06T13:19:39.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-12-12T06:39:06.000Z (over 2 years ago)
- Last Synced: 2025-03-28T14:22:23.169Z (about 1 year ago)
- Topics: amq-streams, camel, cdc, change-data-capture, kafka, kafka-connect, openshift, quarkus, reactjs
- Language: JavaScript
- Homepage:
- Size: 75.5 MB
- Stars: 4
- Watchers: 3
- Forks: 4
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Change Data Capture (CDC) is a design pattern that captures changes made at a source and replicates them to a destination.
In this example, we'll move data from some databases to a Kafka topic, the data will be transformed and saved into another destination database.
# Components
We will use a lot of components and tools to conduct this workshop. The main components are:
* **Kafka:** A distributed streaming platform used to build real-time data pipelines and streaming apps.
* **Kafka Connect:** An integration framework for connecting Kafka with external systems such as databases, key-value stores, and search indexes.
* **Camel:** An open-source integration framework that provides a standardized way to connect to different protocols and technologies.
* **Quarkus:** A Kubernetes-native Java framework tailored for GraalVM and HotSpot, offering fast boot times and low memory usage.
* **ReactJS:** A JavaScript library for building user interfaces, particularly known for its efficient rendering and the concept of component-based architecture."
# Start up
Operator installation:
```bash
oc apply -f gitops/bootstrap/openshift-gitops.yaml
```
Retrieve ArgoCD route:
```bash
oc get route -A | grep openshift-gitops-server | awk '{print $3}'
```
Get the ArgoCD admin password:
```bash
oc -n openshift-gitops get secret openshift-gitops-cluster -o json | jq -r '.data["admin.password"]' | base64 -d
```
ArgoCD needs some privileges to create specific resources. In this demo, we'll apply cluster-role to ArgoCD to avoid the fine-grain RBAC.
```bash
oc apply -f gitops/bootstrap/cluster-role.yaml
```
Now, we apply the bootstrap application:
```bash
oc apply -f gitops/bootstrap/bootstrap.yaml
```