https://github.com/dyrnq/cdc-doris
a poc for cdc to doris.
https://github.com/dyrnq/cdc-doris
apache apache-flink cdc cdc-yaml debezium doris flink flink-stream-processing minio poc vagrantfile
Last synced: about 2 months ago
JSON representation
a poc for cdc to doris.
- Host: GitHub
- URL: https://github.com/dyrnq/cdc-doris
- Owner: dyrnq
- Created: 2024-11-30T05:31:05.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-02-11T05:18:48.000Z (over 1 year ago)
- Last Synced: 2025-03-16T23:56:45.438Z (over 1 year ago)
- Topics: apache, apache-flink, cdc, cdc-yaml, debezium, doris, flink, flink-stream-processing, minio, poc, vagrantfile
- Language: Shell
- Homepage: https://github.com/apache/doris
- Size: 21.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# cdc-doris
a poc for cdc to doris, and simplified version [cdc-vagrant](https://github.com/dyrnq/cdc-vagrant).
it has the following components:
- minio
- flink cluster
- zookeeper
- doris
- mysql
## quick start
```bash
vagrant up cdc
```
| service | url |
|--------------|-----------------------------|
| doris web | http://192.168.56.200:8030 |
| zoonavigator | http://192.168.56.200:39000 |
| minio web | http://192.168.56.200:9000 |
| flink web | http://192.168.56.200:48081 |
| adminer web | http://192.168.56.200:48080 |
## flink-cluster
Run a small flink-standalone cluster with zookeeper and minio.
```bash
vagrant ssh cdc
cd /vagrant/misc
./launch.sh -d
cd /vagrant/minio
./launch.sh -d
cd /vagrant/flink
./launch.sh -d
```
## doris
```bash
vagrant ssh cdc
cd /vagrant/doris
./launch.sh -d
sleep 10;
./init.sh
```
## flink-cdc
```bash
vagrant ssh cdc
cd /vagrant/cdc
./launch.sh -d
docker compose exec -it cdc bash
root@cdc:/opt/flink# /data/install-flink-cdc.sh && /data/cdc.sh
```
```bash
## architecture
minio (s3 cluster)
^
|
|
souce sink
mysql ----> flink standalone cluster ----> apache doris
192.168.56.200:3306 | 192.168.56.200:8030
|
v
zookeeper cluster
```