Projects in Awesome Lists tagged with cdc

https://github.com/debezium/debezium

Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

apache-kafka cdc change-data-capture data-pipeline database debezium event-streaming kafka kafka-connect kafka-producer

Last synced: 09 Sep 2025

https://github.com/apache/seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.

apache batch cdc change-data-capture data-ingestion data-integration elt high-performance offline real-time streaming

Last synced: 12 May 2025

https://github.com/supabase/realtime

Broadcast, Presence, and Postgres Changes via WebSockets

cdc change-data-capture crdt distributed-systems elixir phoenix phoenix-framework postgres postgresql pubsub realtime

Last synced: 05 Jan 2026

https://github.com/apache/flink-cdc

Flink CDC is a streaming data integration tool

batch cdc change-data-capture data-integration data-pipeline distributed elt etl flink kafka mysql paimon postgresql real-time schema-evolution

Last synced: 12 May 2025

https://github.com/peerdb-io/peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

bigquery cdc clickhouse cloud-native distributed-systems etl eventhubs kafka postgres postgresql realtime rust s3 snowflake sql stream-processing

Last synced: 14 May 2025

https://github.com/disease-sh/API

API for Current cases and more stuff about COVID-19 and Influenza

api cdc corona coronavirus coronavirus-tracking covid covid-19 covid19 discord discord-server disease postman redis

Last synced: 03 Apr 2025

https://github.com/disease-sh/api

API for Current cases and more stuff about COVID-19 and Influenza

api cdc corona coronavirus coronavirus-tracking covid covid-19 covid19 discord discord-server disease postman redis

Last synced: 14 May 2025

https://github.com/PeerDB-io/peerdb

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

bigquery cdc clickhouse cloud-native distributed-systems etl eventhubs kafka postgres postgresql realtime rust s3 snowflake sql stream-processing

Last synced: 28 Mar 2025

https://github.com/confluentinc/examples

Apache Kafka, Apache Flink and Confluent Platform examples and demos

avro cdc cloud confluent connect connector debezium demo docker examples jdbc kafka kubernetes microservices monitoring quickstart replicator schema-registry sql

Last synced: 05 Oct 2025

https://github.com/cherry-embedded/cherryusb

CherryUSB is a tiny and beautiful, high performace and portable USB host and device stack for embedded system with USB IP

cdc cdns2 cdns3 cherryusb dfu dwc2 ehci hid hub musb ohci rndis uac usb usb-device usb-host uvc xhci

Last synced: 14 May 2025

https://github.com/eulerto/wal2json

JSON output plugin for changeset extraction

cdc json plugin postgres replication

Last synced: 14 May 2025

https://github.com/cherry-embedded/CherryUSB

CherryUSB is a tiny, beautiful and portable USB host and device stack for embedded system with USB IP

cdc cdns2 cdns3 cherryusb dfu dwc2 ehci hid hub musb ohci rndis uac usb usb-device usb-host uvc xhci

Last synced: 26 Mar 2025

https://github.com/datavane/tis

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI

cdc chunjun dataops datax etl flink flink-streaming java

Last synced: 25 Dec 2025

https://github.com/hazelcast/hazelcast-jet

Distributed Stream and Batch Processing

batch-processing big-data cdc event-processing hacktoberfest java kafka low-latency stream-processing

Last synced: 17 Oct 2025

https://github.com/2ndQuadrant/pglogical

Logical Replication extension for PostgreSQL 17, 16, 15, 14, 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.

cdc data-transformation data-transport database-replication etl logical-decoding postgresql publish-subscribe replication subscription zero-downtime

Last synced: 30 Mar 2025

https://github.com/jumpmind/symmetric-ds

SymmetricDS is database replication and file synchronization software that is platform independent, web enabled, and database agnostic. It is designed to make bi-directional data replication fast, easy, and resilient. It scales to a large number of nodes and works in near real-time across WAN and LAN networks.

cdc data-replication database db2 firebird h2-database informix java mysql oracle platform-independent postgresql replication sqlserver sybase synchronization

Last synced: 14 May 2025

https://github.com/JumpMind/symmetric-ds

SymmetricDS is database replication and file synchronization software that is platform independent, web enabled, and database agnostic. It is designed to make bi-directional data replication fast, easy, and resilient. It scales to a large number of nodes and works in near real-time across WAN and LAN networks.

cdc data-replication database db2 firebird h2-database informix java mysql oracle platform-independent postgresql replication sqlserver sybase synchronization

Last synced: 30 Mar 2025

https://github.com/artie-labs/transfer

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift, Databricks) in real-time.

apache-kafka bigquery cdc change-data-capture data-integration data-pipelines database debezium elt golang kafka redshift snowflake

Last synced: 28 Dec 2025

https://github.com/WangXuan95/FPGA-USB-Device

An FPGA-based USB full-speed device core to implement USB-serial, USB-camera, USB-audio, USB-hid, etc. It requires only 3 FPGA common IOs rather than additional chips. 基于FPGA的USB full-speed device端控制器，可实现USB串口、USB摄像头、USB音频、U盘、USB键盘等设备，只需要3个FPGA普通IO，而不需要额外的接口芯片。

cdc fpga keyboard rtl usb usb-audio usb-camera usb-cdc usb-controller usb-device usb-disk usb-hid usb-keyboard usb-microphone usb-serial usb-speaker usb-uart usb-uvc uvc verilog

Last synced: 22 Apr 2025

https://github.com/dotnetcore/canalsharp

Alibaba mysql database binlog subscription & consumer components Canal's .NET client.

binlog canal canal-client cdc mysql netcore netstandard20 real-time

Last synced: 14 May 2025

https://github.com/dotnetcore/CanalSharp

Alibaba mysql database binlog subscription & consumer components Canal's .NET client.

binlog canal canal-client cdc mysql netcore netstandard20 real-time

Last synced: 04 May 2025

https://github.com/xtoolbox/TeenyUSB

Lightweight USB device and host stack for STM32 and other MCUs. Ready for USB 3.0 device.

cdc ch56x composite embedded hid host hub msc rndis speed stm32 super usb usb3 winusb

Last synced: 04 May 2025

https://github.com/actiontech/dtle

Distributed Data Transfer Service for MySQL

cdc change-data-capture data-migration data-transfer mysql replication

Last synced: 16 May 2025

https://github.com/pingcap/tiflow

This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)

cdc dm kafka mysql ticdc tidb

Last synced: 13 May 2025

https://github.com/apecloud/ape-dts

ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data replication between MySQL, PostgreSQL, Redis, MongoDB, Kafka and ClickHouse, ideal for disaster recovery (DR) and migration scenarios.

binlog cdc clickhouse data-migration data-transfer doris etl kafka-connect mongo mysql postgres redis starrocks

Last synced: 16 May 2025

https://github.com/mongodb/mongo-kafka

MongoDB Kafka Connector

avro bson cdc confluent-hub kafka kafka-connect mongodb sink sink-connector source source-connector

Last synced: 11 Nov 2025

https://github.com/uber/storagetapper

StorageTapper is a scalable realtime MySQL change data streaming, logical backup and logical replication service

avro cdc clickhouse etl hdfs json kafka msgpack mysql postgresql s3

Last synced: 11 Jun 2025

https://github.com/memiiso/debezium-server-iceberg

Low Cost, Simple and Scalable Way of Data Replication to Apache Iceberg/Cloud/Data Lake

batch cdc debezium iceberg iceberg-table realtime

Last synced: 24 Oct 2025

https://github.com/replicase/pgcapture

A scalable Netflix DBLog implementation for PostgreSQL

backup cdc change-data-capture database dblog event-driven grpc logical-replicas migration netflix-dblog pglogical postgres postgresql queue replication

Last synced: 10 Oct 2025

https://github.com/altinity/clickhouse-sink-connector

Replicate data from MySQL, Postgres and MongoDB to ClickHouse®

avro cdc clickhouse debezium kafka kafka-connect mongo mysql postgresql replication sqlserver

Last synced: 16 May 2025

https://github.com/dpretet/async_fifo

A dual clock asynchronous FIFO written in verilog, tested with Icarus Verilog

asic asic-design async cdc cross-clock-domain fifo fifo-cache fifo-queue fpga hdl icarus-verilog synthesis verification verilator verilog verilog-hdl

Last synced: 22 Apr 2025

https://github.com/Altinity/clickhouse-sink-connector

Replicate data from MySQL, Postgres and MongoDB to ClickHouse®

avro cdc clickhouse debezium kafka kafka-connect mongo mysql postgresql replication sqlserver

Last synced: 03 May 2025

https://github.com/cpursley/walex

Postgres change events (CDC) in Elixir

cdc change-data-capture debezium elixir postgres postgresql replication

Last synced: 04 Apr 2025

https://github.com/neo4j-contrib/neo4j-streams

Neo4j Kafka Connector

cdc change-data-capture graph-database hacktoberfest kafka kafka-producer neo4j stream-processing

Last synced: 12 Apr 2025

https://github.com/clickhouse/examples

ClickHouse Examples

cdc crypto docker ethereum observability

Last synced: 04 Apr 2025

https://github.com/ClickHouse/examples

ClickHouse Examples

cdc crypto docker ethereum observability

Last synced: 07 Mar 2025

https://github.com/hpgrahsl/kafka-connect-mongodb

**Unofficial / Community** Kafka Connect MongoDB Sink Connector -> integrated 2019 into the official MongoDB Kafka Connector here: https://www.mongodb.com/kafka-connector

avro azure-cosmosdb bson cdc change-data-capture confluent-hub connector cosmosdb debezium json kafka kafka-connect mongodb sink sink-connector

Last synced: 12 Jun 2025

https://github.com/supabase/walrus

Applying RLS to PostgreSQL WAL

cdc json postgres replication

Last synced: 12 Aug 2025

https://github.com/rusuly/MySqlCdc

MySQL/MariaDB binlog replication client for .NET

binlog cdc change-data-capture dotnet mariadb mysql real-time replication

Last synced: 04 May 2025

https://github.com/linkedin/hoptimator

Multi-hop declarative data pipelines

brooklin cdc data-pipelines flink kafka kafka-connect

Last synced: 06 Jan 2026

https://github.com/izhangzhihao/real-time-data-warehouse

Real-time Data Warehouse with Apache Flink & Apache Kafka & Apache Hudi

cdc change-data-capture data-warehouse data-warehousing datalake debezium delta delta-lake deltalake elasticsearch flink flink-sql hoodie hudi iceberg kafka real-time-data-warehouse spark spark-sql sql

Last synced: 07 Sep 2025

https://github.com/itv/scala-pact

A Scala implementation of CDC using the Pact standard

cdc pact scala streaming-library testing testing-tools

Last synced: 06 Apr 2025

https://github.com/kibae/pg-logical-replication

PostgreSQL Logical Replication client for node.js

cdc decoderbufs logical-decoding logical-replication nodejs pgoutput postgres postgresql typescript wal2json

Last synced: 16 May 2025

https://github.com/transferia/transferia

Open Source Cloud Native Ingestion engine

bigdata cdc clickhouse elt go golang ingestion-platform kafka streaming

Last synced: 03 Apr 2025

https://github.com/Azure-Samples/azure-sql-db-change-stream-debezium

SQL Server Change Stream sample using Debezium

azure-functions azure-sql cdc change-data-capture change-feed change-streams debezium event-hubs kafka sql-server

Last synced: 11 May 2025

https://github.com/disneystreaming/pg2k4j

Postgresql To Kinesis For Java

cdc java kinesis kinesis-stream logical-replication postgresql replication

Last synced: 05 Apr 2025

https://github.com/polardb/polardbx-cdc

polardbx-cdc is a core component of PolarDB-X which is responsible for global binary log generation, publication and subscription.

binlog cdc change-master global-binary-log mysql mysql-binlog mysql-dump mysqlbinlog replication

Last synced: 05 Apr 2025

https://github.com/redis-developer/redis-microservices-demo

Microservice application with various Redis use-cases with RediSearch, RedisGraph and Streams. The data are synchronize between MySQL and Redis using Debezium as a CDC engine

cdc debezium graph microservices mysql rdbms redis search streams

Last synced: 12 Apr 2025

https://github.com/paradigmxyz/sinker

Synchronize Postgres to Elasticsearch

cdc elasticsearch postgres

Last synced: 06 Apr 2025

https://github.com/trendyol/go-pq-cdc

go-pq-cdc is lightweight and efficient Change Data Capture (CDC) system designed for PostgreSQL

cdc go golang postgresql

Last synced: 01 Jul 2025

https://github.com/hrbrmstr/cdcfluview

:mask: R package to Retrieve U.S. Flu Season Data from the CDC FluView Portal (WHO & ILINet)

cdc cdc-fluview-portal national-influenza-statistics r rstats

Last synced: 16 Mar 2025

https://github.com/1ambda/lakehouse

Playground for Lakehouse (Iceberg, Hudi, Spark, Flink, Trino, DBT, Airflow, Kafka, Debezium CDC)

airflow cdc dbt debezium docker flink hudi iceberg kafka spark trino

Last synced: 25 Oct 2025

https://github.com/scylladb/scylla-cdc-source-connector

A Kafka source connector capturing Scylla CDC changes

apache-kafka cdc change-data-capture debezium event-streaming java kafka kafka-connect kafka-producer nosql scylla

Last synced: 20 Jun 2025

https://github.com/jwplayer/southpaw

:baseball: Streaming left joins in Kafka for change data capture

apache-kafka cdc change-data-capture debezium distributed-systems java kafka rocksdb southpaw

Last synced: 20 Sep 2025

https://github.com/pactflow/pact-msw-adapter

Create MSW (mock-service-worker) mocks, and generate pact contracts from the recorded interactions.

cdc contract javascript mock-service-worker msw mswjs pact testing

Last synced: 04 Apr 2025

https://github.com/redis-field-engineering/redis-connect-dist

Real-Time Event Streaming & Change Data Capture

cdc connect csv etl etl-automation etl-framework etl-pipeline event-sourcing event-streaming redis rediscdc redisconnect replication

Last synced: 20 Aug 2025

https://github.com/cdcgov/rnssp

A Signature R package for the National Syndromic Surveillance Program (NSSP) at the Centers for Disease Control and Prevention (CDC). A collection of tools, classes and functions that supports the Community of Practice of the NSSP.

cdc essence essence-api nssp nssp-cop

Last synced: 21 Aug 2025

https://github.com/datastax/cdc-apache-cassandra

Datastax CDC for Apache Cassandra

cassandra cdc change-data-capture datastax pulsar

Last synced: 29 Apr 2025

https://github.com/chobostar/pg_listener

Go application to capture inserts into PostgreSQL table and produce events to Apache Kafka using replication slot and WAL

cdc go golang kafka postgres postgresql wal2json

Last synced: 23 Jul 2025

https://github.com/metakgp/mftp

CDC noticeboard scraper

career-development-center cdc erp hacktoberfest iitkgp-career-development-center iitkgp-erp iitkgp-erp-cdc iitkgp-erp-tnp iitkgp-training-and-placement internship mftp my-freaking-training-and-placement placement tnp training-and-placement

Last synced: 07 Apr 2025

https://github.com/airscholar/changecapture-e2e

This project shows how to capture changes from postgres database and stream them into kafka

apache-spark cdc debezium docker kafka postgres zookeeper

Last synced: 29 Oct 2025

https://github.com/cdcgov/seqsender

Automated Pipeline to Generate FTP Files and Manage Submission of Sequence Data to Public Repositories

bioinformatics-pipeline biosample cdc cdc-influenza-division genbank gisaid gisaid-format gisaid-upload gisaid-uploader ncbi-biosamples ncbi-genbank ncbi-sra ncbi-submission ncird ncird-id seqsender sra

Last synced: 02 Jan 2026

https://github.com/artie-labs/reader

Perform historical snapshots without database locks and read change data capture logs from databases. Artie Reader is compatible with Debezium and is written in Go.

apache-kafka cdc change-data-capture data-integration database debezium golang kafka

Last synced: 16 May 2025

https://github.com/garylee/stm32f103_dual_vcp

An example to show how to use dual CDC VCP USB interfaces.

cdc stm32 uart usb vcp

Last synced: 11 Oct 2025

https://github.com/hexnn/stark

基于Spark+SparkMLlib+Debezium+Deequ打造的简单易用、超高性能大数据治理引擎。适用于批流一体的数据集成和数据分析，支持机器学习算法模型、CDC实时数据采集、数据质量校验、数据标注、敏感数据识别、数据建模、算法建模和OLAP数据分析

cdc dataworks datax debezium deequ etl flink hadoop kettle mllib seatunnel spark sparkml sparkmllib

Last synced: 13 Jun 2025

https://github.com/chucheng92/hadoopdedup

:watermelon:基于Hadoop和HBase的大规模海量数据去重

big-data cdc dedup fsp mapreduce

Last synced: 15 Apr 2025

https://github.com/jana-marie/stm32g4xx_usb_cdc

STM32G474 USB CDC example implementation, combined with Jan Breuer's SCPI parser library v2

cdc otter scpi stm32 stm32g4 stm32g474 usb usb-cdc

Last synced: 23 Oct 2025

https://github.com/green-coder/cdc

A library for performing Content-Defined Chunking (CDC) on data streams.

cdc data-stream rust rust-library

Last synced: 16 Mar 2025

https://github.com/earthobservations/phenodata

An acquisition and processing toolkit for open access phenology data.

biology cdc data dwd dwd-ftp earth-observation earth-observations earthobservation open-data opendata phenological-metrics phenology phenology-data phenology-modelling phenology-models phenometrics plant research research-data research-tool

Last synced: 05 Oct 2025

https://github.com/zabuzard/fastcdc4j

Fast and efficient content-defined chunking for data deduplication. Java implementation of FastCDC as library.

cdc chunking content-defined-chunking data-deduplication fastcdc java library

Last synced: 19 Jul 2025

https://github.com/evanthegrayt/cdc

☣️ Shell plugin for zsh/bash that allows you to cd to subdirectories of user-defined directories from anywhere, without editing CDPATH.

bash bash-it cdc cdpath directory function oh-my-zsh plugin productivity tab-completion zsh

Last synced: 21 Jul 2025

https://github.com/dan-da/cdc_audit

change data capture via audit tables and triggers for mysql.

audit-tables cdc change-data-capture commandline data-warehouse mysql triggers

Last synced: 12 Jul 2025

https://github.com/kutukvpavel/esp32-win7-vcp-drivers

INF files that enable ESP32-S3 Serial/JTAG peripheral and TinyUSB CDC stack on Win7

cdc driver drivers esp32 serialport win7 win8 windows-7 windows-8-1

Last synced: 27 Oct 2025

https://github.com/gh0stblade/cdcenginetools

360 bigfile cdc cdrm crystal drm dynamics engine extract mod pc ps2 ps3 psp raider tomb tools wii xbox xenon

Last synced: 16 Apr 2025

https://github.com/cdcgov/cdcgov.github.io

Portal site for all CDCgov projects, https://cdcgov.github.io

cdc center-innovation collaboration discovery portal web

Last synced: 21 Feb 2025

https://github.com/cdcgov/ephtrackr

R interface to the CDC National Public Health Tracking Network API

cdc environmental-data epidemiology publichealth r rstats

Last synced: 22 Apr 2025

https://github.com/dralshehri/epiweeks

Epidemiological weeks calculation based on the US CDC (MMWR) and ISO week numbering systems

calendar cdc epidemiology iso mmwr public-health python surveillance weeks

Last synced: 29 Dec 2025

https://github.com/xmlking/cdc-kafka-hadoop

MySQL to NoSQL real time dataflow

architecture cdc change-data-capture data-flow debezium groovy hadoop kafka maxwell mysql nifi

Last synced: 27 Jul 2025

https://github.com/z4yx/osx-ch341-serial

CH340/CH341 based USB to Serial Port Adapter driver for Mac OSX

cdc driver osx serialport

Last synced: 10 Sep 2025

https://github.com/juliahealth/icd_gems.jl

ICD_GEMs.jl is a Julia package that allows to translate ICD-9 codes in ICD-10 and viceversa via the General Equivalence Mappings (GEMs) of the International Classification of Diseases (ICD).

cdc clinical-data clinical-research data death-certificates epidemiology health-data icd-10 icd-10-cm icd-9 icd-codes julia julia-language julia-package mortality-data public-health who

Last synced: 22 Apr 2025

https://github.com/oceanbase/oblogclient

A Java library to get commit log data from OceanBase.

binlog cdc oblogproxy oceanbase

Last synced: 19 Jul 2025

https://github.com/siddhi-io/siddhi-io-cdc

Extension which consumes CDC events

cdc change-data-capture database debezium extension io siddhi

Last synced: 05 Apr 2025

https://github.com/sanchitvj/sports_betting_analytics_engine

A data and analytics engineering platform designed for real-time sports betting analytics.

airflow analytics-engineering aws betflow cdc data-engineering dbt druid grafana kafka lambda-architecture olap-database snowflake spark-streaming sports-betting

Last synced: 30 Oct 2025

https://github.com/avanade/ntangle

nTangle is a Change Data Capture (CDC) code generation tool and corresponding runtime. Unlike other CDC-based technologies which replicate changes to rows, nTangle is designed to replicate business entity (aggregate) changes.

cdc change-data-capture

Last synced: 21 Aug 2025

https://github.com/devlive-community/dameng-connector

Dameng Connector 是一个专门为达梦数据库(DM Database)设计的变更数据捕获(CDC)解决方案。该项目通过扩展 Debezium 和 Flink CDC，实现了对达梦数据库的实时数据变更监控、捕获和处理能力，为数据集成、数据同步、实时分析等场景提供强大支持。

cdc dameng dameng-exporter dameng8 debezium debezium-connector flink flink-cdc flink-connector

Last synced: 24 Apr 2025

https://github.com/cdcgov/rnssp-rmd-templates

Rnssp R Markdown templates are custom-built, parameterized R Markdown templates with seamless integration to the Rnssp R package to enhance user experiences and streamline workflows.

cdc essence essence-api nssp nssp-cop

Last synced: 22 Apr 2025