https://github.com/elgca/kafka-connect-transform

https://github.com/elgca/kafka-connect-transform

kafka kafka-connect transform

Last synced: 25 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/elgca/kafka-connect-transform
• Owner: elgca
• Created: 2018-11-19T02:48:36.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2018-12-06T07:33:58.000Z (about 6 years ago)
• Last Synced: 2024-10-15T02:05:34.628Z (2 months ago)
• Topics: kafka, kafka-connect, transform
• Language: Scala
• Size: 56.6 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# kafka-connect-transform

## UnwrapDebeziumCDC

对Debezium的CDC数据进行flatten转换

- io.kafka.connect.UnwrapDebeziumCDC

- 对于insert/update数据保留after字段，并添加op和ts_ms

- 对于delete,保留原始数据,并添加op和ts_ms

```json

{

	"op": "u",

	"source": {

		...

	},

	"ts_ms" : ...,

	"before" : {

		"field1" : "oldvalue1",

		"field2" : "oldvalue2"

	},

	"after" : {

		"field1" : "newvalue1",

		"field2" : "newvalue2"

	}

}

```

```json

{

	"field1" : "newvalue1",

	"field2" : "newvalue2",

  "__operate" : "u"(op),

  "__time" : ...(ts_ms)

}

```