https://github.com/danielpetisme/kstream-ktable-join-edge-cases-master

https://github.com/danielpetisme/kstream-ktable-join-edge-cases-master

Last synced: 5 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/danielpetisme/kstream-ktable-join-edge-cases-master
• Owner: danielpetisme
• Created: 2020-03-18T10:55:15.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2022-01-04T16:40:11.000Z (almost 3 years ago)
• Last Synced: 2024-10-19T21:18:37.795Z (28 days ago)
• Language: Java
• Size: 63.5 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 3
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# kstream-ktable-join-edge-cases

## TL;DR

* Kafka Stream join are not deterministic in sub-topology context (ie. repartitioning before a join)

* `.through()` create a new stream within the current topology thus can't solve the above point

* Whenever there is a repartition operation (`selectKey`, `transform`, etc.) you shouldn't chain a `join` operation. (ie. create dedicated topologies)

* Timestamp synchronizationis dependant of the order Kafka subscriptions polling order thus the timestamp comparison should be strict (ie. `<` strict not `<=`).

  

## About

A suite of KStream-KTable join edge cases spotted during the Stock Booster (SEB).

Have a look to the test `KStreamKTableJoinWithTimestampExtractorTest`.

## Usage

The project runs with Java 11 and build with Maven project.

Uses [HAP config files](https://gitlab.michelin.com/DEV/config-file) to configure Michelin Artifactory repositories if you can't access to the Internet.

```

./mvnw clean test

```