https://github.com/perkss/kotlin-kafka-and-kafka-streams-examples

Kafka with KafkaReactor and Kafka Streams Examples in Kotlin
https://github.com/perkss/kotlin-kafka-and-kafka-streams-examples

coroutines docker docker-compose java kafka kafka-streams kotlin reactive-programming reactive-streams reactor reactor-core reactor-kafka testcontainer testcontainers webflux websocket

Last synced: 11 days ago
JSON representation

Kafka with KafkaReactor and Kafka Streams Examples in Kotlin

• Host: GitHub
• URL: https://github.com/perkss/kotlin-kafka-and-kafka-streams-examples
• Owner: perkss
• Created: 2018-10-12T07:21:01.000Z (about 6 years ago)
• Default Branch: main
• Last Pushed: 2024-09-21T07:16:34.000Z (about 2 months ago)
• Last Synced: 2024-10-12T06:23:37.628Z (27 days ago)
• Topics: coroutines, docker, docker-compose, java, kafka, kafka-streams, kotlin, reactive-programming, reactive-streams, reactor, reactor-core, reactor-kafka, testcontainer, testcontainers, webflux, websocket
• Language: Kotlin
• Homepage:
• Size: 440 KB
• Stars: 52
• Watchers: 3
• Forks: 12
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Reactive Kotlin Kafka and Kafka Streams with Kotlin

Kafka and Kafka Stream Examples in Kotlin with Project Reactor Kafka

Please check out the blog posts for more details.

* [Kotlin Kafka Streams](https://perkss.github.io/#/DistributedSystems/Streaming#KafkaStreamsKotlin)

## Building Locally

As the integration tests run with docker you are required to have docker running locally. 

Then simply execute the maven command you wish.

`mvn clean install`

## Docker Environment

Please use the docker-compose file in the root of each module project to create the Kafka Brokers and Zookeeper and

where appropriate the Schema Registry.

Please check the directory README for details how to run this example.

## Integration Tests

Integration tests can be found for each module project and these require Docker to be running and

use [Testcontainers](https://www.testcontainers.org/) these are powerful tests that fire up Kafka instances and our

application and fully test the flow of messages through our streaming application.

## Kafka Reactive Producer Consumer

This example shows how you can use the reactive API to build a consumer from a `lowercase-topic` map the data and output

it with the same key to a `uppercase-topic` with the data converted to uppercase. Please check the sub module README for

how to execute this. It is a very interesting yet simple example, as you can see when the consume is lazily instantiated

when it connects and then once a message is received it lazily instantiates the producer to send on.

## Kafka Reactive Secure Producer Consumer

Shows how you can run a secured broker cluster using TLS and a application that will will consume and produce with this

secure transport layer to the brokers. Details can be found in the sub folder README.

## Kafka Streams and Kotlin Examples

This module is for examples of using Kafka Streams with Kotlin and Avro. Here we build a stock ordering system that has

the concept of customers to place orders. We use Avro to define schemas for the main topics and use changelog tables to

store down product and customer information which is joined to the OrderRequests. This module depends on the Avro code

generation in the `avro-schemas` module so that needs building before compiling this module.

### Kafka Streams Examples

We also have a package for examples to show the following

features [found here.](kotlin-kafka-streams-examples/src/main/kotlin/com/perkss/kafka/reactive/examples)

* Windowing

* Aggregates

* Joins