https://github.com/adrien-ben/kstream-sales-aggregation-example
A simple application aggregating the data of one topic using Kafka Stream written in Kotlin
https://github.com/adrien-ben/kstream-sales-aggregation-example
java11 kafka kafka-streams spring-boot
Last synced: about 1 year ago
JSON representation
A simple application aggregating the data of one topic using Kafka Stream written in Kotlin
- Host: GitHub
- URL: https://github.com/adrien-ben/kstream-sales-aggregation-example
- Owner: adrien-ben
- Created: 2019-04-04T14:30:14.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2019-07-11T07:27:08.000Z (almost 7 years ago)
- Last Synced: 2025-03-24T14:44:20.067Z (about 1 year ago)
- Topics: java11, kafka, kafka-streams, spring-boot
- Language: Java
- Homepage:
- Size: 1.32 MB
- Stars: 6
- Watchers: 1
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Kafka Stream Sales Aggregation Example
[](https://travis-ci.com/adrien-ben/kstream-sales-aggregation-example)
A simple application written in Kotlin using Spring and Kafka Stream to aggregate the data of a topic.
## What it does
The application listens to an kafka topic containing very simple sale data that contains the id of a shop
and the amount of a sale. The key of the kafka message is the `shopId`.
```json
{
"shopId": "SHOP01",
"amount": 10.2
}
```
This messages are then aggregated before being sent back to another kafka topic. You can control the size of the
aggregation window by setting the `app.window.duration` parameter to any value (5m for 5 minutes for example) or
set it to 0 if you don't want to aggregate the sale just in a timed window.
If set to 0 the updated aggregated value for a given shop will be sent to the topic at a regular interval.
```json
{
"shopId": "SHOP01",
"amount": 124.4
}
```
If a size if configured for the window then the aggregated value for a given shop will be sent once when the window
gets closed. The output message will also contain the start and end date/time of the window.
```json
{
"shopId": "SHOP01",
"amount": 71.4,
"periodStart": "2019-04-04T17:21:35",
"periodEnd": "2019-04-04T17:21:40"
}
```
> See [this link][0] for more information on how to just output the final value of the windowed aggregation.
## Topologies
### Without window
### With window
## Run application
```sh
# Run app
./mvnw spring-boot:run
# Run tests (note the clean to ensure the state store is cleaned up)
./mvnw clean test
```
You will also need a running Kafka instance. If you have docker installed this project has a
docker-compose file that starts a zookeeper, a kafka broker and kafkahq which is a nice ui
that ease the interaction with the kafka broker and that you can access at localhost:8090.
```sh
docker-compose up -d
```
[0]: https://docs.confluent.io/current/streams/developer-guide/dsl-api.html#window-final-results