https://github.com/sombriks/sample-kafka
sampling how to work with spring-boot and apache kafka
https://github.com/sombriks/sample-kafka
docker-compose java javalin kafka kotlin spring-boot
Last synced: 3 months ago
JSON representation
sampling how to work with spring-boot and apache kafka
- Host: GitHub
- URL: https://github.com/sombriks/sample-kafka
- Owner: sombriks
- Created: 2023-07-10T22:13:43.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2023-08-07T19:45:27.000Z (almost 3 years ago)
- Last Synced: 2025-03-06T09:23:05.197Z (over 1 year ago)
- Topics: docker-compose, java, javalin, kafka, kotlin, spring-boot
- Language: Kotlin
- Homepage:
- Size: 209 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# service-broker-kafka
Small proof of concept on how to route log messages from one kafkaConsumerApp to a log
consumer through a kafka broker
## How kafka works
See [this](https://medium.com/swlh/apache-kafka-what-is-and-how-it-works-e176ab31fcd5)
## how to run the broker
- install docker, docker-compose
```bash
cd service-broker-kafka
docker compose -f docker-compose-dev.yml up
```
Install [intellij kafka plugin](https://plugins.jetbrains.com/plugin/21704-kafka)
or any other kafka client
Connect on port **9094**
## how to run the service-producer-1
- install java-jdk-17, some IDE (intellij is better)
```bash
cd service-producer-1
sh ./gradlew build ; sh ./gradlew bootRun
```
First producer uses default logback logging spring boot subsystem and does not
work yet. Messages to kafka streams must be sent in a explicit way
```kotlin
@Service
class SimpleJob(val template: KafkaTemplate) {
private val logger by lazy { LoggerFactory.getLogger(SimpleJob::class.java) }
@Scheduled(fixedRate = 5000)
fun work() {
val date = "${Date()}"
logger.info(date)
template.send("logs", date)
}
}
```
## how to run the service-producer-2
- install java-jdk-17, some IDE (intellij is better)
```bash
cd service-producer-2
sh ./gradlew build ; sh ./gradlew bootRun
```
Second producer uses log4j2 logging spring boot subsystem and logs messages to
a kafka topic in a transparent way. Just log your message and go home.
There is an issue where it starts to log warnings generated by kafka producer
itself, and then it enters into an infinite loop. I am working on it.
## how to run the service-consumer-1
- install java-jdk-17, some IDE (intellij is better)
```bash
cd service-consumer-1
sh ./mvnw compile ; sh ./mvnw exec:java
```
This project aims to use as little boilerplate as possible. it uses
[javalin](https://javalin.io/), a lightweight java/kotlin web framework and
kafka-client libraries directly.
Kafka consumers subscribes to topics and then starts to poll them in real time,
synchronously.
That means you're not supposed to consume kafka streams in the main thread of a
web server, it will hang.
Therefore, this project spawns another thread and then polls kafka messages from
there.
## how to run service-consumer-2
This project uses spring-boot-kafka to consume the streams.
```bash
cd service-consumer-2
sh ./mvnw compile ; sh ./mvnw spring-boot:run
```