Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/thoughtscript/kafka_spring_docker_2024
Experiments with Kafka and Java Webflux
https://github.com/thoughtscript/kafka_spring_docker_2024
docker-compose java kafka pubsub webflux
Last synced: about 1 month ago
JSON representation
Experiments with Kafka and Java Webflux
- Host: GitHub
- URL: https://github.com/thoughtscript/kafka_spring_docker_2024
- Owner: Thoughtscript
- Created: 2024-05-03T16:55:54.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2024-05-04T16:20:05.000Z (8 months ago)
- Last Synced: 2024-05-04T17:29:58.606Z (8 months ago)
- Topics: docker-compose, java, kafka, pubsub, webflux
- Language: Java
- Homepage:
- Size: 12.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# kafka_spring_docker_2024
[](https://hub.docker.com/r/bitnami/kafka) [](https://hub.docker.com/_/maven) [](https://docs.spring.io/spring-framework/reference/web/webflux.html)
Run the following from the root dir:
```bash
docker-compose up
```
Upon running that command you should see:
```bash
consumer_1 | 2024-05-03T23:57:07.747Z INFO 60 --- [or-http-epoll-2] i.t.s.c.ReactiveRestController : webclient message
producer_1 | 2024-05-03T23:57:07.749Z INFO 60 --- [or-http-epoll-3] i.t.publisher.webclient.WebClient : WebClient message sent!
producer_1 | 2024-05-03T23:57:12.335Z INFO 60 --- [pool-5-thread-1] i.t.p.kafka.EmbeddedComponentClient : Kafka message sent!
consumer_1 | 2024-05-03T23:57:12.338Z INFO 60 --- [ntainer#0-0-C-1] i.t.s.kafka.EmbeddedComponentReceiver : hello1
consumer_1 | 2024-05-03T23:57:12.338Z INFO 60 --- [ntainer#0-0-C-1] i.t.s.kafka.EmbeddedComponentReceiver : hello2
consumer_1 | 2024-05-03T23:57:12.338Z INFO 60 --- [ntainer#0-0-C-1] i.t.s.kafka.EmbeddedComponentReceiver : hello3
consumer_1 | 2024-05-03T23:57:12.339Z INFO 60 --- [ntainer#0-0-C-1] i.t.s.kafka.EmbeddedComponentReceiver : hello4
consumer_1 | 2024-05-03T23:57:12.752Z INFO 60 --- [or-http-epoll-2] i.t.s.c.ReactiveRestController : webclient message
producer_1 | 2024-05-03T23:57:12.753Z INFO 60 --- [or-http-epoll-3] i.t.publisher.webclient.WebClient : WebClient message sent!
```
The **Producer** app will repeatedly loop through and send a bunch of HTTP Webflux and Kafka messages to the `test` topic. The **Consumer** app will consume both kinds of messages using the simply wonderful (concise) `EmbeddedComponentClient` approach (great functionalitiy provided by the Spring Kafka libraries) and a `ReactiveRestController`!
> Note: the whole cluster may take a few seconds to stabilize on app initialization, eventually you should just see a loop of messages like hte above.
## Kafka Notes
Trying out:
1. Kafka without Zookeeper! Using KRraft.
2. Addressing [Scenario 5](https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/#scenario-5) without doing the recommended ways (great article and super helpful - not sure exactly what's changed since 2020 but I got this working with two changes: `networks` in [docker-compose.yml](docker-compose.yml) and `spring.kafka.bootstrap-servers` using the Docker-managed DNS `kafka:9092` in [application.properties](src/main/resources/application.properties)).
3. Containerizing, simplifying, and updating my 2022 examples from: [java-reactive-pubsub
](https://github.com/Thoughtscript/java-reactive-pubsub).
4. Also, curious about different ways to containerize Java Spring Boot apps - oftentimes people containerize just the `.jar` - is it easier to just supply the exploded file source code?
## Resources and Links
1. https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/#scenario-5
2. https://www.baeldung.com/java-kafka-bootstrap-server