{"id":21877627,"url":"https://github.com/rbiedrawa/spring-kafka-multiple-event-types-in-same-topic","last_synced_at":"2026-04-11T14:03:43.428Z","repository":{"id":162193124,"uuid":"369964125","full_name":"rbiedrawa/spring-kafka-multiple-event-types-in-same-topic","owner":"rbiedrawa","description":"This repository contains sample Spring Boot setup for handling multiple event types in single Kafka topic.","archived":false,"fork":false,"pushed_at":"2022-01-31T02:02:16.000Z","size":100,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-07-10T01:38:31.684Z","etag":null,"topics":["avro","avro-kafka","docker","docker-compose","gradle","interactive-queries","kafka","kafka-streams","multiple-event","protobuf","schema-registry","spring-boot","spring-cloud-stream","spring-cloud-stream-binder-kafka","spring-kafka","state-store","testing"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rbiedrawa.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-23T04:57:45.000Z","updated_at":"2023-10-01T10:56:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"f5875a29-33e1-4377-8ba2-086a1f82610f","html_url":"https://github.com/rbiedrawa/spring-kafka-multiple-event-types-in-same-topic","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rbiedrawa/spring-kafka-multiple-event-types-in-same-topic","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbiedrawa%2Fspring-kafka-multiple-event-types-in-same-topic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbiedrawa%2Fspring-kafka-multiple-event-types-in-same-topic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbiedrawa%2Fspring-kafka-multiple-event-types-in-same-topic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbiedrawa%2Fspring-kafka-multiple-event-types-in-same-topic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rbiedrawa","download_url":"https://codeload.github.com/rbiedrawa/spring-kafka-multiple-event-types-in-same-topic/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rbiedrawa%2Fspring-kafka-multiple-event-types-in-same-topic/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273377176,"owners_count":25094535,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-03T02:00:09.631Z","response_time":76,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["avro","avro-kafka","docker","docker-compose","gradle","interactive-queries","kafka","kafka-streams","multiple-event","protobuf","schema-registry","spring-boot","spring-cloud-stream","spring-cloud-stream-binder-kafka","spring-kafka","state-store","testing"],"created_at":"2024-11-28T08:09:48.317Z","updated_at":"2025-12-30T21:31:40.525Z","avatar_url":"https://github.com/rbiedrawa.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Putting multiple event types in single kafka topic (PoC)\n\nThis repository contains sample Spring Boot setup for handling multiple event types in single kafka topic.\n\n## Features\n\n* Java Functions using Spring Cloud Stream Kafka Streams binder (`avro` project).\n* Avro's unions with schema references.\n* Avro Schema registration via gradle task.\n* Protobuf schema references.\n* Unit testing with `TopologyTestDriver`.\n\n## Getting Started\n\n### Prerequisites\n\n* Java 11\n* Docker\n* Apache Kafka, Schema Registry\n\n### Usage\n\n* Start kafka platform.\n  ```shell\n  docker-compose -f docker/docker-compose.yml up -d\n  ```\n\n* List containers and check if all are `Up`.\n    ```shell\n    docker-compose -f docker/docker-compose.yml ps \n\n    #      Name                  Command            State                       Ports                     \n    # ----------------------------------------------------------------------------------------------------\n    # broker            /etc/confluent/docker/run   Up      0.0.0.0:9092-\u003e9092/tcp, 0.0.0.0:9101-\u003e9101/tcp\n    # control-center    /etc/confluent/docker/run   Up      0.0.0.0:9021-\u003e9021/tcp                        \n    # rest-proxy        /etc/confluent/docker/run   Up      0.0.0.0:8082-\u003e8082/tcp                        \n    # schema-registry   /etc/confluent/docker/run   Up      0.0.0.0:8081-\u003e8081/tcp                        \n    # zookeeper         /etc/confluent/docker/run   Up      0.0.0.0:2181-\u003e2181/tcp, 2888/tcp, 3888/tcp    \n    ```\n\n* Start one of the sample application:\n  * **Avro**:\n    * Register Schemas via gradle task:\n      ```shell\n      cd avro\n      ./gradlew registerSchemasTask\n      ```\n    * Start application:\n      ```shell\n      ./gradlew bootRun\n      ```\n    * Check application logs to see that messages are correctly serialized and deserialized. Below sample output:\n      ```shell\n      # c.r.k.a.a.t.d.DummyTransactionGenerator  : Successfully sent TransactionEvent ca0e0357-e76e-4d8c-8015-e545111ea853 of type class com.rbiedrawa.app.avro.events.transactions.TransactionStarted\n      # c.r.k.a.a.t.d.DummyTransactionGenerator  : Successfully sent TransactionEvent ca0e0357-e76e-4d8c-8015-e545111ea853 of type class com.rbiedrawa.app.avro.events.transactions.TransactionCompleted\n      # c.r.k.a.a.t.TransactionKStream           : Consumed key ca0e0357-e76e-4d8c-8015-e545111ea853 value class com.rbiedrawa.app.avro.events.transactions.TransactionStarted\n      # c.r.k.a.a.t.TransactionKStream           : Consumed key ca0e0357-e76e-4d8c-8015-e545111ea853 value class com.rbiedrawa.app.avro.events.transactions.TransactionCompleted\n      ```\n    \n    * Use interactive query to get transaction from kafka state store:\n      ```shell\n      curl -X GET --location \"http://localhost:8080/api/transactions/ca0e0357-e76e-4d8c-8015-e545111ea853\"\n      ```\n\n  * **Protobuf**:\n    * Start application:\n      ```shell\n      cd proto\n      ./gradlew bootRun\n      ```\n    * Check application logs to see that messages are correctly serialized and deserialized. Below sample output:\n      ```shell\n      # c.r.k.a.t.d.DummyTransactionGenerator    : Successfully sent TransactionEvent d1e869d5-fb9b-4b13-ac43-678693d5910d of type TRANSACTION_STARTED\n      # c.r.k.a.transactions.TransactionKStream  : Consumed transaction event of type TRANSACTION_STARTED. TransactionId d1e869d5-fb9b-4b13-ac43-678693d5910d\n      # c.r.k.a.t.d.DummyTransactionGenerator    : Successfully sent TransactionEvent d1e869d5-fb9b-4b13-ac43-678693d5910d of type TRANSACTION_COMPLETED\n      # c.r.k.a.transactions.TransactionKStream  : Consumed transaction event of type TRANSACTION_COMPLETED. TransactionId d1e869d5-fb9b-4b13-ac43-678693d5910d\n      ```\n    \n    * Use interactive query to get transaction from kafka state store:\n      ```shell\n      curl -X GET --location \"http://localhost:8080/api/transactions/d1e869d5-fb9b-4b13-ac43-678693d5910d\"\n      ```\n\n* Stop docker-compose.\n  ```shell\n  docker-compose -f docker/docker-compose.yml down -v\n  ```\n\n\n## Important Endpoints\n\n| Name | Endpoint | \n| -------------:|:--------:|\n| `Spring Boot Application` | http://localhost:8080 |\n| `Spring Boot Application - Actuator health` | http://localhost:8080/actuator/health |\n| `Find transaction by id` | http://localhost:8080/api/transactions/{transactionId} |\n| `Schema Registry` | http://localhost:8081 |\n| `Schema Registry - Schemas` | http://localhost:8081/schemas |\n| `Schema Registry - Find schema by id` | http://localhost:8081/schemas/ids/{id} |\n| `Schema Registry - Subjects` | http://localhost:8081/subjects |\n| `Confluent Control Center` | http://localhost:9021 |\n\n## References\n\n* [Should You Put Several Event Types in the Same Kafka Topic?](https://www.confluent.io/blog/put-several-event-types-kafka-topic/)\n\n* [Putting Several Event Types in the Same Topic – Revisited](https://www.confluent.io/blog/multiple-event-types-in-the-same-kafka-topic/)\n\n* [How to Choose Between Strict and Dynamic Schemas](https://www.confluent.io/blog/spring-kafka-protobuf-part-1-event-data-modeling/)\n\n* [Schema evolution in Avro, Protocol Buffers and Thrift](https://martin.kleppmann.com/2012/12/05/schema-evolution-in-avro-protocol-buffers-thrift.html)\n\n## License\n\nDistributed under the MIT License. See `LICENSE` for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frbiedrawa%2Fspring-kafka-multiple-event-types-in-same-topic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frbiedrawa%2Fspring-kafka-multiple-event-types-in-same-topic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frbiedrawa%2Fspring-kafka-multiple-event-types-in-same-topic/lists"}