{"id":15403649,"url":"https://github.com/rogervinas/spring-cloud-stream-kafka-streams-processor","last_synced_at":"2025-04-16T07:36:01.080Z","repository":{"id":44958675,"uuid":"440291383","full_name":"rogervinas/spring-cloud-stream-kafka-streams-processor","owner":"rogervinas","description":"🍀 Spring Cloud Stream \u0026 Kafka Streams Binder + Processor API","archived":false,"fork":false,"pushed_at":"2025-03-26T13:29:18.000Z","size":364,"stargazers_count":1,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-26T14:41:26.573Z","etag":null,"topics":["kafka-streams","spring-boot","spring-cloud-stream","spring-cloud-stream-binder-kafka","spring-cloud-stream-kafka"],"latest_commit_sha":null,"homepage":"https://dev.to/rogervinas/spring-cloud-stream-kafka-streams-binder-processor-api-2hko","language":"Kotlin","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rogervinas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-20T19:57:27.000Z","updated_at":"2025-03-26T13:29:25.000Z","dependencies_parsed_at":"2024-04-22T20:26:30.818Z","dependency_job_id":"2df0269f-33b3-4935-ac6b-4c7e894626a9","html_url":"https://github.com/rogervinas/spring-cloud-stream-kafka-streams-processor","commit_stats":{"total_commits":70,"total_committers":3,"mean_commits":"23.333333333333332","dds":0.5571428571428572,"last_synced_commit":"a30de08dbf66e9d62e47370f2ae2e75f2e7a0c2c"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rogervinas%2Fspring-cloud-stream-kafka-streams-processor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rogervinas%2Fspring-cloud-stream-kafka-streams-processor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rogervinas%2Fspring-cloud-stream-kafka-streams-processor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rogervinas%2Fspring-cloud-stream-kafka-streams-processor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rogervinas","download_url":"https://codeload.github.com/rogervinas/spring-cloud-stream-kafka-streams-processor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249213776,"owners_count":21231096,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka-streams","spring-boot","spring-cloud-stream","spring-cloud-stream-binder-kafka","spring-cloud-stream-kafka"],"created_at":"2024-10-01T16:09:31.787Z","updated_at":"2025-04-16T07:36:01.029Z","avatar_url":"https://github.com/rogervinas.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![CI](https://github.com/rogervinas/spring-cloud-stream-kafka-streams-processor/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/rogervinas/spring-cloud-stream-kafka-streams-processor/actions/workflows/ci.yml)\n![Java](https://img.shields.io/badge/Java-21-blue?labelColor=black)\n![Kotlin](https://img.shields.io/badge/Kotlin-2.x-blue?labelColor=black)\n![SpringBoot](https://img.shields.io/badge/SpringBoot-3.x-blue?labelColor=black)\n![SpringCloud](https://img.shields.io/badge/SpringCloud-2024.x-blue?labelColor=black)\n\n# Spring Cloud Stream \u0026 Kafka Streams Binder + Processor API\n\n[Spring Cloud Stream](https://spring.io/projects/spring-cloud-stream) is the solution provided by **Spring** to build applications connected to shared messaging systems.\n\nIt offers an abstraction (the **binding**) that works the same whatever underneath implementation we use (the **binder**):\n* **Apache Kafka**\n* **Rabbit MQ**\n* **Kafka Streams**\n* **Amazon Kinesis**\n* ...\n\nYou can also check out [Spring Cloud Stream Kafka Streams first steps](https://github.com/rogervinas/spring-cloud-stream-kafka-streams-first-steps) where I got working a simple example using **Kafka Streams binder**.\n\nIn this one the goal is to use the **Kafka Streams binder** and the [Kafka Streams Processor API](https://kafka.apache.org/documentation/streams/developer-guide/processor-api.html) to implement the following scenario:\n\n![Diagram](doc/diagram.png)\n\n1. We receive messages with key = **userId** and value = { userId: string, token: number } from topic **pub.user.token**\n\n2. For every **userId** which we receive **token** 1, 2, 3, 4 and 5 within under **1 minute**, we send a **completed** event to topic **pub.user.state**\n\n3. For every **userId** which we receive at least one **token** but not the complete 1, 2, 3, 4 and 5 sequence within under **1 minute**, we send an **expired** event to topic **pub.user.state**\n\nReady? Let's code! 🤓\n\n* [Test-first using kafka-streams-test-utils](#test-first-using-kafka-streams-test-utils)\n* [UserStateStream implementation](#userstatestream-implementation)\n  * [1. Aggregation by userId](#1-aggregation-by-userid)\n  * [2. Completed UserStateEvents](#2-completed-userstateevents)\n  * [3. UserStateProcessor implementation](#3-userstateprocessor-implementation)\n  * [4. UserStateStream and UserStateProcessor integration](#4-userstatestream-and-userstateprocessor-integration)\n* [Kafka Streams binder configuration](#kafka-streams-binder-configuration)\n* [UserStateStream bean](#userstatestream-bean)\n* [Integration Test](#integration-test)\n  * [1. Kafka helpers](#1-kafka-helpers)\n  * [2. DockerCompose Testcontainer](#2-dockercompose-testcontainer)\n  * [3. Tests](#3-tests)\n* [Important information about caching in the state stores](#important-information-about-caching-in-the-state-stores)\n* [Test this demo](#test-this-demo)\n* [Run this demo](#run-this-demo)\n* See also\n  * :octocat: [Spring Cloud Stream Kafka step by step](https://github.com/rogervinas/spring-cloud-stream-kafka-step-by-step)\n  * :octocat: [Spring Cloud Stream \u0026 Kafka Confluent Avro Schema Registry](https://github.com/rogervinas/spring-cloud-stream-kafka-confluent-avro-schema-registry)\n  * :octocat: [Spring Cloud Stream \u0026 Kafka Streams Binder first steps](https://github.com/rogervinas/spring-cloud-stream-kafka-streams-first-steps)\n  * :octocat: [Spring Cloud Stream Multibinder](https://github.com/rogervinas/spring-cloud-stream-multibinder)\n\nYou can browse older versions of this repo:\n* [Spring Boot 2.x](https://github.com/rogervinas/spring-cloud-stream-kafka-streams-processor/tree/spring-boot-2.x)\n\n## Test-first using kafka-streams-test-utils\n\nOnce [kafka-streams-test-utils](https://kafka.apache.org/documentation/streams/developer-guide/testing.html) is properly setup in our [@BeforeEach](src/test/kotlin/com/rogervinas/kafkastreams/stream/UserStreamTest.kt#L39) we can implement this test:\n\n```kotlin\ndata class UserTokenEvent(val userId: String, val token: Int)\n\nenum class UserStateEventType { COMPLETED, EXPIRED }\ndata class UserStateEvent(val userId: String, val state: UserStateEventType)\n\n@Test\nfun `should publish completed event for one user`() {\n  topicIn.pipeInput(USERNAME_1, UserTokenEvent(USERNAME_1, 1))\n  topicIn.pipeInput(USERNAME_1, UserTokenEvent(USERNAME_1, 2))\n  topicIn.pipeInput(USERNAME_1, UserTokenEvent(USERNAME_1, 3))\n  topicIn.pipeInput(USERNAME_1, UserTokenEvent(USERNAME_1, 4))\n  topicIn.pipeInput(USERNAME_1, UserTokenEvent(USERNAME_1, 5))\n\n  topologyTestDriver.advanceWallClockTime(EXPIRATION.minusMillis(10))\n\n  assertThat(topicOut.readKeyValuesToList()).singleElement().satisfies(Consumer { topicOutMessage -\u003e\n    assertThat(topicOutMessage.key).isEqualTo(USERNAME_1)\n    assertThat(topicOutMessage.value).isEqualTo(UserStateEvent(USERNAME_1, COMPLETED))\n  })\n}\n\n@Test\nfun `should publish expired event for one user`() {\n  topicIn.pipeInput(USERNAME_1, UserTokenEvent(USERNAME_1, 1))\n  topicIn.pipeInput(USERNAME_1, UserTokenEvent(USERNAME_1, 2))\n\n  topologyTestDriver.advanceWallClockTime(EXPIRATION.plus(SCHEDULE).plus(SCHEDULE))\n\n  assertThat(topicOut.readKeyValuesToList()).singleElement().satisfies(Consumer { topicOutMessage -\u003e\n    assertThat(topicOutMessage.key).isEqualTo(USERNAME_1)\n    assertThat(topicOutMessage.value).isEqualTo(UserStateEvent(USERNAME_1, EXPIRED))\n  })\n}\n```\n\n## UserStateStream implementation\n\nWe start first with our **UserStateStream** implementation as a **Function**:\n* Which input is a **KStream\u003cString, UserTokenEvent\u003e**, as we want a **String** as the Kafka message's key and a **UserTokenEvent** as the Kafka message's value\n* Which output is a **KStream\u003cString, UserStateEvent\u003e**, same here, **String** as the key and **UserStateEvent** as the value\n\n```kotlin\nclass UserStateStream(\n  private val schedule: Duration,\n  private val expiration: Duration\n) : Function\u003cKStream\u003cString, UserTokenEvent\u003e, KStream\u003cString, UserStateEvent\u003e\u003e {\n\n  override fun apply(input: KStream\u003cString, UserTokenEvent\u003e): KStream\u003cString, UserStateEvent\u003e {\n    TODO()\n  }\n}\n```\n\nNow step by step ...\n\n### 1. Aggregation by userId\n\n```kotlin\nprivate const val USER_STATE_STORE = \"user-state\"\n\ndata class UserState(val userId: String = \"\", val tokens: List\u003cInt\u003e = emptyList()) {\n  operator fun plus(event: UserTokenEvent) = UserState(event.userId, tokens + event.token)\n}\n\nclass UserStateStream(\n  private val schedule: Duration,\n  private val expiration: Duration\n) : Function\u003cKStream\u003cString, UserTokenEvent\u003e, KStream\u003cString, UserStateEvent\u003e\u003e {\n  override fun apply(input: KStream\u003cString, UserTokenEvent\u003e): KStream\u003cString, UserStateEvent\u003e {\n    return input\n      .selectKey { _, event -\u003e event.userId } // just in case but the key should be userId already\n      .groupByKey()\n      .aggregate(\n        { UserState() },\n        { userId, event, state -\u003e\n          logger.info(\"Aggregate $userId ${state.tokens} + ${event.token}\")\n          state + event // we use the UserState's plus operator\n        },\n        Materialized.`as`\u003cString, UserState, KeyValueStore\u003cBytes, ByteArray\u003e\u003e(USER_STATE_STORE)\n          .withKeySerde(Serdes.StringSerde())\n          .withValueSerde(JsonSerde(UserState::class.java))\n      )\n      .toStream()\n      // From here down it is just to avoid compilation errors\n      .mapValues { userId, _ -\u003e\n        UserStateEvent(userId, COMPLETED) \n      }\n  }\n}\n```\n\n### 2. Completed UserStateEvents\n\nWe can generate **completed** **UserStateEvents** straightaway once we receive the last **UserTokenEvent**:\n\n```kotlin\ndata class UserState(val userId: String = \"\", val tokens: List\u003cInt\u003e = emptyList()) {\n  // ...\n  val completed = tokens.containsAll(listOf(1, 2, 3, 4, 5))\n}\n\nclass UserStateStream(\n  private val schedule: Duration,\n  private val expiration: Duration\n) : Function\u003cKStream\u003cString, UserTokenEvent\u003e, KStream\u003cString, UserStateEvent\u003e\u003e {\n  override fun apply(input: KStream\u003cString, UserTokenEvent\u003e): KStream\u003cString, UserStateEvent\u003e {\n    return input\n      // ...\n      .toStream()\n      .mapValues { state -\u003e\n        logger.info(\"State $state\")\n        when {\n          state.completed -\u003e UserStateEvent(state.userId, COMPLETED)\n          else -\u003e null\n        }\n      }\n      .filter { _, event -\u003e event != null }\n      .mapValues { event -\u003e\n        logger.info(\"Publish $event\")\n        event!!\n      }\n  }\n}\n```\n\n### 3. UserStateProcessor implementation\n\nOur **UserStateProcessor** will scan periodically the **\"user-state\"** store and it will apply our expiration logic to every **UserState**:\n\n```kotlin\nclass UserStateProcessor(\n  private val schedule: Duration,\n  private val expiration: Duration\n) : Processor\u003cString, UserState, Void, Void\u003e {\n\n  override fun init(context: ProcessorContext\u003cVoid, Void\u003e) {\n    context.schedule(schedule, PunctuationType.WALL_CLOCK_TIME) { time -\u003e\n      val stateStore = context.getStateStore\u003cKeyValueStore\u003cString, ValueAndTimestamp\u003cUserState\u003e\u003e\u003e(USER_STATE_STORE)\n      stateStore.all().forEachRemaining { it : KeyValue\u003cString, ValueAndTimestamp\u003cUserState\u003e\u003e -\u003e\n        logger.info(\"Do something with $it!!\") // TODO\n      }\n    }\n  }\n\n  override fun process(record: Record\u003cString, UserState\u003e?) {\n    // we do not need to do anything here\n  }\n}\n```\n\nJust apply the expiration logic this way:\n\n```kotlin\ndata class UserState(val userId: String = \"\", val tokens: List\u003cInt\u003e = emptyList(), val expired: Boolean = false) {\n  // ...\n  fun expire() = UserState(userId, tokens, true)\n}\n\nclass UserStateProcessor(\n  private val schedule: Duration,\n  private val expiration: Duration\n) : Processor\u003cString, UserState, Void, Void\u003e {\n\n  override fun init(context: ProcessorContext\u003cVoid, Void\u003e) {\n    context.schedule(schedule, PunctuationType.WALL_CLOCK_TIME) { time -\u003e\n      val stateStore = context.getStateStore\u003cKeyValueStore\u003cString, ValueAndTimestamp\u003cUserState\u003e\u003e\u003e(USER_STATE_STORE)\n      stateStore.all().forEachRemaining {\n        val age = Duration.ofMillis(time - it.value.timestamp())\n        if (age \u003e expiration) {\n          if (it.value.value().expired) {\n            // if it is already expired from a previous execution, we delete it\n            logger.info(\"Delete ${it.key}\")\n            stateStore.delete(it.key)\n          } else {\n            // if it has expired right now, we mark it as expired and we update it\n            logger.info(\"Expire ${it.key}\")\n            stateStore.put(it.key, ValueAndTimestamp.make(it.value.value().expire(), it.value.timestamp()))\n          }\n        }\n      }\n    }\n  }\n}\n```\n\n### 4. UserStateStream and UserStateProcessor integration\n\n```kotlin\nclass UserStateStream(\n  private val schedule: Duration,\n  private val expiration: Duration\n) : Function\u003cKStream\u003cString, UserTokenEvent\u003e, KStream\u003cString, UserStateEvent\u003e\u003e {\n  override fun apply(input: KStream\u003cString, UserTokenEvent\u003e): KStream\u003cString, UserStateEvent\u003e {\n    return input\n      // ...\n      .toStream()\n      // we add the UserStateProcessor\n      .apply { process(ProcessorSupplier { UserStateProcessor(schedule, expiration) }, USER_STATE_STORE) }\n      // downstream we will both receive upstream realtime values as the ones \"generated\" by the UserStateProcessor\n      .mapValues { state -\u003e\n        logger.info(\"State $state\")\n        when {\n          // null states are sent downstream by UserStateProcessor when deleting entries from the store\n          state == null -\u003e null // \"null\" value generated by UserStateProcessor deleting values from the store\n          // completed states are sent downstream from upstream\n          state.completed -\u003e UserStateEvent(state.userId, COMPLETED)\n          // expired states are sent downstream by UserStateProcessor when updating entries from the store\n          state.expired -\u003e UserStateEvent(state.userId, EXPIRED)\n          else -\u003e null\n        }\n      }\n      .filter { _, event -\u003e event != null }\n      .mapValues { event -\u003e\n        logger.info(\"Publish $event\")\n        event!!\n      }\n  }\n}\n```\n\nAnd just at this point our [UserStreamTest](src/test/kotlin/com/rogervinas/kafkastreams/stream/UserStreamTest.kt) should pass 🟩 👌\n\n## Kafka Streams binder configuration\n\nEasy!\n\n```yaml\nspring:\n  application:\n    name: \"spring-cloud-stream-kafka-streams-processor\"\n  cloud:\n    stream:\n      function:\n        definition: userStateStream\n        bindings:\n          userStateStream-in-0: \"pub.user.token\"\n          userStateStream-out-0: \"pub.user.state\"\n      kafka:\n        streams:\n          binder:\n            applicationId: \"${spring.application.name}\"\n            brokers: \"localhost:9094\"\n            configuration:\n              default:\n                key.serde: org.apache.kafka.common.serialization.Serdes$StringSerde\n                value.serde: org.apache.kafka.common.serialization.Serdes$StringSerde\n```\n\nWith this configuration:\n* **Spring Cloud Stream** will create a **Kafka Streams binder** connected to **localhost:9094**\n* We need to create a **@Bean** named **userStateStream** that should implement **Function\u003cKStream, KStream\u003e** interface\n    * This **@Bean** will connect a **KStream** subscribed to **pub.user.token** topic to another **KStream** publishing to **pub.user.state** topic\n\nYou can find all the available configuration properties documented in [Kafka Streams Properties](https://cloud.spring.io/spring-cloud-stream-binder-kafka/spring-cloud-stream-binder-kafka.html#_kafka_streams_properties).\n\n## UserStateStream bean\n\nAs required by our configuration we need to create a @Bean named `userStateStream`:\n\n```kotlin\n@Configuration\nclass ApplicationConfiguration {\n\n  @Bean\n  fun userStateStream(\n    @Value(\"\\${user.schedule}\") schedule: Duration,\n    @Value(\"\\${user.expiration}\") expiration: Duration\n  ): Function\u003cKStream\u003cString, UserTokenEvent\u003e, KStream\u003cString, UserStateEvent\u003e\u003e = UserStateStream(schedule, expiration)\n}\n```\n\n## Integration Test\n\nWe already \"unit test\" our **UserStateStream** with **kafka-streams-test-utils** but we need also an integration test using a Kafka container ... [Testcontainers](https://www.testcontainers.org) to the rescue!\n\n### 1. Kafka helpers\n\nFirst we need utility classes to produce to Kafka and consume from Kafka using **kafka-clients** library:\n\n```kotlin\n\nclass KafkaConsumerHelper(bootstrapServers: String, topic: String) {\n  \n  fun consumeAll(): List\u003cConsumerRecord\u003cString, String\u003e\u003e {\n    // ...\n  }\n\n  fun consumeAtLeast(numberOfRecords: Int, timeout: Duration): List\u003cConsumerRecord\u003cString, String\u003e\u003e {\n    // ...\n  }\n}\n\nclass KafkaProducerHelper(bootstrapServers: String) {\n  \n  fun send(topic: String?, key: String, body: String) {\n    // ...\n  }\n}\n```\n\n### 2. DockerCompose Testcontainer\n\nAs described in [Testcontainers + Junit5](https://www.testcontainers.org/test_framework_integration/junit_5/) we can use `@Testcontainers` annotation:\n\n```kotlin\n@SpringBootTest\n@Testcontainers\n@ActiveProfiles(\"test\")\nclass ApplicationIntegrationTest {\n\n  companion object {\n\n    @Container\n    val container = DockerComposeContainerHelper().createContainer()\n  }\n  \n  // ...\n}\n```\n\n### 3. Tests\n\nAnd finally the tests, using [Awaitility](https://github.com/awaitility/awaitility) as we are testing asynchronous stuff:\n\n```kotlin\nclass ApplicationIntegrationTest {\n  \n  // ...\n\n  @Test\n  fun `should publish completed event`() {\n    val username = UUID.randomUUID().toString()\n\n    kafkaProducerHelper.send(TOPIC_USER_TOKEN, username, \"\"\"{\"userId\": \"$username\", \"token\": 1}\"\"\")\n    kafkaProducerHelper.send(TOPIC_USER_TOKEN, username, \"\"\"{\"userId\": \"$username\", \"token\": 2}\"\"\")\n    kafkaProducerHelper.send(TOPIC_USER_TOKEN, username, \"\"\"{\"userId\": \"$username\", \"token\": 3}\"\"\")\n    kafkaProducerHelper.send(TOPIC_USER_TOKEN, username, \"\"\"{\"userId\": \"$username\", \"token\": 4}\"\"\")\n    kafkaProducerHelper.send(TOPIC_USER_TOKEN, username, \"\"\"{\"userId\": \"$username\", \"token\": 5}\"\"\")\n\n    await().atMost(ONE_MINUTE).untilAsserted {\n      val record = kafkaConsumerHelper.consumeAtLeast(1, ONE_SECOND)\n      assertThat(record).singleElement().satisfies(Consumer {\n        assertThat(it.key()).isEqualTo(username)\n        JSONAssert.assertEquals(\"\"\"{\"userId\": \"$username\", \"state\": \"COMPLETED\"}\"\"\", it.value(), true)\n      })\n    }\n  }\n\n  @Test\n  fun `should publish expired event`() {\n    val username = UUID.randomUUID().toString()\n\n    kafkaProducerHelper.send(TOPIC_USER_TOKEN, username, \"\"\"{\"userId\": \"$username\", \"token\": 1}\"\"\")\n    kafkaProducerHelper.send(TOPIC_USER_TOKEN, username, \"\"\"{\"userId\": \"$username\", \"token\": 2}\"\"\")\n    kafkaProducerHelper.send(TOPIC_USER_TOKEN, username, \"\"\"{\"userId\": \"$username\", \"token\": 3}\"\"\")\n    kafkaProducerHelper.send(TOPIC_USER_TOKEN, username, \"\"\"{\"userId\": \"$username\", \"token\": 4}\"\"\")\n\n    await().atMost(ONE_MINUTE).untilAsserted {\n      val record = kafkaConsumerHelper.consumeAtLeast(1, ONE_SECOND)\n      assertThat(record).singleElement().satisfies(Consumer {\n        assertThat(it.key()).isEqualTo(username)\n        JSONAssert.assertEquals(\"\"\"{\"userId\": \"$username\", \"state\": \"EXPIRED\"}\"\"\", it.value(), true)\n      })\n    }\n  }\n}\n```\n\nAnd just at this point all our tests should pass 🟩 👏\n\nThat's it, happy coding! 💙\n\n## Important information about caching in the state stores\n\nIf you [run this demo](#run-this-demo) you will notice that the **completed** **UserStateEvents** are not sent inmediately:\n```\n12:36:34.593 : Aggregate 1 [] + 1\n12:36:36.110 : Aggregate 1 [1] + 2\n12:36:37.660 : Aggregate 1 [1, 2] + 3\n12:36:38.061 : Aggregate 1 [1, 2, 3] + 4\n12:36:48.890 : Aggregate 1 [1, 2, 3, 4] + 5\n12:36:58.256 : State UserState(userId=1, tokens=[1, 2, 3, 4, 5], expired=false)\n12:36:58.262 : Publish UserStateEvent(userId=1, state=COMPLETED)\n```\n\nIn this example ☝️ the **UserStateEvent** is sent 10 seconds after the last **UserTokenEvent** is received. \n\nWhy? The answer is: **caching**!\n\nAs stated in [Kafka Streams \u003e Developer Guide \u003e Memory Management](https://kafka.apache.org/documentation/streams/developer-guide/memory-mgmt):\n\u003e The semantics of caching is that data is flushed to the state store and forwarded to the next downstream processor node whenever the earliest of commit.interval.ms or cache.max.bytes.buffering (cache pressure) hits.\n\nAnd also in [A Guide to Kafka Streams and Its Uses](https://www.confluent.io/blog/how-kafka-streams-works-guide-to-stream-processing/#ktable):\n\u003e KTable objects are backed by state stores, which enable you to look up and track these latest values by key. Updates are likely buffered into a cache, which gets flushed by default every 30 seconds.\n\n## Test this demo\n\n```shell\n./gradlew test\n```\n\n## Run this demo\n\nRun with docker-compose:\n```shell\ndocker-compose up -d\n./gradlew bootRun\ndocker-compose down\n```\n\nThen you can use [kcat](https://github.com/edenhill/kcat) to produce/consume to/from **Kafka**:\n```shell\n# consume\nkcat -b localhost:9094 -C -t pub.user.token -f '%k %s\\n'\nkcat -b localhost:9094 -C -t pub.user.state -f '%k %s\\n'\n\n# produce\necho '1:{\"userId\":\"1\", \"token\":1}' | kcat -b localhost:9094 -P -t pub.user.token -K:\necho '1:{\"userId\":\"1\", \"token\":2}' | kcat -b localhost:9094 -P -t pub.user.token -K:\necho '1:{\"userId\":\"1\", \"token\":3}' | kcat -b localhost:9094 -P -t pub.user.token -K:\necho '1:{\"userId\":\"1\", \"token\":4}' | kcat -b localhost:9094 -P -t pub.user.token -K:\necho '1:{\"userId\":\"1\", \"token\":5}' | kcat -b localhost:9094 -P -t pub.user.token -K:\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frogervinas%2Fspring-cloud-stream-kafka-streams-processor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frogervinas%2Fspring-cloud-stream-kafka-streams-processor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frogervinas%2Fspring-cloud-stream-kafka-streams-processor/lists"}