https://github.com/daggerok/functional-streaming

4del: This is going to be removed... or refactored if not so useless as I'm currently thinking... spring-cloud-stream, spring-cloud-reactive-streams, kafka, docker, gradle composed builds, streaming using reactor Mono/Flux
https://github.com/daggerok/functional-streaming

Last synced: 9 months ago
JSON representation

4del: This is going to be removed... or refactored if not so useless as I'm currently thinking... spring-cloud-stream, spring-cloud-reactive-streams, kafka, docker, gradle composed builds, streaming using reactor Mono/Flux

• Host: GitHub
• URL: https://github.com/daggerok/functional-streaming
• Owner: daggerok
• Created: 2017-05-09T20:09:30.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2019-06-09T22:43:24.000Z (over 6 years ago)
• Last Synced: 2025-04-05T10:23:16.368Z (9 months ago)
• Language: Java
• Homepage:
• Size: 126 KB
• Stars: 4
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.adoc

Awesome Lists containing this project

README

          
= functional-streaming image:https://travis-ci.org/daggerok/functional-streaming.svg?branch=master["Build Status", link="https://travis-ci.org/daggerok/functional-streaming"]

see other repo:

- link:https://github.com/daggerok/spring-cloud-function-stream-integration[spring-cloud-function and spring-cloud-stream integration]

== functional streaming using spring cloud (reactive) streams, schema registry, avro

see https://github.com/daggerok/functional-streaming/tree/master/streaming-reactive-processor[streaming-reactive-processor] and https://github.com/daggerok/functional-streaming/tree/master/streaming-reactive-consumer[streaming-reactive-consumer] projects

in our messaging pipeline we can use Mono / Flux from reactor project, which makes developing streaming cloud native application more exciting!

[source,java]

----

  @Output(Processor.OUTPU)

  @StreamListener(Processor.INPUT)

  public Flux> correctMessagesReceiver(final Flux> payload) {

    return payload.map(Map::entrySet)

                  .map(Set::iterator)

                  .filter(Iterator::hasNext)

                  .map(Iterator::next)

                  .map(this::doSomeTransformationLogic)

                  .window(3) // pagination or could be also time interval collector

                  .flatMap(window -> window.reduce("|", (s1, s2) -> s1 + s2 + "|"))

                  .map(body -> MessageBuilder.withPayload(body).build());

  }

----

right now is only one limitation: spring-cloud-stream doesn't suppotr spring-webflux yet, but I hope It will be changed soon (plus see streaming-reactive-*)

== build, run, clean

. bootstrap your message broker (docker-compose required)

. start apps

. cleanup

[source,bash]

----

gradle composeUp

gradle bootRun-producer

gradle bootRun-processor

gradle bootRun-aggregator

gradle bootRun-consumer

gradle bootRun-reactive-producer

gradle bootRun-reactive-aggregator

gradle bootRun-reactive-processor

gradle bootRun-reactive-consumer

gradle composeDown

gradle --stop

----

.run integration tests

[source,bash]

gradle integrationTests

other repositories:

. link:https://github.com/daggerok/serverless-examples[Serveless examples (spring-cloud-functions)]