https://github.com/evolution-gaming/kafka-flow

library for reliable processing records received from kafka
https://github.com/evolution-gaming/kafka-flow

cats cats-effect kafka kafka-streams scala

Last synced: 3 months ago
JSON representation

library for reliable processing records received from kafka

• Host: GitHub
• URL: https://github.com/evolution-gaming/kafka-flow
• Owner: evolution-gaming
• License: mit
• Created: 2019-08-28T19:36:59.000Z (almost 6 years ago)
• Default Branch: master
• Last Pushed: 2025-04-03T10:20:30.000Z (3 months ago)
• Last Synced: 2025-04-23T00:15:45.041Z (3 months ago)
• Topics: cats, cats-effect, kafka, kafka-streams, scala
• Language: Scala
• Homepage:
• Size: 1.77 MB
• Stars: 21
• Watchers: 12
• Forks: 11
• Open Issues: 21
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# kafka-flow

[![Build Status](https://github.com/evolution-gaming/kafka-flow/workflows/CI/badge.svg)](https://github.com/evolution-gaming/kafka-flow/actions?query=workflow%3ACI)

[![Coverage Status](https://coveralls.io/repos/github/evolution-gaming/kafka-flow/badge.svg?branch=master)](https://coveralls.io/github/evolution-gaming/kafka-flow?branch=master)

[![Codacy Badge](https://app.codacy.com/project/badge/Grade/9d1c127fef824d4faaf775a1a5c5fb0d)](https://app.codacy.com/gh/evolution-gaming/kafka-flow/dashboard?utm_source=gh&utm_medium=referral&utm_content=&utm_campaign=Badge_grade)

[![Version](https://img.shields.io/badge/version-click-blue)](https://evolution.jfrog.io/artifactory/api/search/latestVersion?g=com.evolutiongaming&a=kafka-flow_2.13&repos=public)

[![License: MIT](https://img.shields.io/badge/License-MIT-yellowgreen.svg)](https://opensource.org/licenses/MIT)

## Microsite

https://evolution-gaming.github.io/kafka-flow

## Scala 3 compatibility

Starting from version `6.1.0` all of the modules are cross-compiled to Scala 3 except `kafka-flow-kafka-journal` which doesn't support Scala 3 yet.

## Setup

```scala

addSbtPlugin("com.evolution" % "sbt-artifactory-plugin" % "0.0.2")

lazy val version = "" // see the latest one in the badge above or in Releases page 

libraryDependencies ++= Seq(

  "com.evolutiongaming" %% "kafka-flow" % version,

  // if you want to use Cassandra for storing persistent state

  "com.evolutiongaming" %% "kafka-flow-persistence-cassandra" % version,

  // if you want to use Kafka compact topic for storing persistent state

  "com.evolutiongaming" %% "kafka-flow-persistence-kafka" % version,

  // if you want to use predefined metrics

  "com.evolutiongaming" %% "kafka-flow-metrics" % version,

  // if you want to use kafka-journal integration

  "com.evolutiongaming" %% "kafka-flow-kafka-journal" % version,

)

```

## Release process

The release process is based on Git tags and makes use of [evolution-gaming/scala-github-actions](https://github.com/evolution-gaming/scala-github-actions) which uses [sbt-dynver](https://github.com/sbt/sbt-dynver) to automatically obtain the version from the latest Git tag. The flow is defined in `.github/workflows/release.yml`.  

A typical release process is as follows:

1. Create and push a new Git tag. The version should be in the format `vX.Y.Z` (example: `v4.1.0`). Example: `git tag v4.1.0 && git push origin v4.1.0`

2. On success, a new GitHub release is automatically created with a calculated diff and auto-generated release notes. You can see it on `Releases` page, change the description if needed

3. On failure, the tag is deleted from the remote repository. Please note that your local tag isn't deleted, so if the failure is recoverable then you can delete the local tag and try again (an example of *unrecoverable* failure is successfully publishing only a few of the artifacts to Artifactory which means a new attempt would fail since Artifactory doesn't allow overwriting its contents)