{"id":13640775,"url":"https://github.com/zio/zio-kafka","last_synced_at":"2026-02-25T01:01:19.444Z","repository":{"id":34964833,"uuid":"173121216","full_name":"zio/zio-kafka","owner":"zio","description":"A fast Kafka client for ZIO and ZIO Streams","archived":false,"fork":false,"pushed_at":"2026-02-24T03:46:15.000Z","size":675286,"stargazers_count":366,"open_issues_count":21,"forks_count":144,"subscribers_count":17,"default_branch":"master","last_synced_at":"2026-02-24T09:33:17.442Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://zio.dev/zio-kafka","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":"CLA.md"}},"created_at":"2019-02-28T13:54:30.000Z","updated_at":"2026-02-24T03:02:58.000Z","dependencies_parsed_at":"2026-02-15T05:06:24.885Z","dependency_job_id":null,"html_url":"https://github.com/zio/zio-kafka","commit_stats":{"total_commits":882,"total_committers":87,"mean_commits":"10.137931034482758","dds":0.8344671201814059,"last_synced_commit":"5f371c641f6c22012a8746615396bdf278422bf2"},"previous_names":[],"tags_count":83,"template":false,"template_full_name":null,"purl":"pkg:github/zio/zio-kafka","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zio%2Fzio-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zio%2Fzio-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zio%2Fzio-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zio%2Fzio-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zio","download_url":"https://codeload.github.com/zio/zio-kafka/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zio%2Fzio-kafka/sbom","scorecard":{"id":698237,"data":{"date":"2025-07-21","repo":{"name":"github.com/zio/zio-kafka","commit":"4812adb02606c83ccd40cc19483f72e9d05e7771"},"scorecard":{"version":"v5.2.1-24-gc29a04d4","commit":"c29a04d46d1570393e94662bc34e9906398e1bfa"},"score":6.1,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#binary-artifacts"}},{"name":"Maintained","score":10,"reason":"26 commit(s) and 2 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#maintained"}},{"name":"Code-Review","score":10,"reason":"all changesets reviewed","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#code-review"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#cii-best-practices"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/auto-approve.yml:1","Warn: topLevel 'deployments' permission set to 'write': .github/workflows/benchs.yml:29","Warn: topLevel 'contents' permission set to 'write': .github/workflows/benchs.yml:31","Warn: no topLevel permission defined: .github/workflows/ci.yml:1","Warn: no topLevel permission defined: .github/workflows/dependency-graph.yml:1","Warn: topLevel 'contents' permission set to 'write': .github/workflows/profile.yml:9","Warn: topLevel 'deployments' permission set to 'write': .github/workflows/profile.yml:10","Warn: no topLevel permission defined: .github/workflows/release-drafter.yml:1","Warn: topLevel 'contents' permission set to 'write': .github/workflows/scala-steward.yml:10","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: third-party GitHubAction not pinned by hash: .github/workflows/auto-approve.yml:10: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/auto-approve.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/benchs.yml:40: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/benchs.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/benchs.yml:44: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/benchs.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/benchs.yml:50: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/benchs.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/benchs.yml:51: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/benchs.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/benchs.yml:77: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/benchs.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/benchs.yml:90: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/benchs.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:112: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:118: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:124: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:126: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:137: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:143: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:188: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:194: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:200: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:202: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:219: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:225: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:231: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:233: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:235: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:252: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:29: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:35: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:41: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:43: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:56: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:62: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:68: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:70: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:90: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:96: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:98: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:100: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/ci.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/dependency-graph.yml:11: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/dependency-graph.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/dependency-graph.yml:12: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/dependency-graph.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/dependency-graph.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/dependency-graph.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/profile.yml:18: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/profile.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/profile.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/profile.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/profile.yml:28: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/profile.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/profile.yml:29: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/profile.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/profile.yml:30: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/profile.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/release-drafter.yml:11: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/release-drafter.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/scala-steward.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/scala-steward.yml/master?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/scala-steward.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/zio/zio-kafka/scala-steward.yml/master?enable=pin","Info:   0 out of  20 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of  26 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#pinned-dependencies"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#vulnerabilities"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#security-policy"}},{"name":"Packaging","score":10,"reason":"packaging workflow detected","details":["Info: Project packages its releases by way of GitHub Actions.: .github/workflows/ci.yml:179"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#packaging"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/c29a04d46d1570393e94662bc34e9906398e1bfa/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-22T04:22:47.592Z","repository_id":34964833,"created_at":"2025-08-22T04:22:47.592Z","updated_at":"2025-08-22T04:22:47.592Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29807689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-24T22:43:48.403Z","status":"ssl_error","status_checked_at":"2026-02-24T22:43:18.536Z","response_time":75,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:01:14.324Z","updated_at":"2026-02-25T01:01:19.426Z","avatar_url":"https://github.com/zio.png","language":"Scala","funding_links":[],"categories":["Official ZIO Libraries","Scala","\u003ca name=\"Scala\"\u003e\u003c/a\u003eScala","Clients"],"sub_categories":["Scala"],"readme":"[//]: # (This file was autogenerated using `zio-sbt-website` plugin via `sbt generateReadme` command.)\n[//]: # (So please do not edit it manually. Instead, change \"docs/index.md\" file or sbt setting keys)\n[//]: # (e.g. \"readmeDocumentation\" and \"readmeSupport\".)\n\n# ZIO Kafka\n\n[ZIO Kafka](https://github.com/zio/zio-kafka) is a Kafka client for ZIO. It provides a purely functional, streams-based interface to the Kafka\nclient and integrates effortlessly with ZIO and zio-streams. Often zio-kafka programs have a _higher_ throughput than\nprograms that use the Java Kafka client directly (see section [Performance](#performance) below).\n\n[![Production Ready](https://img.shields.io/badge/Project%20Stage-Production%20Ready-brightgreen.svg)](https://github.com/zio/zio/wiki/Project-Stages) ![CI Badge](https://github.com/zio/zio-kafka/workflows/CI/badge.svg) [![Sonatype Releases](https://img.shields.io/nexus/r/https/oss.sonatype.org/dev.zio/zio-kafka_2.13.svg?label=Sonatype%20Release)](https://oss.sonatype.org/content/repositories/releases/dev/zio/zio-kafka_2.13/) [![Sonatype Snapshots](https://img.shields.io/nexus/s/https/oss.sonatype.org/dev.zio/zio-kafka_2.13.svg?label=Sonatype%20Snapshot)](https://oss.sonatype.org/content/repositories/snapshots/dev/zio/zio-kafka_2.13/) [![javadoc](https://javadoc.io/badge2/dev.zio/zio-kafka-docs_2.13/javadoc.svg)](https://javadoc.io/doc/dev.zio/zio-kafka-docs_2.13) [![ZIO Kafka](https://img.shields.io/github/stars/zio/zio-kafka?style=social)](https://github.com/zio/zio-kafka) [![Scala Steward badge](https://img.shields.io/badge/Scala_Steward-helping-blue.svg?style=flat\u0026logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAA4AAAAQCAMAAAARSr4IAAAAVFBMVEUAAACHjojlOy5NWlrKzcYRKjGFjIbp293YycuLa3pYY2LSqql4f3pCUFTgSjNodYRmcXUsPD/NTTbjRS+2jomhgnzNc223cGvZS0HaSD0XLjbaSjElhIr+AAAAAXRSTlMAQObYZgAAAHlJREFUCNdNyosOwyAIhWHAQS1Vt7a77/3fcxxdmv0xwmckutAR1nkm4ggbyEcg/wWmlGLDAA3oL50xi6fk5ffZ3E2E3QfZDCcCN2YtbEWZt+Drc6u6rlqv7Uk0LdKqqr5rk2UCRXOk0vmQKGfc94nOJyQjouF9H/wCc9gECEYfONoAAAAASUVORK5CYII=)](https://scala-steward.org)\n\n## Introduction\n\nApache Kafka is a distributed event streaming platform that acts as a distributed publish-subscribe messaging system. It enables us to build distributed streaming data pipelines and event-driven applications.\n\nKafka has a mature Java client for producing and consuming events, but it has a low-level API. Zio-kafka is a ZIO native client for Apache Kafka. It has a high-level streaming API on top of the Java client. So we can produce and consume events using the declarative concurrency model of zio-streams. In addition, zio-kafka supports an even higher level API where you only write the processing part and the rest is handled by zio-kafka.\n\n## Features\n\n- Exposes the Java Kafka consumer, producer and admin clients with a ZIO based interface.\n- Consuming:\n  - 2 APIs: streaming and ZIO workflow based\n  - supports custom deserialization\n  - process each partition in parallel for highest throughput\n  - allows batched processing for highest throughput\n  - configurable per partition pre-fetching (with back-pressure)\n  - the only async Kafka consumer [without duplicates](https://zio.dev/zio-kafka/preventing-duplicates) after a rebalance (as far as we know)\n  - very configurable\n  - automatic or manual starting offset\n  - supports external commits\n  - retries after authentication/authorization errors\n  - exposes metrics\n  - diagnostics API\n- Producing:\n  - 2 APIs: streaming and ZIO workflow based\n  - supports custom serialization\n  - allows for batches for highest throughput\n  - optionally await broker acknowledgements\n  - optional retries after authentication/authorization errors\n- Admin API:\n  - exposes all the admin client methods with a ZIO based interface\n- Proper errors when broker expects SSL (no [OOM crashes](https://issues.apache.org/jira/browse/KAFKA-4090))\n- Test kit with embedded kafka broker\n- Well documented\n- Community support via [Discord](https://discord.com/channels/629491597070827530/629497941719121960)\n- Commercial support via [Ziverge](https://www.ziverge.com)\n\n## Getting started\n\nSee the [zio-kafka tutorial](https://zio.dev/guides/tutorials/producing-consuming-data-from-kafka-topics/) for a grand tour of the different ways you can use zio-kafka.\n\nIn order to use this library, we need to add the following line in our `build.sbt` file:\n\n```scala\nlibraryDependencies += \"dev.zio\" %% \"zio-kafka\"         % \"3.2.0\"\nlibraryDependencies += \"dev.zio\" %% \"zio-kafka-testkit\" % \"3.2.0\" % Test\n```\n\nSnapshots are available on Sonatype's snapshot repository https://oss.sonatype.org/content/repositories/snapshots.\n[Browse here](https://oss.sonatype.org/content/repositories/snapshots/dev/zio/zio-kafka_3/) to find available versions.\n\nIf you use `zio-kafka` 3.x, scala 3, `zio-kafka-testkit` and `embedded-kafka`, you also need to add the following to\nyour `build.sbt` file:\n\n```scala\nexcludeDependencies += \"org.scala-lang.modules\" % \"scala-collection-compat_2.13\"\n```\n\n## Example\n\nLet's write a simple Kafka producer and consumer using zio-kafka with zio-streams. Before everything, we need a running instance of Kafka. We can do that by saving the following docker-compose script in the `docker-compose.yml` file and run `docker compose up`:\n\n```docker-compose\nservices:\n  broker:\n    image: apache/kafka:3.9.0\n    container_name: broker\n    ports:\n      - \"9092:9092\"\n    environment:\n      KAFKA_NODE_ID: 1\n      KAFKA_BROKER_ID: 1\n      KAFKA_PROCESS_ROLES: broker,controller\n      KAFKA_LISTENERS: PLAINTEXT://broker:29092,CONTROLLER://broker:29093,PLAINTEXT_HOST://0.0.0.0:9092\n      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092\n      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER\n      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT,CONTROLLER:PLAINTEXT\n      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@broker:29093\n      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1\n      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1\n      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1\n      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0\n      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT\n```\n\nNow, we can run our ZIO Kafka Streaming application:\n\n```scala\nimport zio._\nimport zio.kafka.consumer._\nimport zio.kafka.producer.{ Producer, ProducerSettings }\nimport zio.kafka.serde._\nimport zio.stream.ZStream\n\nobject ReadmeExample extends ZIOAppDefault {\n\n  private val producerRun: ZIO[Any, Throwable, Unit] =\n    ZIO.scoped {\n      for {\n        producer \u003c-\n          Producer.make(\n            ProducerSettings(List(\"localhost:9092\"))\n          )\n        _ \u003c- ZStream\n          .fromSchedule(Schedule.fixed(2.seconds))\n          .mapZIO(_ =\u003e Random.nextIntBetween(0, Int.MaxValue))\n          .mapZIO { random =\u003e\n            producer.produce[Any, Long, String](\n              topic = \"random-topic\",\n              key = (random % 4).toLong,\n              value = random.toString,\n              keySerializer = Serde.long,\n              valueSerializer = Serde.string\n            )\n          }\n          .runDrain\n      } yield ()\n    }\n\n  private val consumerRun: ZIO[Any, Throwable, Unit] =\n    ZIO.scoped {\n      for {\n        consumer \u003c-\n          Consumer.make(\n            ConsumerSettings(List(\"localhost:9092\"))\n              .withGroupId(\"group\")\n          )\n        _ \u003c- consumer\n          .plainStream(Subscription.topics(\"random\"), Serde.long, Serde.string)\n          .tap(r =\u003e Console.printLine(r.value))\n          .map(_.offset)\n          .aggregateAsync(Consumer.offsetBatches)\n          .mapZIO(_.commit)\n          .runDrain\n      } yield ()\n    }\n\n  override def run: ZIO[Any, Throwable, Unit] =\n    ZIO.raceFirst(producerRun, List(consumerRun))\n}\n```\n\n## Documentation\n\nIf you are reading this page from https://zio.dev/zio-kafka/, the documentation is in the sub-pages and can be found by\nscrolling the left panel of this page.\n\nIf you are reading this from GitHub, the documentation can be found in\n[the docs folder](https://github.com/zio/zio-kafka/tree/master/docs).\n\n## Resources\n\n### Articles\n\n- [ZIO Kafka tutorial](https://zio.dev/guides/tutorials/producing-consuming-data-from-kafka-topics/) by the zio-kafka team\n- [ZIO Kafka faster than Java Kafka](https://day-to-day-stuff.blogspot.com/2024/12/zio-kafka-faster-than-java-kafka.html) by Erik van Oosten (December 2024)\n- [Introduction to zio-kafka](https://www.baeldung.com/scala/zio-kafka-intro) by Stefanos Georgakis, Baeldung (January 2023)\n- [How to implement streaming microservices with ZIO 2 and Kafka](https://scalac.io/blog/streaming-microservices-with-zio-and-kafka/) by Jorge Vasquez, Scalac (June 2023)\n- [Zio Kafka](https://medium.com/@knoldus/zio-kafka-d865fc20174a) by Knoldus Inc (January 2022)\n- [Writing a Simple Producer and Consumer Using ZIO Workflows](https://pramodshehan.medium.com/writing-a-simple-producer-and-consumer-using-zio-workflows-a57def08210c) by Pramod Shehan (December 2022)\n- [ZIO Kafka with transactions - a debugging story](https://www.ziverge.com/post/zio-kafka-with-transactions---a-debugging-story/) by Daniel Vigovszky, Ziverge (June 2022)\n- [Introduction to Zio-Kafka](https://blog.knoldus.com/introduction-to-zio-kafka/) by Akash Kumar (March 2022)\n- [Introduction to Zio-Kafka](https://blog.nashtechglobal.com/introduction-to-zio-kafka/) by Khalid Ahmed, Nash Tech (March 2022)\n- [ZIO Kafka: A Practical Streaming Tutorial](https://rockthejvm.com/articles/zio-kafka/) by Riccardo Cardin, Rock the JVM (August 2021)\n- [Using ZIO Kafka with offset storage in Postgres for transactional processing](https://functional.works-hub.com/learn/using-zio-kafka-with-offset-storage-in-postgres-for-transactional-processing-be4a2) by Marek Kadek (March 2021)\n- [Streaming microservices with ZIO and Kafka](https://scalac.io/streaming-microservices-with-zio-and-kafka/) by Aleksandar Skrbic (February 2021)\n- [An Introduction to ZIO Kafka](https://www.ziverge.com/post/an-introduction-to-zio-kafka/) by Ziverge (April 2020)\n\n### Video\n\n- [Free ZIO Kafka course](https://www.learnscala.dev/) Free video training courses by Alvin Alexander, sponsored by Ziverge (2025) to learn Scala 3, Functional Programming, and ZIO 2. The zio-kafka module can be followed separately.\n- [Optimizing Data Transfer Kafka to BQ: let's use Scala to make it custom](https://www.youtube.com/watch?v=McnC2UU-RIE) by Dario Amorosi, Adevinta (November 2024)\n- [Making ZIO-Kafka Safer and Faster in 2023](https://www.youtube.com/watch?v=MJoRwEyyVxM) by Erik van Oosten (November 2023)\n- [ZIO Kafka with Scala: A Tutorial](https://www.youtube.com/watch?v=ExFjjczwwHs) by Rock the JVM (August 2021)\n- [ZIO WORLD - ZIO Kafka](https://www.youtube.com/watch?v=GECv1ONieLw) by Aleksandar Skrbic (March 2020) — Aleksandar Skrbic presented ZIO Kafka, a critical library for the modern Scala developer, which hides some of the complexities of Kafka.\n\n### Example projects\n\n- [Kafka BigQuery Express](https://github.com/adevinta/kafka-bigquery-express/) by Adevinta (November 2024) A production system to copy data from Kafka to BigQuery, safely and cost-effectively. (See also the video \"Optimizing Data Transfer...\".)\n- [zio-kafka-showcase](https://github.com/ScalaConsultants/zio-kafka-showcase) by Jorge Vásquez, Example project that demonstrates how to build Kafka based microservices with Scala and ZIO\n- [zio-kafka-demo1](https://github.com/pramodShehan5/zio-kafka-demo1) (December 2022), example consumer and producer using zio-kafka 2.0.5\n- [zio-kafka-example-app](https://github.com/zivergetech/zio-kafka-example-app) by Ziverge (December 2020), example application using zio-kafka 0.8.0\n\n## Adopters\n\nHere is a partial list of companies using zio-kafka in production.\n\nWant to see your company here? [Submit a PR](https://github.com/zio/zio-kafka/edit/master/docs/index.md)!\n\n* [Conduktor](https://www.conduktor.io)\n* [KelkooGroup](https://www.kelkoogroup.com)\n* [Rocker](https://rocker.com)\n\n## Performance\n\nOften, _zio-kafka programs consume with a higher throughput_ than programs that use the java-kafka client directly.\nRead on for the details.\n\nBy default, zio-kafka programs process partitions in parallel. The default java-kafka client does not provide parallel\nprocessing. Of course, there is some overhead in buffering records and distributing them to the fibers that need them.\nOn 2024-11-23, we estimated that zio-kafka consumes faster than the java-kafka client when processing takes more than\n~1.2ms per 1000 records. The precise time depends on many factors. Please\nsee [this article](https://day-to-day-stuff.blogspot.com/2024/12/zio-kafka-faster-than-java-kafka.html) for more\ndetails.\n\nIf you do not care for the convenient ZStream based API that zio-kafka brings, and latency is of absolute importance,\nusing the java based Kafka client directly is still the better choice.\n\n## Developers\n\n* [Benchmarks and Flame graphs](https://github.com/zio/zio-kafka/blob/master/zio-kafka-bench/README.md)\n\n## Documentation\n\nLearn more on the [ZIO Kafka homepage](https://zio.dev/zio-kafka)!\n\n## Contributing\n\nFor the general guidelines, see ZIO [contributor's guide](https://zio.dev/contributor-guidelines).\n\n## Code of Conduct\n\nSee the [Code of Conduct](https://zio.dev/code-of-conduct)\n\n## Support\n\nCome chat with us on [![Badge-Discord]][Link-Discord].\n\n[Badge-Discord]: https://img.shields.io/discord/629491597070827530?logo=discord \"chat on discord\"\n[Link-Discord]: https://discord.gg/2ccFBr4 \"Discord\"\n\n## Credits\n\nThis library is heavily inspired and made possible by the research and implementation done in [Alpakka Kafka](https://github.com/akka/alpakka-kafka), a library maintained by the Akka team and originally written as Reactive Kafka by SoftwareMill.\n\n## License\n\n[License](LICENSE)\n\nCopyright 2021-2026 Itamar Ravid and the zio-kafka contributors.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzio%2Fzio-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzio%2Fzio-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzio%2Fzio-kafka/lists"}