{"id":13566468,"url":"https://github.com/Banno/kafka4s","last_synced_at":"2025-04-04T00:30:48.190Z","repository":{"id":37088993,"uuid":"193121841","full_name":"Banno/kafka4s","owner":"Banno","description":"Functional programming with Kafka and Scala","archived":false,"fork":false,"pushed_at":"2024-10-29T19:49:04.000Z","size":3298,"stargazers_count":92,"open_issues_count":11,"forks_count":18,"subscribers_count":11,"default_branch":"main","last_synced_at":"2024-10-29T21:42:48.828Z","etag":null,"topics":["cats","cats-effect","fp","fs2","functional-programming","kafka","scala","stream-processing"],"latest_commit_sha":null,"homepage":"https://banno.github.io/kafka4s","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Banno.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-21T15:30:50.000Z","updated_at":"2024-10-29T19:47:28.000Z","dependencies_parsed_at":"2023-09-28T01:45:10.544Z","dependency_job_id":"61a6ab33-4489-4ed6-bc63-97bfd5719fda","html_url":"https://github.com/Banno/kafka4s","commit_stats":null,"previous_names":[],"tags_count":99,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Banno%2Fkafka4s","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Banno%2Fkafka4s/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Banno%2Fkafka4s/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Banno%2Fkafka4s/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Banno","download_url":"https://codeload.github.com/Banno/kafka4s/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247103306,"owners_count":20884023,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cats","cats-effect","fp","fs2","functional-programming","kafka","scala","stream-processing"],"created_at":"2024-08-01T13:02:10.375Z","updated_at":"2025-04-04T00:30:45.586Z","avatar_url":"https://github.com/Banno.png","language":"Scala","funding_links":[],"categories":["Scala"],"sub_categories":[],"readme":"# kafka4s - Functional programming with Kafka and Scala\n\n![CI](https://github.com/Banno/kafka4s/workflows/CI/badge.svg)\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.banno/kafka4s_2.13/badge.svg)](https://maven-badges.herokuapp.com/maven-central/com.banno/kafka4s_2.13)\n[![Javadocs](https://www.javadoc.io/badge/com.banno/kafka4s_2.13.svg?color=red\u0026label=scaladoc)](https://www.javadoc.io/doc/com.banno/kafka4s_2.13/latest/com/banno/kafka/index.html)\n[![License](http://img.shields.io/:license-Apache%202-green.svg)](http://www.apache.org/licenses/LICENSE-2.0.txt)\n![Code of Conduct](https://img.shields.io/badge/Code%20of%20Conduct-Scala-blue.svg)\n\nkafka4s provides pure, referentially transparent functions for working with Kafka, and integrates with FP libraries such as [cats-effect](https://typelevel.org/cats-effect) and [fs2](https://fs2.io).\n\n## [Head on over to the microsite](https://banno.github.io/kafka4s)\n\n## Quick Start\n\nTo use `kafka4s` in an existing SBT project with Scala 2.13 or a later version, add the following dependencies to your\n`build.sbt` depending on your needs:\n\n```scala\nlibraryDependencies ++= Seq(\n  \"com.banno\" %% \"kafka4s\" % \"\u003cversion\u003e\"\n)\n```\n\nNote: If your project uses fs2 1.x, you'll want releases from the 2.x series. For fs2 2.x projects, you'll want 3.x series releases.\n\nSending records to Kafka is an effect. If we wanted to periodically write random integers to a Kafka topic, we could do:\n\n```scala\nStream\n  .resource(ProducerApi.resource[F, Int, Int](BootstrapServers(kafkaBootstrapServers)))\n  .flatMap { producer =\u003e\n    Stream\n      .awakeDelay[F](1.second)\n      .evalMap { _ =\u003e\n        Sync[F].delay(Random.nextInt()).flatMap { i =\u003e\n          producer.sendAndForget(new ProducerRecord(topic.name, i, i))\n        }\n      }\n  }\n```\n\nPolling Kafka for records is also an effect, and we can obtain a stream of records from a topic. We can print the even random integers from the above topic using:\n\n```scala\nStream.resource(\n   ConsumerApi\n      .resource[F, Int, Int](\n        BootstrapServers(kafkaBootstrapServers),\n        GroupId(\"example3\"),\n        AutoOffsetReset.earliest,\n        EnableAutoCommit(true)\n      )\n  )\n  .evalTap(_.subscribe(topic.name))\n  .flatMap(\n    _.recordStream(1.second)\n      .map(_.value)\n      .filter(_ % 2 == 0)\n      .evalMap(i =\u003e Sync[F].delay(println(i)))\n  )\n```\n\n## Learning more\n\nTo learn more about kafka4s, start with our [Getting Started Guide](https://banno.github.io/kafka4s/docs/), play with some [example apps](https://github.com/Banno/kafka4s/tree/master/examples/src/main/scala), and check out the [kafka4s Scaladoc](https://www.javadoc.io/doc/com.banno/kafka4s_2.13) for more info.\n\n## Running the examples\n\nTo run the examples, setup the following:\n- Pull down the docker image in the project directory: \n```\ndocker-compose up -d\n```\n- Add local host alias `kafka.local` to your machines `/etc/hosts` file. You will need to use sudo access to edit this file.\n- Run example in sbt, for example: \n```\nexamples/runMain example3.CatsEffectApp\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBanno%2Fkafka4s","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBanno%2Fkafka4s","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBanno%2Fkafka4s/lists"}