{"id":15010637,"url":"https://github.com/source-c/clj-kafka-x","last_synced_at":"2025-04-09T18:34:55.485Z","repository":{"id":47227487,"uuid":"75947003","full_name":"source-c/clj-kafka-x","owner":"source-c","description":"Clojure client for Apache Kafka","archived":false,"fork":false,"pushed_at":"2023-07-03T19:24:15.000Z","size":92,"stargazers_count":6,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-06T05:16:30.380Z","etag":null,"topics":["apache-kafka","automation","clj","clojure-library","dkdhub","kafka","kafka-client","kafka-consumer","kafka-producer","tbt-cloud"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/source-c.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-08T14:54:10.000Z","updated_at":"2023-01-07T03:57:25.000Z","dependencies_parsed_at":"2024-09-28T17:40:38.473Z","dependency_job_id":"338f1653-9022-4bcc-9469-d8e8d8fdffee","html_url":"https://github.com/source-c/clj-kafka-x","commit_stats":{"total_commits":75,"total_committers":4,"mean_commits":18.75,"dds":0.36,"last_synced_commit":"6c604599e4ed62e9f99cd496484e23c10fd30b5b"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/source-c%2Fclj-kafka-x","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/source-c%2Fclj-kafka-x/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/source-c%2Fclj-kafka-x/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/source-c%2Fclj-kafka-x/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/source-c","download_url":"https://codeload.github.com/source-c/clj-kafka-x/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248088336,"owners_count":21045693,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-kafka","automation","clj","clojure-library","dkdhub","kafka","kafka-client","kafka-consumer","kafka-producer","tbt-cloud"],"created_at":"2024-09-24T19:35:11.045Z","updated_at":"2025-04-09T18:34:55.441Z","avatar_url":"https://github.com/source-c.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"= clj-kafka-x\n\nA Clojure library for the Apache Kafka (distributed stream-processing software platform).\n\nUses KF protocol and does not rely on ZooKeeper.\n\nNOTE: Still contains zookeeper as a dependency.\n\nTries to be as lightweigh as possible thus depends only on\n\n- `org.apache.kafka/kafka_2.12 \"3.4.0\"`\n- `org.apache.kafka/kafka-clients \"3.4.0\"`\n- `org.apache.zookeeper/zookeeper \"3.8.1\"`\n\nbut excluding `jms`,`jmx*` and logging.\n\nNOTE: Some builds (for instance of `v0.4.x` branch) may partially (sometimes even fully) be incompatible with some versions of other libraries that also use NIO! If you’re experiencing build problems and/or your application is unexpectedly crashed on start - try check your project dependencies more deeply, may be you will need to correct existing dependencies version or to add an actual version of full `[io.netty/netty-all]`\n\nActual library info:\n\nimage:https://img.shields.io/github/license/source-c/clj-kafka-x?style=for-the-badge[GitHub]\nimage:https://img.shields.io/clojars/v/net.tbt-post/clj-kafka-x.svg?style=for-the-badge[]\nimage:https://img.shields.io/clojars/dt/net.tbt-post/clj-kafka-x?style=for-the-badge[ClojarsDownloads]\nimage:https://img.shields.io/github/v/release/source-c/clj-kafka-x?style=for-the-badge[GitHub release (latest by date)]\nimage:https://img.shields.io/github/release-date/source-c/clj-kafka-x?style=for-the-badge[GitHub Release Date]\nimage:https://img.shields.io/github/v/tag/source-c/clj-kafka-x?style=for-the-badge[GitHub tag (latest by date)]\nimage:https://img.shields.io/github/last-commit/source-c/clj-kafka-x?style=for-the-badge[GitHub last commit]\n\n\n== Installation\n\nAdd the following to your http://github.com/technomancy/leiningen[Leiningen's]\n`project.clj`:\n\n[source,clojure]\n----\n[net.tbt-post/clj-kafka-x \"0.7.5\"]\n----\n\n== Usage\n\n=== Producer\n\n[source,clojure]\n----\n(require '[clj-kafka-x.producer :as kp])\n\n(with-open [p (kp/producer {\"bootstrap.servers\" \"localhost:9092\"}\n                           (kp/string-serializer)\n                           (kp/string-serializer))]\n  @(kp/send p (kp/record \"topic-a\" \"Hi there!\")))\n----\n\n=== Consumer\n\n[source,clojure]\n----\n(require '[clj-kafka-x.consumers.simple :as kc])\n\n(with-open [c (kc/consumer {\"bootstrap.servers\" \"localhost:9092\"\n                            \"group.id\" \"consumer-id\"}\n                            (kc/string-deserializer)\n                            (kc/string-deserializer))]\n  (kc/subscribe c \"topic-a\")\n  (kc/messages c))\n----\n\nNOTE: When you use multiple partitions per topic it is required\nto specify them explicitly when subscribing, i.e.\n`(kc/subscribe\n    c [{:topic \"topic-a\" :partitions #{0 1}}\n       {:topic \"topic-b\" :partitions #{0 1 2}}])`\n\n.Real-life (almost) example\n[source,clojure]\n----\n(ns buzz.consumer.kafka\n  (:require [clj-kafka-x.consumers.simple :as kc]\n            [clojure.tools.logging :as log]))\n\n(defn processor [msg schema] msg)\n(def schema nil)\n(def config {\"bootstrap.servers\" \"localhost:9092\"\n             \"group.id\" \"consumer-id\"})\n\n(defn process-message [msg]\n  (let [{:keys [value topic partition offset]} msg\n        processor processor ;; choose one by topic name\n        schema schema]      ;; choose one by topic name\n    (if (fn? processor) (processor value schema) value)))\n\n(defn consume []\n  (with-open [c (kc/consumer config\n                             (kc/byte-array-deserializer)\n                             (kc/byte-array-deserializer))]\n    (kc/subscribe c (config/kafka-topics))\n    (let [pool (kc/messages c)]\n      (doseq [message pool]\n        (log/warn (process-message message))))))\n----\n\nyou may also use specific timeouts form\n\n[source, clojure]\n----\n(defn- consume [instance process-message]\n  (when-let [co (kc/consumer config\n                             (kc/byte-array-deserializer)\n                             (kc/byte-array-deserializer))\n             messages (kc/messages\n                        co\n                        :timeout (:request-timeout-ms config))]\n    (doall (map process-message messages))))\n----\n\nmessage count per poll execution may be specified by `max.poll.records` field of configuration\n\n== Manual Build\n\n[source,text]\n----\n$ lein install\n----\n\n== License\n\nCopyright © 2016-2023\n\nDistributed under the\nhttp://www.apache.org/licenses/LICENSE-2.0[Apache License v 2.0]\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsource-c%2Fclj-kafka-x","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsource-c%2Fclj-kafka-x","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsource-c%2Fclj-kafka-x/lists"}