{"id":13692753,"url":"https://github.com/ccann/gregor","last_synced_at":"2025-05-02T19:32:38.000Z","repository":{"id":62433079,"uuid":"80588247","full_name":"ccann/gregor","owner":"ccann","description":"Lightweight Clojure bindings for Apache Kafka","archived":true,"fork":true,"pushed_at":"2021-01-25T15:56:40.000Z","size":166,"stargazers_count":23,"open_issues_count":2,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-06T12:48:22.723Z","etag":null,"topics":["clojure","java","kafka","queue"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"vrih/gregor","license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ccann.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-02-01T04:05:51.000Z","updated_at":"2023-01-28T05:36:48.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ccann/gregor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccann%2Fgregor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccann%2Fgregor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccann%2Fgregor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccann%2Fgregor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ccann","download_url":"https://codeload.github.com/ccann/gregor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252095448,"owners_count":21693919,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","java","kafka","queue"],"created_at":"2024-08-02T17:01:01.688Z","updated_at":"2025-05-02T19:32:37.636Z","avatar_url":"https://github.com/ccann.png","language":"Clojure","funding_links":[],"categories":["Kafka Producers \u0026 Consumers"],"sub_categories":[],"readme":"# Gregor\n\nLightweight Clojure wrapper for [Apache Kafka](http://kafka.apache.org/) Consumer + Producer APIs.\n\n```clojure\n :dependencies [[org.clojure/clojure \"1.10.0\"]\n                [org.apache.kafka/kafka_2.12 \"2.1.1\"]]\n\n```\n[![cljdoc Badge](https://cljdoc.org/badge/io.weft/gregor)](https://cljdoc.org/d/io.weft/gregor/CURRENT)\n\n[![Clojars Project](https://clojars.org/io.weft/gregor/latest-version.svg)](https://clojars.org/io.weft/gregor)\n\n[**CHANGELOG**](https://github.com/ccann/gregor/blob/master/CHANGELOG.md)\n\nGregor wraps most of the Java API for the Kafka [Producer](http://kafka.apache.org/0100/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html) and [New Consumer](http://kafka.apache.org/0100/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html) The goal of this project is to stay very close to the Kafka API instead of adding more advanced features.\n\n## Example\n\nHere's an example of at-least-once processing (using [`mount`](https://github.com/tolitius/mount)):\n\n```clojure\n(ns gregor-sample-app.core\n  (:gen-class)\n  (:require [clojure.repl :as repl]\n            [gregor.core :as gregor]\n            [mount.core :as mount :refer [defstate]]))\n\n(def run (atom true))\n\n(defstate consumer\n  :start (gregor/consumer \"localhost:9092\"\n                          \"testgroup\"\n                          [\"test-topic\"]\n                          {\"auto.offset.reset\" \"earliest\"\n                           \"enable.auto.commit\" \"false\"})\n  :stop (gregor/close consumer))\n\n(defstate producer\n  :start (gregor/producer \"localhost:9092\")\n  :stop (gregor/close producer))\n\n(defn -main\n  [\u0026 args]\n  (mount/start)\n  (repl/set-break-handler! (fn [sig] (reset! run false)))\n  (while @run\n    (let [consumer-records (gregor/poll consumer)\n          values (process-records consumer-records)]\n      (doseq [v values]\n        (gregor/send producer \"other-topic\" v))\n      (gregor/commit-offsets! consumer)))\n  (mount/stop))\n```\n\nTransformations over consumer records are applied in `process-records`. Each record in\nthe `seq` returned by `poll` is a map. Here's an example with a JSON object as the\n`:value`:\n\n```clojure\n{:value \"{\\\"foo\\\":42}\"\n :key nil\n :partition 0\n :topic \"test-topic\"\n :offset 939}\n```\n\n## Producing\n\nGregor provides the `send` function for asynchronously sending a record to a topic. There\nare multiple arities which correspond to those of the `ProducerRecord`\n[Java constructor](https://kafka.apache.org/0100/javadoc/org/apache/kafka/clients/producer/ProducerRecord.html). If\nyou'd like to provide a callback to be invoked when the send has been acknowledged use\n`send-then` instead.\n\n## Topic Management\n\nCreate a topic:\n\n```clojure\n(create-topic {:connection-string \"localhost:2181\"} \"some-topic\" {})\n```\nThat empty map can be used to specify configuration for number of topic partitions, replication factor,\n\nDelete a topic:\n\n``` clojure\n(delete-topic {:connection-string \"localhost:2181\"} \"some-topic\")\n```\n\nQuery about a topic's existence:\n\n``` clojure\n(topic-exists? {:connection-string \"localhost:2181\"} \"some-topic\")\n```\n\nList existing topics:\n\n``` clojure\n(topics {:connection-string \"localhost:2181\"})\n```\n\n\n## License\n\nDistributed under the Eclipse Public License either version 1.0 or (at your option) any\nlater version.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fccann%2Fgregor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fccann%2Fgregor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fccann%2Fgregor/lists"}