{"id":19426605,"url":"https://github.com/oscaro/felice","last_synced_at":"2025-10-11T08:03:50.866Z","repository":{"id":39535499,"uuid":"506942023","full_name":"oscaro/felice","owner":"oscaro","description":"Felice is a simple client library for Apache Kafka in Clojure, with JSON, Transit, Nippy and more!","archived":false,"fork":false,"pushed_at":"2025-08-30T23:44:18.000Z","size":753,"stargazers_count":8,"open_issues_count":1,"forks_count":3,"subscribers_count":16,"default_branch":"devel","last_synced_at":"2025-10-01T18:31:03.209Z","etag":null,"topics":["clojure","kafka","kafka-client"],"latest_commit_sha":null,"homepage":"https://oscaro.github.io/felice","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oscaro.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-06-24T08:52:40.000Z","updated_at":"2025-08-28T11:54:26.000Z","dependencies_parsed_at":"2025-04-24T17:41:10.834Z","dependency_job_id":"d4ef35e8-74ee-4c11-915b-e26641f3f625","html_url":"https://github.com/oscaro/felice","commit_stats":{"total_commits":98,"total_committers":7,"mean_commits":14.0,"dds":0.6224489795918368,"last_synced_commit":"4d6c1910a15774f6b4e0a9d2545f16982f1f28f9"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/oscaro/felice","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscaro%2Ffelice","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscaro%2Ffelice/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscaro%2Ffelice/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscaro%2Ffelice/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oscaro","download_url":"https://codeload.github.com/oscaro/felice/tar.gz/refs/heads/devel","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oscaro%2Ffelice/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006746,"owners_count":26084148,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","kafka","kafka-client"],"created_at":"2024-11-10T14:08:19.113Z","updated_at":"2025-10-11T08:03:50.839Z","avatar_url":"https://github.com/oscaro.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Felice [![Clojars Project](https://img.shields.io/clojars/v/com.oscaro/felice.svg)](https://clojars.org/com.oscaro/felice) [![Clojure CI](https://github.com/oscaro/felice/actions/workflows/clojure.yml/badge.svg?branch=master)](https://github.com/oscaro/felice/actions/workflows/clojure.yml)\n\n\u003ca href=\"https://github.com/oscaro/felice\"\u003e\u003cimg\n  src=\"https://raw.githubusercontent.com/oscaro/felice/devel/.github/image.png\" alt=\"Felice\"\n  height=\"400\" align=\"right\"\u003e\u003c/a\u003e\n\nFelice is client library for [Apache Kafka](http://kafka.apache.org) in Clojure. \n\nBuilt with simplicity it mind, it support by default JSON, Transit \u0026 Nippy (Fast | LZ4) and\nprovide also custom Serializer / Deserializer mechanism\n\n\u003e Note: the version contains in a first time the [Kafka Client](https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients) then the felice version, separated by hyphen.\n\n## De/Serializers\n\n| TYPE                | KEYWORD      |\n|---------------------|--------------|\n| String              | `:string`    |\n| Json                | `:json`      |\n| Json safe           | `:json-safe` |\n| Transit MessagePack | `:t+mpack`   |\n| Transit Json        | `:t+json`    |\n| Nippy Fast          | `:nippy+fast`|\n| Nippy LZ4           | `:nippy+lz4` |\n\nBeware that any exception during the deserialization process (eg: malformed json) will be thrown by the poll call. \nThis may result in a silent dead poll-loop. \n\nUsing a safe deserializer is an option, it will return a map containing the raw value (as a string) \nand a `:felice.serialization/error` key containing the exception instead of the deserialized value of the record.\n\n## Producing records\n\n```clojure\n(require '[felice.producer :as fp])\n\n;see http://kafka.apache.org/documentation/#producerconfigs\n(let [producer (fp/producer {:bootstrap.servers \"localhost:9092\" ;required\n                             :key.serializer :string             ;required\n                             :value.serializer :t+json           ;required\n                             :close.timeout.ms Long/MAX_VALUE}\n  ;; publish without key\n  (fp/send! producer \"topic1\" \"value\")\n  ;; publish with a key\n  (fp/send! producer \"topic2\" \"key\" \"value\")\n  ;; map syntax\n  (fp/send! producer {:topic \"topic3\" :key \"key\" :value \"value\"})\n  ;; remember to close you producer\n  (fp/close! producer))\n```\n\n## Consuming records\n\n```clojure\n(require '[felice.consumer :as fc])\n\n(defn print-record [{:keys [topic partition offset timestamp key value]}]\n   (println (format \"Record %s(%d) %d @%d - Key = %s, Value = %s\"\n                    topic partition offset timestamp key value)))\n\n;see http://kafka.apache.org/documentation/#producerconfigs\n(def consumer-cfg {:bootstrap.servers \"localhost:9092\" ;required\n                    :group.id \"my-group\"               ;required\n                    :auto.offset.reset \"latest\"        ;or \"earliest\", used at first startup only\n                    :key.serializer    :string         ;required\n                    :value.serializer  :json           ;required\n                    :enable.auto.commit true\n                    :auto.commit.interval.ms 5000      ;delay between auto commits\n                    :topics #{\"topic1\" \"topic2\"}       ;auto subscribes at startup\n                    :close.timeout.ms Long/MAX_VALUE})\n```\n\n### Do it yourself\n\n```clojure\n(let [consumer (fc/consumer consumer-cfg)]\n\n;; subscribe can take multiple topics at once\n  (fc/subscribe consumer \"topic1\" \"topic2\")\n;; and keep your previous subscriptions\n  (fc/subscribe consumer \"topic3\")\n\n;; basic polling\n  (let [cr (fc/poll consumer 100)\n        records (fc/consumer-records-\u003eall-records cr)]\n    (doseq [record records] (print-record record))\n    (fc/close! consumer))\n```\n\n### Using `poll-loop` or `poll-loops`\n\n```clojure\n;; poll-loop\n  (let [stop-fn (fc/poll-loop consumer-cfg print-record)]\n    ;; consumes in a future thread\n    ;; call the returned fn when you want to stop polling\n    (stop-fn))\n;; poll-loops\n  (let [stop-fn (fc/poll-loops consumer-cfg print-record {:threads-by-topic 4})]\n    ;; spawns 4 consumers by registered topic, each in its own future thread\n    ;; call the returned fn when you want to stop polling\n    (stop-fn))\n```\n\n#### Commit policy\n\n* `:never`   does nothing (use it if you enabled client auto commit)\n* `:poll`    commit last read offset after processing all the items of a poll\n* `:record`  commit the offset of every processed record\n\n#### Multi-threading\n\nYou can set either :threads-by-topic or :threads option (if both are set, :threads-by-topic will win)\n* `:threads`           spawn N threads total (each thread listening all registered topic)\n* `:threads-by-topic`  spawn N threads for each registered topic\n* you can also provide a map {:topic :threads} instead of a list of topics\n\n#### Examples\n\n```clojure\n;; Commit after each record processed spawning 8 threads (4 for topic1 and 4 for topic2)\n(fc/poll-loops consumer-cfg print-record [\"topic1\" \"topic2\"] {:commit-policy :record :threads-by-topic 4})\n;; No committing 1 thread for topic1 and 4 for topic2\n(fc/poll-loops consumer-cfg print-record {\"topic1\" 1 \"topic2\" 4} {:commit-policy :never})\n```\n\n\n#### Administration\n\nThe `felice.admin` provide interface to `Admin` kafka class, used to perform administration\ntasks on your cluster:\n\n```clojure\n(require [felice.admin :as fa])\n\n(with-open [ac (fa/admin-client {:bootstrap.servers \"localhost:9092\"})]\n  (fa/list-topics ac) ;;=\u003e #{\"foo\" \"bar\"}\n  )\n```\nSee all the administration methods [on documentation page](https://oscaro.github.io/felice/#felice.admin).\n\n\n#### Partitionning\n\nThe partitionner used by a producer can be set using this [configuration key](https://kafka.apache.org/documentation/#producerconfigs_partitioner.class).\n\nSee [default](https://github.com/a0x8o/kafka/blob/master/clients/src/main/java/org/apache/kafka/clients/producer/internals/DefaultPartitioner.java#L65-L71) partitionner implementation\n\nThere is a comment at the bottom of the felice.producer namespace mimiking the default partitionner if you want to see the result for some keys.\n\n```clojure\n;; default partitionner\n(import 'org.apache.kafka.common.utils.Utils)\n(defn partition-from-bytes [partition-count bytes]\n  (mod (Utils/toPositive (Utils/murmur2 bytes)) partition-count))\n(defn partition-from-string [partition-count string]\n  (partition-from-bytes partition-count (.getBytes string)))\n```\n\n## License\n\nCopyright © 2018 - 2024 Oscaro\n\nThis program and the accompanying materials are made available under the\nterms of the Eclipse Public License 2.0 which is available at\nhttp://www.eclipse.org/legal/epl-2.0.\n\nThis Source Code may also be made available under the following Secondary\nLicenses when the conditions for such availability set forth in the Eclipse\nPublic License, v. 2.0 are satisfied: GNU General Public License as published by\nthe Free Software Foundation, either version 2 of the License, or (at your\noption) any later version, with the GNU Classpath Exception which is available\nat https://www.gnu.org/software/classpath/license.html.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foscaro%2Ffelice","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foscaro%2Ffelice","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foscaro%2Ffelice/lists"}