{"id":16729830,"url":"https://github.com/dfdx/kafka.jl","last_synced_at":"2025-10-10T06:17:42.547Z","repository":{"id":138552137,"uuid":"58386841","full_name":"dfdx/Kafka.jl","owner":"dfdx","description":"Client for Apache Kafka in Julia","archived":false,"fork":false,"pushed_at":"2020-04-29T13:46:41.000Z","size":46,"stargazers_count":17,"open_issues_count":6,"forks_count":7,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-15T17:22:55.091Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dfdx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2016-05-09T15:20:08.000Z","updated_at":"2025-01-19T11:05:47.000Z","dependencies_parsed_at":null,"dependency_job_id":"217d2f9b-d5a0-421d-9dcb-253f9ab81f00","html_url":"https://github.com/dfdx/Kafka.jl","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/dfdx/Kafka.jl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfdx%2FKafka.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfdx%2FKafka.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfdx%2FKafka.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfdx%2FKafka.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dfdx","download_url":"https://codeload.github.com/dfdx/Kafka.jl/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfdx%2FKafka.jl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279002965,"owners_count":26083488,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T23:30:01.932Z","updated_at":"2025-10-10T06:17:42.542Z","avatar_url":"https://github.com/dfdx.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kafka\n\n[![Build Status](https://travis-ci.org/dfdx/Kafka.jl.svg?branch=master)](https://travis-ci.org/dfdx/Kafka.jl)\n\nClient for Apache Kafka in Julia\n\nThis project is **DISCONTINUED**. Please use [RDKafka.jl](https://github.com/dfdx/RDKafka.jl) instead.\n\n## Status\n\nBasically, Kafka provides 4 principal APIs:\n\n 1. Metadata retrieval \n 2. Producing messages\n 3. Fetching messages\n 4. Listing offsets\n 5. Consumer group management (a.k.a. storing offsets)\n\nFirst 4 are implemented and should be sufficient for most real-life use cases. The last one, however, is somewhat fast-moving target without single approach (e.g. Kafka 0.8.x uses Zookeeper to store offsets, 0.9.x provides broker API, while external systems like Apache Spark and Apache Storm use their own means to store offsets). Given instability and variety of options this part is postponed for now. Though, proposals and discussions are heavily welcome.\n\n## Installation\n```julia\n(v1.0) pkg\u003e add Kafka#master\n```\n\n## Usage example\n\nHere's short version of what you can do with Kafka.jl. For full example see `examples/all.jl`.\n\n```julia\nusing Kafka\n\n# create KafkaClient using single bootstrap broker\nkc = KafkaClient(\"127.0.0.1\", 9092)\n\n# get metadata about a topic(s)\nmd_channel = metadata(kc, [\"test\"])\nmd = take!(md_channel)\n# if you prefer synchronous logic, use one-linear\ntake!(metadata(kc, [\"test\"]))\ntake!(metadata(kc))\n\n# get earliest and latest available offsets for topic \"test\" and partition 0\ntake!(earliest_offset(kc, \"test\", 0))\ntake!(latest_offset(kc, \"test\", 0))\n\n# produce new messages\nkeys = [convert(Vector{UInt8}, key) for key in [\"1\", \"2\", \"3\"]]\nvalues = [convert(Vector{UInt8}, value) for value in [\"feel\", \"good\", \"inc.\"]]\nmessages = collect(zip(keys, values))\noffset = take!(produce(kc, \"test\", 0, messages))\n\n# fetch messages\nstart_offset = 0\noffset_messages = take!(fetch(kc, \"test\", 0, start_offset))\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfdx%2Fkafka.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdfdx%2Fkafka.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfdx%2Fkafka.jl/lists"}