{"id":16729844,"url":"https://github.com/dfdx/rdkafka.jl","last_synced_at":"2026-01-28T04:35:17.627Z","repository":{"id":51258161,"uuid":"130768346","full_name":"dfdx/RDKafka.jl","owner":"dfdx","description":"Wrapper for librdkafka","archived":false,"fork":false,"pushed_at":"2024-11-13T23:07:02.000Z","size":83,"stargazers_count":42,"open_issues_count":4,"forks_count":24,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-02-23T14:05:31.644Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dfdx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-23T23:27:19.000Z","updated_at":"2025-01-19T11:05:52.000Z","dependencies_parsed_at":"2022-09-07T21:00:48.607Z","dependency_job_id":"1ff0abb7-684d-4527-942a-22ff1435327c","html_url":"https://github.com/dfdx/RDKafka.jl","commit_stats":{"total_commits":34,"total_committers":10,"mean_commits":3.4,"dds":0.7352941176470589,"last_synced_commit":"d3dbfed29979ca6766a34426e36f5114237787c3"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfdx%2FRDKafka.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfdx%2FRDKafka.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfdx%2FRDKafka.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dfdx%2FRDKafka.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dfdx","download_url":"https://codeload.github.com/dfdx/RDKafka.jl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243763216,"owners_count":20344184,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-12T23:30:08.284Z","updated_at":"2026-01-28T04:35:17.594Z","avatar_url":"https://github.com/dfdx.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RDKafka\n\nJulia wrapper for [librdkafka](https://github.com/edenhill/librdkafka).\n\n## Build\n\n```julia\nusing Pkg\nPkg.add(\"RDKafka\")\n```\n\nPrebuilt binaries of the `librdkafka` native library is downloaded. The binaries are available for all supported Julia platforms. \n\n## Usage\n\n### Start Kafka server\n\nIf you don't have one already started, download Kafka server and run it according to the official [QuickStart Guide](https://kafka.apache.org/quickstart). Here's a short version of that guide:\n\n`cd` to the kafka folder and in run the following commands in 2 different terminals:\n\n```\n# start ZooKeeper server\nbin/zookeeper-server-start.sh config/zookeeper.properties\n# start Kafka broker\nbin/kafka-server-start.sh config/server.properties\n```\n\nIn yet another terminal create a topic:\n\n```\n# create topic\nbin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server localhost:9092\n# describe it\nbin/kafka-topics.sh --describe --topic quickstart-events --bootstrap-server localhost:9092\n```\n\n### Produce and consume some messages\n\nNow in Julia console start polling using `KafkaConsumer`:\n\n```julia\nusing RDKafka\n\nc = KafkaConsumer(\"localhost:9092\", \"my-consumer-group\")\nparlist = [(\"quickstart-events\", 0)]\nsubscribe(c, parlist)\ntimeout_ms = 1000\nwhile true\n    msg = poll(String, String, c, timeout_ms)\n    @show(msg)\nend\n```\n\nAnd produce a few messages using `KafkaProducer`\n\n```julia\nusing RDKafka\nimport RDKafka.produce\n\np = KafkaProducer(\"localhost:9092\")\npartition = 0\nproduce(p, \"quickstart-events\", partition, \"message key\", \"message payload\")\n```\n\nIn the consumer window you should see something like:\n\n```\nmsg = nothing\nmsg = Message(message key: message payload)\nmsg = nothing\nmsg = nothing\n```\nwhere `nothing` means that there were no new messages in that polling interval while `Message(...)` is actual message we sent from producer.\n\n### Configuration\n\n`librdkafka` is highly customizable, see [CONFIGURATION.md](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) for the list of supported properties. To set a particular property, pass a `conf` object to `KafkaProducer` or `KafkaConsumer`, e.g.:\n\n```julia\nconf = Dict(\"socket.timeout.ms\" =\u003e 300000)\np = KafkaProducer(\"localhost:9092\", conf)\n```\n\n### Error handling\n\nAdd the `err_cb` argument to either KafkaConsumer or KafkaProducer.\n\n```\nc = KafkaConsumer(\"localhost:9092\", \"my-consumer-group\", err_cb=(err::Int, reason::String) -\u003e throw(error(reason)))\n```\n\n### Delivery reports\n\nAdd the `dr_cb` argument to a KafkaProducer.\n\n```\np = KafkaProducer(\"localhost:9092\", dr_cb=msg -\u003e if msg.err != 0 throw(error(\"Delivery failed\") end))\n```\n\n### Seeking\n\n```\nc = KafkaConsumer(\"localhost:9092\", \"my-consumer-group\")\nRDKafka.seek(c, timestamp_ms, timeout_ms)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfdx%2Frdkafka.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdfdx%2Frdkafka.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdfdx%2Frdkafka.jl/lists"}