https://github.com/bhftbootcamp/librdkafka.jl
Lightweight Julia wrapper for librdkafka (Apache Kafka)
https://github.com/bhftbootcamp/librdkafka.jl
kafka rdkafka
Last synced: 5 months ago
JSON representation
Lightweight Julia wrapper for librdkafka (Apache Kafka)
- Host: GitHub
- URL: https://github.com/bhftbootcamp/librdkafka.jl
- Owner: bhftbootcamp
- Created: 2026-02-06T09:27:27.000Z (6 months ago)
- Default Branch: master
- Last Pushed: 2026-02-06T11:07:52.000Z (6 months ago)
- Last Synced: 2026-02-06T17:39:08.523Z (6 months ago)
- Topics: kafka, rdkafka
- Language: C++
- Homepage:
- Size: 287 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Librdkafka.jl
[](https://deepwiki.com/bhftbootcamp/Librdkafka.jl)
[](https://bhftbootcamp.github.io/Librdkafka.jl/stable/)
[](https://bhftbootcamp.github.io/Librdkafka.jl/dev/)
[](https://github.com/bhftbootcamp/Librdkafka.jl/actions/workflows/ci.yml?query=branch%3Amaster)
[](https://codecov.io/gh/bhftbootcamp/Librdkafka.jl)
[](https://github.com/bhftbootcamp/Green)
Julia wrapper for librdkafka for producing and consuming Apache Kafka messages from Julia.
Library based on [modern-cpp-kafka](https://github.com/morganstanley/modern-cpp-kafka).
Platform
x86_64
aarch64
Notes
Linux (glibc)
✓
(planned)
Mainstream Linux distributions (x86_64 supported)
Linux (musl)
(planned)
(planned)
Alpine / minimal containers
macOS
(planned)
✓
Apple Silicon supported, Intel planned
FreeBSD
(planned)
(planned)
Community interest
Windows
(planned)
(planned)
Future support
Prebuilt binaries are published automatically for supported platforms. If you need to build the wrapper from source, run:
```
cmake -S deps/src -B deps/src/build && cmake --build deps/src/build
```
## Installation
If you haven't installed our local registry yet, do that first:
```
] registry add https://github.com/bhftbootcamp/Green.git
```
To install Librdkafka, simply use the Julia package manager:
```
] add Librdkafka
```
## Usage
### Producer
```julia
using Librdkafka
topic = "julia-demo"
p = KafkaProducer("localhost:9092")
try
i = 1
while true
msg = "hello #$i from julia"
produce(p, topic, 0, "key", msg)
@info "Sent" msg
i += 1
sleep(1)
end
finally
close(p) # unreachable here, Ctrl+C to stop
end
```
### Consumer
```julia
using Librdkafka
topic = "julia-demo"
c = KafkaConsumer(
"localhost:9092";
group_id = "julia-demo-consumer",
config = Dict(
AUTO_OFFSET_RESET => "earliest",
ENABLE_AUTO_COMMIT => "false",
),
)
subscribe!(c, [topic])
try
while true
for r in poll(c; timeout_ms=1000)
@info "Got" key=r.key value=r.value offset=r.offset
commit_record(c, r)
end
end
finally
close(c) # unreachable here, Ctrl+C to stop
end
```
## Useful Links
- [librdkafka](https://github.com/confluentinc/librdkafka) – Official library repository.
- [librdkafka_jll](https://github.com/JuliaBinaryWrappers/librdkafka_jll.jl) – Julia binary wrapper for librdkafka.
- [modern-cpp-kafka](https://github.com/morganstanley/modern-cpp-kafka) – a layer of C++ wrapper based on librdkafka.
- [doc kafka-clients](https://docs.confluent.io/kafka-clients/python/current/overview.html) – configuration refers to the various settings and parameters that can be adjusted to optimize the performance, reliability, and security of a Kafka cluster and its clients.
## Contributing
Contributions to Librdkafka are welcome! If you encounter a bug, have a feature request, or would like to contribute code, please open an issue or a pull request on GitHub.