{"id":18073471,"url":"https://github.com/iamazy/kafkas","last_synced_at":"2025-04-12T05:23:35.320Z","repository":{"id":101767148,"uuid":"576646239","full_name":"iamazy/kafkas","owner":"iamazy","description":"async kafka client in pure Rust","archived":false,"fork":false,"pushed_at":"2024-03-22T11:40:03.000Z","size":258,"stargazers_count":15,"open_issues_count":10,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-26T00:51:13.753Z","etag":null,"topics":["async-std","kafka","redpanda","rust","tokio"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iamazy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-12-10T14:18:04.000Z","updated_at":"2024-10-06T11:06:32.000Z","dependencies_parsed_at":"2023-07-17T02:15:08.408Z","dependency_job_id":null,"html_url":"https://github.com/iamazy/kafkas","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamazy%2Fkafkas","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamazy%2Fkafkas/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamazy%2Fkafkas/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iamazy%2Fkafkas/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iamazy","download_url":"https://codeload.github.com/iamazy/kafkas/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248520766,"owners_count":21117924,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async-std","kafka","redpanda","rust","tokio"],"created_at":"2024-10-31T10:07:42.648Z","updated_at":"2025-04-12T05:23:35.278Z","avatar_url":"https://github.com/iamazy.png","language":"Rust","readme":"# kafkas\n\nAsync kafka client in pure Rust.\n\n## Features\n\n- Multiple async runtime (`tokio`, `async-std`, etc.)\n- All versions of kafka are supported\n- Compression (`gzip`, `snappy`, `lz4`)\n\n## APIs\n\n- [x] Producer\n- [x] Consumer\n- [ ] Streams\n- [ ] Connect\n- [ ] Admin client\n\n## Usage\n\n```toml\n[dependencies]\nkafkas = { git = \"https://github.com/iamazy/kafkas\", branch = \"main\" }\n```\n\nTo get started using kafkas:\n\n- Producer\n\n```rust\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Box\u003cError\u003e\u003e {\n    let client = Kafka::new(\"127.0.0.1:9092\", KafkaOptions::default(), TokioExecutor).await?;\n\n    let producer = Producer::new(client, ProducerOptions::default()).await?;\n\n    let (mut tx, mut rx) = futures::channel::mpsc::unbounded();\n    tokio::task::spawn(Box::pin(async move {\n        while let Some(fut) = rx.next().await {\n            if let Err(e) = fut.await {\n                error!(\"{e}\");\n            }\n        }\n    }));\n\n    let topic = topic_name(\"kafka\");\n    for _ in 0..10000_0000 {\n        let record = TestData::new(\"hello kafka\");\n        let ret = producer.send(\u0026topic, record).await?;\n        let _ = tx.send(ret).await;\n    }\n}\n```\n\n- Consumer\n\n```rust\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Box\u003cError\u003e\u003e {\n    let client = Kafka::new(\"127.0.0.1:9092\", KafkaOptions::default(), TokioExecutor).await?;\n\n    let mut consumer_options = ConsumerOptions::new(\"default\");\n    consumer_options.auto_commit_enabled = false;\n    \n    let mut consumer = Consumer::new(kafka_client, consumer_options).await?;\n    \n    let consume_stream = consumer.subscribe::\u003c\u0026str, ConsumerRecord\u003e(vec![\"kafka\"]).await?;\n    pin_mut!(consume_stream);\n\n    while let Some(records) = consume_stream.next().await {\n        for record in records {\n            if let Some(value) = record.value {\n                println!(\"{:?} - {}\", String::from_utf8(value.to_vec())?, record.offset);\n            }\n        }\n        // needed only when `auto_commit_enabled` is false\n        consumer.commit_async().await?;\n    }\n}\n```\n\n## Examples\n\nExamples can be found in [`examples`](https://github.com/iamazy/kafkas/blob/main/examples).\n\n## Flame graph\n\n\u003cimg style=\"width:800px\" src=\"benches/flamegraph.svg\"  alt=\"flamegraph\"/\u003e\n\n## Rust version requirements\n\nThe rust version used for `kafkas` development is `1.65`.\n\n## Acknowledgments\n\n- [kafka-protocol-rs](https://github.com/tychedelia/kafka-protocol-rs) : Rust implementation of the [Kafka wire protocol](https://kafka.apache.org/protocol.html).\n- [pulsar-rs](https://github.com/streamnative/pulsar-rs) : Rust Client library for Apache Pulsar\n- [rskafka](https://github.com/influxdata/rskafka) : A minimal Rust client for Apache Kafka","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamazy%2Fkafkas","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiamazy%2Fkafkas","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiamazy%2Fkafkas/lists"}