{"id":13806922,"url":"https://github.com/cyngn/vertx-kafka","last_synced_at":"2026-01-28T08:16:33.466Z","repository":{"id":36436061,"uuid":"40741072","full_name":"cyngn/vertx-kafka","owner":"cyngn","description":null,"archived":false,"fork":false,"pushed_at":"2016-03-21T22:50:05.000Z","size":155,"stargazers_count":30,"open_issues_count":3,"forks_count":11,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-05-14T00:11:31.180Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cyngn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-08-15T00:19:22.000Z","updated_at":"2023-02-16T10:23:01.000Z","dependencies_parsed_at":"2022-08-17T23:05:37.501Z","dependency_job_id":null,"html_url":"https://github.com/cyngn/vertx-kafka","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cyngn/vertx-kafka","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cyngn%2Fvertx-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cyngn%2Fvertx-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cyngn%2Fvertx-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cyngn%2Fvertx-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cyngn","download_url":"https://codeload.github.com/cyngn/vertx-kafka/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cyngn%2Fvertx-kafka/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28842836,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T07:39:25.367Z","status":"ssl_error","status_checked_at":"2026-01-28T07:39:24.487Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T01:01:18.011Z","updated_at":"2026-01-28T08:16:33.441Z","avatar_url":"https://github.com/cyngn.png","language":"Java","funding_links":[],"categories":["Integration"],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/cyngn/vertx-kafka.svg?branch=master)](https://travis-ci.org/cyngn/vertx-kafka)\n\n# Vert.x Kafka\n\nThe Vert.x kafka library allows asynchronous publishing and receiving of messages on Kafka topic through the vert.x event bus.\n\n####To use this library you must have kafka and zookeeper up and running. Follow instructions at [Kafka quick start guide](http://kafka.apache.org/documentation.html#quickstart)\n\nThis is a multi-threaded worker library that consumes kafka messages and then re-broadcast them on an address on the vert.x event bus.\n\n## Getting Started\n\nAdd a dependency to vertx-kafka:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.cyngn.vertx\u003c/groupId\u003e\n    \u003cartifactId\u003evertx-kafka\u003c/artifactId\u003e\n    \u003cversion\u003e3.3.0-SNAPSHOT\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n| vertx-kafka | vert.x | kafka |\n| ------- | --------------:| --------: |\n| 3.3.0-SNAPSHOT | 3.3.0-SNAPSHOT | 0.9.0 |\n| 0.4.1   | 3.1.0 | 0.9.0 |\n\n## Consumer\n\nListening for messages coming from a kafka broker.\n\n### Configuration\n\n```json\n{\n    \"zookeeper.connect\" : \"\u003chost1:2181,host:2181...\u003e\",\n    \"group.id\" : \"\u003ckafkaConsumerGroupId\u003e\",\n    \"bootstrap.servers\" \"\u003chost1:9092,host2:9092...\u003e\",\n    \"backoff.increment.ms\" : \"\u003cbackTimeInMilli\u003e\",\n    \"autooffset.reset\" : \"\u003ckafkaAutoOffset\u003e\",\n    \"topics\" : [\"\u003ctopic1\u003e\", \"\u003ctopic2\u003e\"],\n    \"eventbus.address\" : \"\u003cdefault kafka.message.consumer\u003e\",\n    \"consumer.poll.interval.ms\" : \u003cdefault 100 ms\u003e\n}\n```\n\nFor example:\n\n```json\n{\n    \"zookeeper.host\" : \"localhost:2181\",\n    \"group.id\" : \"testGroup\",\n    \"bootstrap.servers\" \"localhost:9092\",\n    \"backoff.increment.ms\" : \"100\",\n    \"autooffset.reset\" : \"smallest\",\n    \"topics\" : [\"testTopic\"],\n    \"eventbus.address\" : \"kafka.to.vertx.bridge\",\n    \"consumer.poll.interval.ms\" : 1000\n}\n```\nField breakdown:\n\n* `zookeeper.connect` a zookeeper connection string of form hostname1:port1,hostname2:port2,hostname3:port3/chroot/path used with your kafka clusters\n* `group.id` the kafka consumer group name that will be consuming related to\n* `bootstrap.servers` the list of initial kafka hosts to connect to\n* `backoff.increment.ms` backoff interval for contacting broker without messages in milliseconds\n* `autooffset.reset` how to reset the offset\n* `topics` the kafka topics to listen for\n* `eventbus.address` the vert.x address to publish messages onto when received form kafka\n* `consumer.poll.interval.ms` how often to try and consume messages\n\nFor a deeper look at kafka configuration parameters check [this](http://kafka.apache.org/documentation.html) page out.\n\n### Usage\n\nYou should only need one consumer per application.\n\n#### Deploy the verticle in your server\n\n```java\n\nvertx = Vertx.vertx();\n\n// sample config\nJsonObject consumerConfig = new JsonObject();\nconsumerConfig.put(ConfigConstants.GROUP_ID, \"testGroup\");\nList\u003cString\u003e topics = new ArrayList\u003c\u003e();\ntopics.add(\"testTopic\");\nconsumerConfig.put(\"topics\", new JsonArray(topics));\n\ndeployKafka(config);\n\npublic void deployKafka(JsonObject config) {\n   // use your vert.x reference to deploy the consumer verticle\n\t vertx.deployVerticle(MessageConsumer.class.getName(),\n      new DeploymentOptions().setConfig(config),\n\t\t\tdeploy -\u003e {\n   \t\t     if(deploy.failed()) {\n        \t     System.err.println(String.format(\"Failed to start kafka consumer verticle, ex: %s\", deploy.cause()));\n               vertx.close()\n               return;\n            }\n            System.out.println(\"kafka consumer verticle started\");\n      }\n\t);\n}\n```\n#### Listen for messages\n\n```java\n\nvertx.eventBus().consumer(MessageConsumer.EVENTBUS_DEFAULT_ADDRESS,\n\tmessage -\u003e {\n\t\t   System.out.println(String.format(\"got message: %s\", message.body()))\n\t\t   // message handling code\n\t\t   KafkaEvent event = new KafkaEvent(message.body());\n\t });\n```\n\n#### Consumer Errors\n\nYou can listen on the address `kafka.producer.error` for errors from the kafka producer.\n\n## Producer\n\nSend a message to a kafka cluster on a predefined topic.\n\n### Configuration\n\n```json\n{\n    \"serializer.class\":\"\u003cthe default encoder\u003e\",\n    \"key.serializer\":\"\u003cthe key encoder\u003e\",\n    \"value.serializer\":\"\u003cthe value encoder\u003e\",\n    \"bootstrap.servers\":\"\u003chost1:9092,host2:9092\u003e,\"\n    \"default_topic\":\"\u003cdefault kafka topic to send to\u003e,\"\n    \"eventbus.address\":\"\u003cthe event bus topic where you send messages to send to kafka\u003e\"\n    \"max.block.ms\" : \u003cdefaults to 60000\u003e\n}\n```\n\nFor example:\n\n```json\n{\n    \"serializer.class\":\"org.apache.kafka.common.serialization.StringSerializer\",\n    \"bootstrap.servers\":\"localhost:9092\",\n    \"default_topic\":\"testTopic\"\n}\n```\n\n* `serializer.class` The serializer class for messages\n* `key.serializer` The serializer class for keys, defaults to the serializel.class if not set\n* `value.serializer` The serializer class for values, defaults to the serializel.class if not set\n* `bootstrap.servers` The socket connections for sending the actual data will be established based on the broker information returned in the metadata. The format is host1:port1,host2:port2, and the list can be a subset of brokers or a VIP pointing to a subset of brokers.\n* `default_topic` The default topic in kafka to send to\n* `eventbus.address` The address to listen to on the event bus, defaults to 'kafka.message.publisher'\n* `max.block.ms` How long should the sender wait before getting meta data or time out in ms.\n\nFor a deeper look at kafka configuration parameters check [this](https://kafka.apache.org/08/configuration.html) page out.\n\n### Usage\n\nYou should only need one producer per application.\n\n#### Deploy the verticle in your server\n\n```java\n\nvertx = Vertx.vertx();\n\n// sample config\nJsonObject producerConfig = new JsonObject();\nproducerConfig.put(\"bootstrap.servers\", \"localhost:9092\");\nproducerConfig.put(\"serializer.class\", \"org.apache.kafka.common.serialization.StringSerializer\");\nproducerConfig.put(\"default_topic\", \"testTopic\");\n\ndeployKafka(producerConfig);\n\npublic void deployKafka(JsonObject config) {\n   // use your vert.x reference to deploy the consumer verticle\n\t vertx.deployVerticle(MessageProducer.class.getName(),\n     new DeploymentOptions().setConfig(config),\n\t\t deploy -\u003e {\n   \t   if(deploy.failed()) {\n         System.err.println(String.format(\"Failed to start kafka producer verticle, ex: %s\", deploy.cause()));\n          vertx.close()\n          return;\n       }\n       System.out.println(\"kafka producer verticle started\");\n   });\n}\n```\n#### Send message to kafka topic\n\n```java\n\nKafkaPublisher publisher = new KafkaPublisher(vertx.eventBus());\n\n// send to the default topic\npublisher.send(\"a test message on a default topic\");\n// send to a specific topic\npublisher.send(\"SomeSpecialTopic\", \"a test message on a default topic\");\n// send to a specific topic with custom key\npublisher.send(\"SomeSpecialTopic\", \"aUserId\", \"a test message on a default topic\");\n// send to a specific topic and partition\npublisher.send(\"SomeSpecialTopic\", \"\", 5, \"a test message on a default topic\");\n\n```\n#### Producer Errors\n\nYou can listen on the address `kafka.producer.error` for errors from the kafka producer.\n\n\n### Test setup\n\n* cd [yourKafkaInstallDir]\n* bin/zookeeper-server-start.sh config/zookeeper.properties\n* bin/kafka-server-start.sh config/server.properties\n* bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 8 --topic [yourTestTopic]\n* bin/kafka-console-producer.sh --broker-list localhost:9092 --topic [yourTestTopic]\n* bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic [yourTestTopic]\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyngn%2Fvertx-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcyngn%2Fvertx-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcyngn%2Fvertx-kafka/lists"}