{"id":18250516,"url":"https://github.com/dropwizard/dropwizard-kafka","last_synced_at":"2025-04-04T16:31:26.350Z","repository":{"id":35844203,"uuid":"185870861","full_name":"dropwizard/dropwizard-kafka","owner":"dropwizard","description":"A convenience library for Apache Kafka integration in a Dropwizard service.","archived":false,"fork":false,"pushed_at":"2024-10-29T17:27:40.000Z","size":480,"stargazers_count":21,"open_issues_count":9,"forks_count":12,"subscribers_count":11,"default_branch":"release/4.0.x","last_synced_at":"2024-10-29T18:50:35.984Z","etag":null,"topics":["dropwizard-kafka","kafka","kafka-client","kafka-consumers"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dropwizard.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-09T20:59:02.000Z","updated_at":"2024-09-17T03:48:21.000Z","dependencies_parsed_at":"2023-12-20T08:52:17.160Z","dependency_job_id":"404e5192-0006-47fb-9c50-cf316e394f43","html_url":"https://github.com/dropwizard/dropwizard-kafka","commit_stats":null,"previous_names":[],"tags_count":26,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dropwizard%2Fdropwizard-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dropwizard%2Fdropwizard-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dropwizard%2Fdropwizard-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dropwizard%2Fdropwizard-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dropwizard","download_url":"https://codeload.github.com/dropwizard/dropwizard-kafka/tar.gz/refs/heads/release/4.0.x","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247209371,"owners_count":20901764,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dropwizard-kafka","kafka","kafka-client","kafka-consumers"],"created_at":"2024-11-05T09:44:57.580Z","updated_at":"2025-04-04T16:31:26.006Z","avatar_url":"https://github.com/dropwizard.png","language":"Java","readme":"# dropwizard-kafka\n[![Build](https://github.com/dropwizard/dropwizard-kafka/workflows/Build/badge.svg)](https://github.com/dropwizard/dropwizard-kafka/actions?query=workflow%3ABuild)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=dropwizard_dropwizard-kafka\u0026metric=alert_status)](https://sonarcloud.io/dashboard?id=dropwizard_dropwizard-kafka)\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/io.dropwizard.modules/dropwizard-kafka/badge.svg)](https://maven-badges.herokuapp.com/maven-central/io.dropwizard.modules/dropwizard-kafka/)\n\nProvides easy integration for Dropwizard applications with the Apache Kafka client.\n\nThis bundle comes with out-of-the-box support for:\n* YAML Configuration integration\n* Producer and Consumer lifecycle management\n* Producer and Cluster connection health checks\n* Metrics integration for the Kafka client\n* An easier way to create/configure Kafka consumers/producers than is offered by the base Kafka client\n* Distributed tracing integration, using the [Brave Kafka client instrumentation library](https://github.com/openzipkin/brave/tree/master/instrumentation/kafka-clients).\n\nFor more information on Kafka, take a look at the official documentation here: http://kafka.apache.org/documentation/\n\n## Dropwizard Version Support Matrix\n| dropwizard-kafka | Dropwizard v1.3.x  | Dropwizard v2.0.x  | Dropwizard v2.1.x  | Dropwizard v3.0.x  | Dropwizard v4.0.x  |\n|------------------|--------------------|--------------------|--------------------|--------------------|--------------------|\n| v1.3.x           | :white_check_mark: | :white_check_mark: | :white_check_mark: | :x:                | :x:                |\n| v1.4.x           | :white_check_mark: | :white_check_mark: | :white_check_mark: | :x:                | :x:                |\n| v1.5.x           | :white_check_mark: | :white_check_mark: | :white_check_mark: | :x:                | :x:                |\n| v1.6.x           | :x:                | :white_check_mark: | :white_check_mark: | :x:                | :x:                |\n| v1.7.x           | :x:                | :white_check_mark: | :white_check_mark: | :x:                | :x:                |\n| v1.8.x           | :x:                | :x:                | :white_check_mark: | :x:                | :x:                |\n| v3.0.x           | :x:                | :x:                | :x:                | :white_check_mark: | :x:                |\n| v4.0.x           | :x:                | :x:                | :x:                | :x:                | :white_check_mark: |\n\n## Usage\nAdd dependency on library.\n\nMaven:\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003eio.dropwizard.modules\u003c/groupId\u003e\n  \u003cartifactId\u003edropwizard-kafka\u003c/artifactId\u003e\n  \u003cversion\u003e1.7.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nGradle:\n```groovy\ncompile \"io.dropwizard.modules:dropwizard-kafka:$dropwizardVersion\"\n```\n\n### Basic Kafka Producer\nIn your Dropwizard `Configuration` class, configure a `KafkaProducerFactory`:\n```java\n@Valid\n@NotNull\n@JsonProperty(\"producer\")\nprivate KafkaProducerFactory\u003cString, String\u003e kafkaProducerFactory;\n```\n\nThen, in your `Application` class, you'll want to do something similar to the following:\n```java\nprivate final KafkaProducerBundle\u003cString, String, ExampleConfiguration\u003e kafkaProducer = new KafkaProducerBundle\u003cString, String, ExampleConfiguration\u003e() {\n    @Override\n    public KafkaProducerFactory\u003cString, String\u003e getKafkaProducerFactory(ExampleConfiguration configuration) {\n        return configuration.getKafkaProducerFactory();\n    }\n};\n\n@Override\npublic void initialize(Bootstrap\u003cExampleConfiguration\u003e bootstrap) {\n    bootstrap.addBundle(kafkaProducer);\n}\n\n@Override\npublic void run(ExampleConfiguration config, Environment environment) {\n    final PersonEventProducer personEventProducer = new PersonEventProducer(kafkaProducer.getProducer());\n    environment.jersey().register(new PersonEventResource(personEventProducer));\n}\n```\n\nConfigure your factory in your `config.yml` file:\n\n```yaml\nproducer:\n  type: basic\n  bootstrapServers:\n    - 127.0.0.1:9092\n    - 127.0.0.1:9093\n    - 127.0.0.1:9094\n  name: producerNameToBeUsedInMetrics\n  keySerializer:\n    type: string\n  valueSerializer:\n    type: string\n  acks: all\n  retries: 2147483647 # int max value\n  maxInFlightRequestsPerConnection: 1\n  maxPollBlockTime: 10s\n  security:\n    securityProtocol: sasl_ssl\n    sslProtocol: TLSv1.2\n    saslMechanism: PLAIN\n    saslJaas: \"org.apache.kafka.common.security.plain.PlainLoginModule required username=\\\"\u003cusername\u003e\\\" password=\\\"\u003cpassword\u003e\\\";\"\n```\n\n### Basic Kafka Consumer\nIn your Dropwizard `Configuration` class, configure a `KafkaConsumerFactory`:\n\n```java\n@Valid\n@NotNull\n@JsonProperty(\"consumer\")\nprivate KafkaConsumerFactory\u003cString, String\u003e kafkaConsumerFactory;\n```\n\nThen, in your `Application` class, you'll want to do something similar to the following:\n```java\nprivate final KafkaConsumerBundle\u003cString, String, ExampleConfiguration\u003e kafkaConsumer = new KafkaConsumerBundle\u003cString, String, ExampleConfiguration\u003e() {\n    @Override\n    public KafkaConsumerFactory\u003cString, String\u003e getKafkaConsumerFactory(ExampleConfiguration configuration) {\n        return configuration.getKafkaConsumerFactory();\n    }\n};\n\n@Override\npublic void initialize(Bootstrap\u003cExampleConfiguration\u003e bootstrap) {\n    bootstrap.addBundle(kafkaConsumer);\n}\n\n@Override\npublic void run(ExampleConfiguration config, Environment environment) {\n    final PersonEventConsumer personEventConsumer = new PersonEventConsumer(kafkaConsumer.getConsumer());\n    personEventConsumer.startConsuming();\n}\n```\n\nConfigure your factory in your `config.yml` file:\n\n```yaml\nconsumer:\n  type: basic\n  bootstrapServers:\n    - 127.0.0.1:9092\n    - 127.0.0.1:9093\n    - 127.0.0.1:9094\n  consumerGroupId: consumer1\n  name: consumerNameToBeUsedInMetrics\n  keyDeserializer:\n    type: string\n  valueDeserializer:\n    type: string\n  security:\n    securityProtocol: sasl_ssl\n    sslProtocol: TLSv1.2\n    saslMechanism: PLAIN\n    saslJaas: \"org.apache.kafka.common.security.plain.PlainLoginModule required username=\\\"\u003cusername\u003e\\\" password=\\\"\u003cpassword\u003e\\\";\"\n```\n\n## Using an older version of the Kafka Client\nThis library *should* remain backwards compatible, such that you can override the version of the kafka client that this library includes. If this becomes no longer possible, we will need to create separate branches for differing major versions of the Kafka client.\n\nFor example, say you would like to use version `1.1.1` of the Kafka client. One option would be to explicitly define a dependency on version `1.1.1` of `kafka-clients` before you declare a dependency on `dropwizard-kafka`.\n\n```xml\n\u003cdependencies\u003e\n  \u003cdependency\u003e\n    \u003cgroupId\u003eorg.apache.kafka\u003c/groupId\u003e\n    \u003cartifactId\u003ekafka-clients\u003c/artifactId\u003e\n    \u003cversion\u003e1.1.1\u003c/version\u003e\n  \u003c/dependency\u003e\n  \u003cdependency\u003e\n    \u003cgroupId\u003eio.dropwizard\u003c/groupId\u003e\n    \u003cartifactId\u003edropwizard-kafka\u003c/artifactId\u003e\n    \u003cversion\u003e${dropwizard.version}\u003c/version\u003e\n  \u003c/dependency\u003e\n\u003c/dependencies\u003e\n```\n\n## Adding support for additional serializers and/or deserializers\nIn order to support additional serializers or deserializers, you'll need to create a new factory:\n```java\n@JsonTypeName(\"my-serializer\")\npublic class MySerializerFactory extends SerializerFactory {\n\n    @NotNull\n    @JsonProperty\n    private String someConfig;\n\n    public String getSomeConfig() {\n        return someConfig;\n    }\n\n    public void setSomeConfig(final String someConfig) {\n        this.someConfig = someConfig;\n    }\n\n\n    @Override\n    public Class\u003c? extends Serializer\u003e getSerializerClass() {\n        return MySerializer.class;\n    }\n}\n```\n\nThen you will need to add the following files to your `src/main/resources/META-INF/services` directory in order to support Jackson\npolymorphic serialization:\n\nFile named `io.dropwizard.jackson.Discoverable`:\n\n```\nio.dropwizard.kafka.serializer.SerializerFactory\n```\n\nFile named `io.dropwizard.kafka.serializer.SerializerFactory`:\n\n```\npackage.name.for.your.MySerializerFactory\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdropwizard%2Fdropwizard-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdropwizard%2Fdropwizard-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdropwizard%2Fdropwizard-kafka/lists"}