{"id":22833044,"url":"https://github.com/kroxylicious/kroxylicious-junit5-extension","last_synced_at":"2026-02-18T12:07:31.300Z","repository":{"id":65309321,"uuid":"572536107","full_name":"kroxylicious/kroxylicious-junit5-extension","owner":"kroxylicious","description":"JUnit5 extension and helpers for writing tests parameterised over Kafka clusters","archived":false,"fork":false,"pushed_at":"2025-04-16T21:59:45.000Z","size":1326,"stargazers_count":9,"open_issues_count":24,"forks_count":10,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-04-23T22:04:13.339Z","etag":null,"topics":["apache-kafka","java","junit5"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kroxylicious.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-11-30T13:40:44.000Z","updated_at":"2025-04-16T21:58:49.000Z","dependencies_parsed_at":"2024-01-08T09:28:28.597Z","dependency_job_id":"56b4f24d-1592-4981-a727-96df181b6b6b","html_url":"https://github.com/kroxylicious/kroxylicious-junit5-extension","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kroxylicious%2Fkroxylicious-junit5-extension","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kroxylicious%2Fkroxylicious-junit5-extension/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kroxylicious%2Fkroxylicious-junit5-extension/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kroxylicious%2Fkroxylicious-junit5-extension/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kroxylicious","download_url":"https://codeload.github.com/kroxylicious/kroxylicious-junit5-extension/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250522300,"owners_count":21444511,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-kafka","java","junit5"],"created_at":"2024-12-12T21:11:43.206Z","updated_at":"2026-01-05T19:16:57.243Z","avatar_url":"https://github.com/kroxylicious.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kafka Cluster JUnit 5 extension\n\n\u003c!-- TOC --\u003e\n* [Kafka Cluster JUnit 5 extension](#kafka-cluster-junit-5-extension)\n  * [What?](#what)\n  * [Dependency](#dependency)\n  * [Example](#example)\n  * [Adding Constraints](#adding-constraints)\n  * [Configuring Kafka Brokers](#configuring-kafka-brokers)\n  * [Configuring SASL](#configuring-sasl)\n  * [Configuring Kafka Clients](#configuring-kafka-clients)\n  * [Creating Test Topics](#creating-test-topics)\n  * [Node topology](#node-topology)\n  * [Provisioning mechanisms](#provisioning-mechanisms)\n    * [Choosing the Provisioning mechanism](#choosing-the-provisioning-mechanism)\n  * [Field injection and Parameter Resolution](#field-injection-and-parameter-resolution)\n  * [Template tests](#template-tests)\n  * [Custom cluster provisioning and constraints](#custom-cluster-provisioning-and-constraints)\n  * [Developer Guide](#developer-guide)\n  * [Releasing this project](#releasing-this-project)\n  * [Contributing](#contributing)\n\u003c!-- TOC --\u003e\n\n## What?\n\nThis is a JUnit 5 extension that allows writing tests that require a Kafka cluster\nwithout having to hard-code exactly _how_ that cluster is provisioned.\nAs such it allows you to write very flexible tests that abstract over different kinds of cluster.\n\n## Dependency\nCurrent version: ![Maven Central Version](https://img.shields.io/maven-central/v/io.kroxylicious.testing/testing-junit5-extension)\n\nGradle \n```\ntestImplementation 'io.kroxylicious.testing:testing-junit5-extension:0.1'\n```\n\nMaven\n\n```xml\n\u003cdependencies\u003e\n    \u003cdependency\u003e\n        \u003cgroupId\u003eio.kroxylicious.testing\u003c/groupId\u003e\n        \u003cartifactId\u003etesting-junit5-extension\u003c/artifactId\u003e\n        \u003cversion\u003e0.1\u003c/version\u003e\n    \u003c/dependency\u003e\n    \u003c!-- Kafka Broker version you want to use --\u003e\n    \u003cdependency\u003e\n        \u003cgroupId\u003eorg.apache.kafka\u003c/groupId\u003e\n        \u003cartifactId\u003ekafka_2.13\u003c/artifactId\u003e\n        \u003cversion\u003e${kafka.version}\u003c/version\u003e \u003c!-- versions from 3.3.0 have been tested  --\u003e\n        \u003cscope\u003etest\u003c/scope\u003e\n    \u003c/dependency\u003e\n    \u003c!-- Optional, required if you want to use in-VM kafka in Zookeeper mode. --\u003e\n    \u003cdependency\u003e\n        \u003cgroupId\u003eorg.apache.zookeeper\u003c/groupId\u003e\n        \u003cartifactId\u003ezookeeper\u003c/artifactId\u003e\n        \u003cversion\u003e${zookeeper.version}\u003c/version\u003e \u003c!-- versions from 3.6.3 have been tested --\u003e\n        \u003cscope\u003etest\u003c/scope\u003e\n    \u003c/dependency\u003e\n\u003c/dependencies\u003e\n```\n\n## Example\n\nHere's a minimal example:\n\n```java\nimport io.kroxylicious.testing.kafka.api.KafkaCluster;\nimport io.kroxylicious.testing.kafka.junit5ext.KafkaClusterExtension;\n\nimport org.apache.kafka.clients.producer.Producer;\nimport org.apache.kafka.clients.producer.ProducerRecord;\n\nimport org.junit.jupiter.api.Test;\nimport org.junit.jupiter.api.extension.ExtendWith;\n\n@ExtendWith(KafkaClusterExtension.class) // \u003c1\u003e\nclass MyTest {\n\n    KafkaCluster cluster; // \u003c2\u003e\n\n    @Test\n    public void testProducer(\n            Producer\u003cString, String\u003e producer // \u003c3\u003e\n    ) throws Exception {\n        producer.send(new ProducerRecord\u003c\u003e(\"hello\", \"world\")).get();\n    }\n}\n```\n\nNotes:\n\n1. You have to tell Junit that you're using the extension using `@ExtendWith`.\n2. An instance field of type `KafkaCluster` will cause a new cluster to be provisioned for each test in the class. Alternatively you could use a parameter on a `@Test`-annotated method. If you use a `static` field then a single cluster will be provisioned for _all_ the tests in the class.\n3. Your test methods can declare `Producer`, `Consumer` and `Admin`-typed parameters. They will be configured to bootstrap against the `cluster`.\n\nBy default, this would use a single-broker KRaft-based Kafka cluster running in the same VM as your test.\n\n## Adding Constraints\n\nSuch a cluster might not be what you need for the scenario you're trying to test.\nYou can configure different clusters by annotating the `KafkaCluster` field or parameter:\n\n* `@BrokerCluster(numBrokers=3)` will use a Kafka cluster with the given number of brokers\n* `@KRaftCluster` will ensure a KRaft-based cluster is used. `@KRaftCluster(numControllers=3)` will use a controller quorum with 3 controllers.\n* `@ZooKeeperCluster` will ensure a ZooKeeper-based Kafka cluster (unsurprisingly this is mutually exclusive with `@KRaftCluster`)\n* `@SaslMechanism` will provide cluster with the external listener configured for the given SASL\n   mechanism.  Use of this option requires the client to use SASL authentication. For PLAIN and SCRAM mechanism a\n   database of principals must be provided.\n* `@Version(value=\"3.3.1\")` will provide a container-based cluster with the kafka/zookeeper version indicated\n\nWhen multiple constraints are provided they will _all_ be satisfied.\n\nThe cluster will be provisioned using the fastest available mechanism, because your development inner loop is a precious thing.\n\n## Configuring Kafka Brokers\n\nTo configure a kafka broker, apply the `@BrokerConfig` annotation to the `KafkaCluster` type.\n\nFor example:\n\n```java\n@ExtendWith(KafkaClusterExtension.class)\nclass MyTest {\n    @BrokerConfig(name = \"compression.type\", value = \"zstd\") KafkaCluster cluster;\n\n    // ...\n}\n```\n\n## Configuring SASL\n\nTo config the Broker to use SASL, use the annotations `@SaslMechanism` to specify the SASL\nmechanism and, if required, a database of principals.\n\n```java\nclass MyTest {\n@SaslMechanism(value = \"PLAIN\", principals = { @Principal(user = \"alice\", password = \"foo\") }) KafkaCluster cluster;\n\n    // ...\n}\n```\n\n## Configuring Kafka Clients\n\nTo configure kafka clients, apply the `@ClientConfig` annotation.  It can be applied to the following types:\n\n* `Producer`\n* `Consumer`\n* `Admin`\n\nFor example:\n\n```java\n@ExtendWith(KafkaClusterExtension.class)\nclass MyTest {\n    @ClientConfig(name = \"client.id\", value = \"myclient\") Producer\u003cString, String\u003e producer;\n\n    // ...\n}\n```\n\n## Creating Test Topics\n\nThe test framework can create topic(s) for the test.  It is\nguaranteed that the test will exist before the test method is invoked.\n\nYou can also specify configuration for the topic using the annotation `@TopicConfig`,\nor set its partition count or replication factor with `@TopicPartitions` and `@TopicReplicationFactor`\nrespectively.\n\n```java\n@ExtendWith(KafkaClusterExtension.class)\nclass MyTest {\n    Topic myTopic;\n    @TopicConfig(name = \"cleanup.policy\", value = \"compact\") Topic myTopicWithConfig;\n    // ...\n}\n```\n\n\n## Node topology\n\nWhen generating a cluster using KRaft (the default), you declare how many brokers and controllers you want and the extension will provision the minimum number of nodes to satisfy those conditions. It will create as many nodes as it can that are both KRaft controllers and brokers using [process.roles](https://kafka.apache.org/documentation/#brokerconfigs_process.roles) (process.roles = \"broker,controller\"). For example:\n\n\n| numBrokers | numControllers | roles                                                               |\n|------------|----------------|---------------------------------------------------------------------|\n| 1          | 1              | `\"broker,controller\"`                                               |\n| 3          | 1              | `\"broker,controller\"`, `\"broker\"`, `\"broker\"`                       |\n| 1          | 3              | `\"broker,controller\"`, `\"controller\"`, `\"controller\"`               |\n| 3          | 3              | `\"broker,controller\"`, `\"broker,controller\"`, `\"broker,controller\"` |\n\n## Provisioning mechanisms\n\nThe following provisioning mechanisms are currently supported:\n\n* an in-VM Kafka cluster (by creating a `KafkaServer` within the same JVM as your tests)\n* a container-based cluster using [testcontainers](https://www.testcontainers.org/).\n\nWhich kind of cluster is chosen depends on the requirements of your test.\nFor example, using containers allows to easily test against different broker versions. \n\n---\n**NOTE**\n\nIn case you use podman for testcontainers, some tips must be taken into account:\n* `TESTCONTAINERS_RYUK_DISABLED=true` env variable shall be declared as per https://github.com/containers/podman/issues/7927#issuecomment-731525556\n* `podman-plugins` linux package shall be installed on your machine in order to communicate containers among each other.\n\n---\n\n### Choosing the Provisioning mechanism\n\nBy default, `KafkaCluster` prefers to run in the same JVM as your tests. You can override this globally or for specific \ntests.\n\n* **Global Container Mode**: To prefer running the cluster in a container, set this environment variable:\n    ```bash\n    export TEST_CLUSTER_EXECUTION_MODE=CONTAINER\n    ```\n\n* **Per-Test Override**: For a specific test, directly inject the cluster implementation you need:\n    * `InVMKafkaCluster` (in-JVM)\n    * `TestcontainersKafkaCluster` (container)\n\n\n## Field injection and Parameter Resolution\n\nThe extension supports injecting clusters and clients:\n- into fields of the test class \n- as parameters to `@BeforeAll`\n- as parameters to `@BeforeEach`\n- as parameters to test methods\n\nTo avoid collisions with other extensions, such as Mockito, we will only inject into fields which:\n- have no annotations  \nOR are annotated with annotations from the following packages\n- `io.kroxylicious`\n- `org.junit`\n- `java.lang`\n\nThe extension will not try to overwrite fields which have already been initialised. \n\n## Template tests\n\nYou can also use test templates to execute the same test over a number of different cluster configurations. Here's an example:\n\n```java\nimport java.util.stream.Stream;\n\nimport io.kroxylicious.testing.kafka.common.ConstraintUtils;\nimport io.kroxylicious.testing.kafka.api.KafkaCluster;\nimport io.kroxylicious.testing.kafka.common.BrokerCluster;\nimport io.kroxylicious.testing.kafka.junit5ext.DimensionMethodSource;\n\nimport org.junit.jupiter.api.TestTemplate;\nimport org.junit.jupiter.api.extension.ExtendWith;\n\nimport static io.kroxylicious.junit5.constraint.ConstraintUtils;\nimport static org.junit.jupiter.api.Assertions.assertEquals;\n\n@ExtendWith(KafkaClusterExtension.class) // 1\npublic class TemplateTest {\n    @TestTemplate // \u003c2\u003e\n    public void multipleClusterSizes(\n            @DimensionMethodSource(\"clusterSizes\") // \u003c3\u003e\n            KafkaCluster cluster) throws Exception {\n        // ... your test code\n    }\n\n    static Stream\u003cBrokerCluster\u003e clusterSizes() {// 4\n        return Stream.of(\n                ConstraintUtils.brokerCluster(1),\n                ConstraintUtils.brokerCluster(3));\n    }\n}\n```\n\n1. You have to tell Junit that you're using the extension using `@ExtendWith`.\n2. You use [JUnit Jupiter's `@TestTemplate`](https://junit.org/junit5/docs/current/user-guide/#writing-tests-test-templates) rather than a `@Test` to tell JUnit that this method is a _factory_ for tests.\n3. You provide a reference to source of additional constraints. Here the source is a method named via the `@DimensionMethodSource` annotation.\n4. The method must be `static`, take no parameters and return a `Stream`, `Collection` or array of constraint annotations. \n\nA test will be executed as many times as there are constraints in the stream with a different Kafka cluster each time. These constraints are _in addition to_ any constraint annotations applied to the `KafkaCluster cluster` parameter of the `multipleClusterSizes()` method. (There are none in the example above, but we could have applied any from the above list).\n\nYou can use multiple instances of `@DimensionMethodSource`. In that case every combination of constraints will be tested (i.e. the full cartesian product over all the constraint sources). For example, we could write:\n\n```java\nimport java.util.stream.Stream;\n\nimport io.kroxylicious.testing.kafka.common.ConstraintUtils;\nimport io.kroxylicious.testing.kafka.api.KafkaCluster;\nimport io.kroxylicious.testing.kafka.junit5ext.DimensionMethodSource;\n\nimport org.junit.jupiter.api.TestTemplate;\nimport org.junit.jupiter.api.extension.ExtendWith;\n\nimport org.apache.kafka.clients.admin.Admin;\n\nimport static io.kroxylicious.junit5.constraint.ConstraintUtils;\nimport static org.junit.jupiter.api.Assertions.assertEquals;\n\n@ExtendWith(KafkaClusterExtension.class)\npublic class TemplateTest {\n    @TestTemplate\n    public void multipleClusterSizes(\n            @DimensionMethodSource(\"clusterSizes\")\n            @DimensionMethodSource(\"compression\")\n            KafkaCluster cluster,\n            Admin admin) throws Exception {\n        // ...\n    }\n\n    static Stream\u003cBrokerConfig\u003e compression() {\n        return Stream.of(\n                ConstraintUtils.brokerConfig(\"compression.type\", \"zstd\"),\n                ConstraintUtils.brokerConfig(\"compression.type\", \"snappy\"));\n    }\n}\n```\n  And the template would produce the following test combinations:\n\n* 1 broker with `compression.type=zstd`\n* 3 brokers with `compression.type=zstd`\n* 1 broker with `compression.type=snappy`\n* 3 brokers with `compression.type=snappy`\n\nAlternatively if you don't need to test _every_ combination, you could use a `@ConstraintsMethodSource` source that returns just those combinations that you _do_ want to test. In this case the return type of the source method must be `Stream\u003cList\u003cAnnotation\u003e\u003e`, `Collection\u003cList\u003cAnnotation\u003e\u003e` or `List\u003cAnnotation\u003e[]`.\n\n```java\nimport java.util.stream.Stream;\n\nimport org.junit.jupiter.api.TestTemplate;\nimport org.junit.jupiter.api.extension.ExtendWith;\n\nimport org.apache.kafka.clients.admin.Admin;\n\nimport io.kroxylicious.testing.kafka.api.KafkaCluster;\nimport io.kroxylicious.testing.kafka.junit5ext.ConstraintsMethodSource;\n\nimport static io.kroxylicious.junit5.constraint.ConstraintUtils;\nimport static org.junit.jupiter.api.Assertions.assertEquals;\n\n@ExtendWith(KafkaClusterExtension.class)\npublic class TemplateTest {\n    @TestTemplate\n    public void testTuples(@ConstraintsMethodSource(\"tuples\")\n                           KafkaCluster cluster,\n                           Admin admin)\n            throws Exception {\n        // ...\n    }\n\n    static Stream\u003cList\u003cAnnotation\u003e\u003e tuples() {\n        return Stream.of(\n                List.of(brokerCluster(1), kraftCluster(1)),\n                List.of(brokerCluster(3), kraftCluster(1)),\n                List.of(brokerCluster(3), zooKeeperCluster()));\n    }\n}\n```\n\nThis will produce tests as follows:\n\n* 1 broker, KRaft-based\n* 3 brokers, KRaft-based\n* 3 brokers, ZK-based\n\n## Custom cluster provisioning and constraints\n\nProvisioning mechanisms can be provided externally by implementing `io.kroxylicious.testing.kafka.api.KafkaClusterProvisioningStrategy` and declaring it as a [Java service](https://www.baeldung.com/java-spi) (e.g. in a  `META-INF/services/io.kroxylicious.testing.kafka.api.KafkaClusterProvisioningStrategy` file that is available on the test-time classpath).\n\nYou can also provide your own constraint annotation types by annotating them with the `@io.kroxylicious.testing.kafka.api.KafkaClusterConstraint` meta-annotation. Such custom constraint annotations will only be understood by your custom provisioning strategy, so the in-JVM and testcontainers-based clusters provided by this project then can't be used.\n\n## Developer Guide\n\nSee the [developer guide](DEV_GUIDE.md).\n\n## Releasing this project\n\nSee the [releasing guide](RELEASING.md).\n\n## Contributing\n\nWe welcome contributions! Please see our [contributing guidelines](https://github.com/kroxylicious/.github/blob/main/CONTRIBUTING.md) to get started.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkroxylicious%2Fkroxylicious-junit5-extension","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkroxylicious%2Fkroxylicious-junit5-extension","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkroxylicious%2Fkroxylicious-junit5-extension/lists"}