{"id":25050598,"url":"https://github.com/steadybit/extension-kafka","last_synced_at":"2026-04-02T13:24:45.384Z","repository":{"id":261766452,"uuid":"859277435","full_name":"steadybit/extension-kafka","owner":"steadybit","description":null,"archived":false,"fork":false,"pushed_at":"2026-02-19T16:49:30.000Z","size":743,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-02-19T20:12:21.855Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/steadybit.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-09-18T11:51:03.000Z","updated_at":"2026-02-19T16:49:35.000Z","dependencies_parsed_at":"2024-11-08T10:38:45.087Z","dependency_job_id":"b4c98fce-e551-47a6-b2f9-ddb7ea0a7384","html_url":"https://github.com/steadybit/extension-kafka","commit_stats":null,"previous_names":["steadybit/extension-kafka"],"tags_count":52,"template":false,"template_full_name":"steadybit/extension-scaffold","purl":"pkg:github/steadybit/extension-kafka","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steadybit%2Fextension-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steadybit%2Fextension-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steadybit%2Fextension-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steadybit%2Fextension-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/steadybit","download_url":"https://codeload.github.com/steadybit/extension-kafka/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/steadybit%2Fextension-kafka/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29670063,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-21T00:11:43.526Z","status":"ssl_error","status_checked_at":"2026-02-20T23:52:33.807Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-06T09:16:57.401Z","updated_at":"2026-02-21T01:08:48.705Z","avatar_url":"https://github.com/steadybit.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Steadybit extension-kafka\n\nA [Steadybit](https://www.steadybit.com/) extension to integrate [Kafka](https://kafka.apache.org/) into Steadybit.\n\nLearn about the capabilities of this extension in\nour [Reliability Hub](https://hub.steadybit.com/extension/com.steadybit.extension_kafka).\n\n## Prerequisites\n\nThe extension-kafka is using these capacities, thus may need elevated rights on kafka side :\n\n- List brokers / topics / consumer groups / offsets\n- Elect leaders for partitions\n- Alter broker configuration\n- Create / Delete ACLs\n- Delete Records\n\n## Configuration\n\n### Single Cluster Configuration\n\nFor connecting to a single Kafka cluster, use the following configuration:\n\n| Environment Variable                                                | Helm value                               | Meaning                                                                                                                                 | Required | Default |\n|---------------------------------------------------------------------|------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------|----------|---------|\n| `STEADYBIT_EXTENSION_SEED_BROKERS`                                  | `kafka.seedBrokers`                      | Brokers hosts (without scheme) with port separated by comma (example: \"localhost:9092,localhost:9093\"                                   | yes      |         |\n| `STEADYBIT_EXTENSION_SASL_MECHANISM`                                | `kafka.auth.saslMechanism`               | PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512                                                                                                  | no       |         |\n| `STEADYBIT_EXTENSION_SASL_USER`                                     | `kafka.auth.saslUser`                    | Sasl User                                                                                                                               | no       |         |\n| `STEADYBIT_EXTENSION_SASL_PASSWORD`                                 | `kafka.auth.saslPassword`                | Sasl Password                                                                                                                           | no       |         |\n| `STEADYBIT_EXTENSION_KAFKA_CLUSTER_CERT_CHAIN_FILE`                 | `kafka.auth.kafkaClusterCertChainFile`   | The client certificate in PEM format.                                                                                                   | no       |         |\n| `STEADYBIT_EXTENSION_KAFKA_CLUSTER_CERT_KEY_FILE`                   | `kafka.auth.kafkaClusterCertKeyFile`     | The private key associated with the client certificate.                                                                                 | no       |         |\n| `STEADYBIT_EXTENSION_KAFKA_CLUSTER_CA_FILE`                         | `kafka.auth.kafkaClusterCaFile`          | The Certificate Authority (CA) certificate in PEM format.                                                                               | no       |         |\n| `STEADYBIT_EXTENSION_KAFKA_CONNECTION_USE_TLS`                      | `kafka.auth.useTLS`                      | Switch to \"true\" to use a TLS connection with default system certs, fill the certs fields above if you want to tune the tls connection. | no       |         |\n| `STEADYBIT_EXTENSION_DISCOVERY_ATTRIBUTES_EXCLUDES_BROKERS`         | `discovery.attributes.excludes.broker`   | List of Broker Attributes which will be excluded during discovery. Checked by key equality and supporting trailing \"*\"                  | no       |         |\n| `STEADYBIT_EXTENSION_DISCOVERY_ATTRIBUTES_EXCLUDES_TOPICS`          | `discovery.attributes.excludes.topic`    | List of Broker Attributes which will be excluded during discovery. Checked by key equality and supporting trailing \"*\"                  | no       |         |\n| `STEADYBIT_EXTENSION_DISCOVERY_ATTRIBUTES_EXCLUDES_CONSUMER_GROUPS` | `discovery.attributes.excludes.consumer` | List of Broker Attributes which will be excluded during discovery. Checked by key equality and supporting trailing \"*\"                  | no       |         |\n\n### Multi-Cluster Configuration\n\nThe extension supports connecting to multiple Kafka clusters simultaneously. When using multi-cluster configuration, all clusters are discovered in parallel and each target includes a `kafka.cluster.name` attribute to identify which cluster it belongs to.\n\n**Important:** When `kafka.clusters` is defined, the single-cluster configuration (`kafka.seedBrokers`, `kafka.auth.*`) is ignored.\n\n#### Helm Values\n\nConfigure multiple clusters using the `kafka.clusters` array:\n\n```yaml\nkafka:\n  clusters:\n    - name: production                              # Descriptive name (for your reference)\n      seedBrokers: \"broker1:9092,broker2:9092\"      # Required: comma-separated broker list\n      auth:\n        saslMechanism: \"SCRAM-SHA-256\"              # Optional: PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512\n        saslUser: \"prod-user\"                       # Optional: SASL username\n        saslPassword: \"prod-pass\"                   # Optional: SASL password\n        useTLS: \"true\"                              # Optional: enable TLS\n        caFile: \"/path/to/ca.pem\"                   # Optional: CA certificate path\n        certChainFile: \"/path/to/cert.pem\"          # Optional: client certificate path\n        certKeyFile: \"/path/to/key.pem\"             # Optional: client key path\n\n    - name: staging\n      seedBrokers: \"staging-broker:9092\"\n      auth:\n        existingSecret: \"staging-kafka-secret\"      # Optional: use existing K8s secret for auth\n```\n\n#### Using Existing Secrets\n\nFor each cluster, you can reference an existing Kubernetes secret containing the authentication credentials. The secret should contain the following keys:\n\n- `sasl-mechanism` - SASL mechanism (PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512)\n- `sasl-user` - SASL username\n- `sasl-password` - SASL password\n\nExample secret:\n\n```yaml\napiVersion: v1\nkind: Secret\nmetadata:\n  name: staging-kafka-secret\ntype: Opaque\nstringData:\n  sasl-mechanism: \"SCRAM-SHA-256\"\n  sasl-user: \"staging-user\"\n  sasl-password: \"staging-password\"\n```\n\n#### Environment Variables (Alternative)\n\nMulti-cluster configuration can also be done via indexed environment variables. Replace `X` with the cluster index (0, 1, 2, ...):\n\n| Environment Variable                                      | Meaning                                      |\n|-----------------------------------------------------------|----------------------------------------------|\n| `STEADYBIT_EXTENSION_CLUSTER_X_SEED_BROKERS`              | Comma-separated broker list                  |\n| `STEADYBIT_EXTENSION_CLUSTER_X_SASL_MECHANISM`            | PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512       |\n| `STEADYBIT_EXTENSION_CLUSTER_X_SASL_USER`                 | SASL username                                |\n| `STEADYBIT_EXTENSION_CLUSTER_X_SASL_PASSWORD`             | SASL password                                |\n| `STEADYBIT_EXTENSION_CLUSTER_X_KAFKA_CONNECTION_USE_TLS`  | Enable TLS (true/false)                      |\n| `STEADYBIT_EXTENSION_CLUSTER_X_KAFKA_CLUSTER_CA_FILE`     | CA certificate path                          |\n| `STEADYBIT_EXTENSION_CLUSTER_X_KAFKA_CLUSTER_CERT_CHAIN_FILE` | Client certificate path                  |\n| `STEADYBIT_EXTENSION_CLUSTER_X_KAFKA_CLUSTER_CERT_KEY_FILE`   | Client key path                          |\n\nExample:\n\n```bash\n# First cluster\nSTEADYBIT_EXTENSION_CLUSTER_0_SEED_BROKERS=prod-broker1:9092,prod-broker2:9092\nSTEADYBIT_EXTENSION_CLUSTER_0_SASL_MECHANISM=SCRAM-SHA-256\nSTEADYBIT_EXTENSION_CLUSTER_0_SASL_USER=prod-user\nSTEADYBIT_EXTENSION_CLUSTER_0_SASL_PASSWORD=prod-pass\n\n# Second cluster\nSTEADYBIT_EXTENSION_CLUSTER_1_SEED_BROKERS=staging-broker:9092\nSTEADYBIT_EXTENSION_CLUSTER_1_SASL_MECHANISM=PLAIN\nSTEADYBIT_EXTENSION_CLUSTER_1_SASL_USER=staging-user\nSTEADYBIT_EXTENSION_CLUSTER_1_SASL_PASSWORD=staging-pass\n```\n\nThe extension supports all environment variables provided\nby [steadybit/extension-kit](https://github.com/steadybit/extension-kit#environment-variables).\n\n## Installation\n\n### Using Docker\n\n```sh\ndocker run \\\n  --rm \\\n  -p 8080 \\\n  --name steadybit-extension-kafka \\\n  --env STEADYBIT_EXTENSION_SEED_BROKERS=\"localhost:9092\" \\\n  ghcr.io/steadybit/extension-kafka:latest\n```\n\n### Using Helm in Kubernetes\n\n```sh\nhelm repo add steadybit-extension-kafka https://steadybit.github.io/extension-kafka\nhelm repo update\nhelm upgrade steadybit-extension-kafka \\\n    --install \\\n    --wait \\\n    --timeout 5m0s \\\n    --create-namespace \\\n    --namespace steadybit-agent \\\n    --set kafka.seedBrokers=\"localhost:9092\" \\\n    steadybit-extension-kafka/steadybit-extension-kafka\n```\n\n## Register the extension\n\nMake sure to register the extension on the Steadybit platform. Please refer to\nthe [documentation](https://docs.steadybit.com/integrate-with-steadybit/extensions/extension-installation) for more\ninformation.\n\n## Generate pem files from truststore and keystore\n\n### Prerequisites\n\n- **Keystore file**: `kafka.keystore.jks` (contains the client certificate and private key).\n- **Truststore file**: `kafka.truststore.jks` (contains the CA certificate).\n- **Tools Required**: `keytool` and `openssl` must be installed.\n\n---\n\n### Steps to Generate PEM Files\n\n1. **Export the CA Certificate (`ca-cert.pem`)**\n\t Extract the CA certificate from the truststore using the following command:\n\n```bash\nkeytool -exportcert \\\n -keystore kafka.truststore.jks \\\n -alias CARoot \\\n -storepass \u003ctruststore-password\u003e \\\n -rfc -file ca-cert.pem\n```\n\n• Replace \u003ctruststore-password\u003e with the password for the truststore.\n• The output file ca-cert.pem will contain the CA certificate in PEM format.\n\n2. **Convert the Keystore to PKCS12 Format**\n\nConvert the keystore to a PKCS12 file to facilitate extracting the certificate and private key:\n\n```bash\nkeytool -importkeystore \\\n-srckeystore kafka.keystore.jks \\\n-srcstorepass \u003ckeystore-password\u003e \\\n-srcalias kafka \\\n-destkeystore kafka-keystore.p12 \\\n-deststoretype PKCS12 \\\n-deststorepass \u003cp12-password\u003e\n```\n\n• Replace \u003ckeystore-password\u003e with the password for the keystore.\n• Replace \u003cp12-password\u003e with a new password for the PKCS12 file.\n• This will generate the file kafka-keystore.p12, which contains both the client certificate and private key.\n\n3. **Extract the Private Key (client-key.pem)**\n\t Use the following command to extract the private key from the PKCS12 file:\n\n```bash\n openssl pkcs12 -in kafka-keystore.p12 \\\n -nocerts -nodes -out client-key.pem \\\n -passin pass:\u003cp12-password\u003e\n```\n\n• Replace \u003cp12-password\u003e with the password set for the PKCS12 file.\n• This will generate the file client-key.pem, which contains the private key in PEM format.\n\n4. **Extract the Client Certificate (client-cert.pem)**\n\t Use the following command to extract the client certificate from the PKCS12 file:\n\n```bash\n openssl pkcs12 -in kafka-keystore.p12 \\\n -clcerts -nokeys -out client-cert.pem \\\n -passin pass:\u003cp12-password\u003e\n```\n\n• Replace \u003cp12-password\u003e with the password set for the PKCS12 file.\n• This will generate the file client-cert.pem, which contains the client certificate in PEM format.\n\n5. **(Optional) Verifying the Generated PEM Files**\n\n```bash\nopenssl x509 -in ca-cert.pem -text -noout\nopenssl rsa -in client-key.pem -check\nopenssl x509 -in client-cert.pem -text -noout\n```\n\nEnsure that:\n• The CA certificate includes the correct issuer and validity period.\n• The private key matches the client certificate.\n\n#### Issue: “Alias not found”\n\nVerify the contents of the keystore and truststore:\n\n```bash\nkeytool -list -v -keystore kafka.keystore.jks -storepass \u003ckeystore-password\u003e\nkeytool -list -v -keystore kafka.truststore.jks -storepass \u003ctruststore-password\u003e\n```\n\n#### Notes\n\n1. The private key (client-key.pem) must be kept secure. Unauthorized access to this file can compromise the client.\n2. Ensure the Kafka broker’s hostname or IP address matches the Subject Alternative Name (SAN) in the server’s\n\t certificate.\n3. Always use strong passwords for your keystore, truststore, and PKCS12 files.\n\n## Version and Revision\n\nThe version and revision of the extension:\n\n- are printed during the startup of the extension\n- are added as a Docker label to the image\n- are available via the `version.txt`/`revision.txt` files in the root of the image\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsteadybit%2Fextension-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsteadybit%2Fextension-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsteadybit%2Fextension-kafka/lists"}