{"id":17650223,"url":"https://github.com/mongey/terraform-provider-kafka","last_synced_at":"2025-04-13T23:55:44.744Z","repository":{"id":37444738,"uuid":"103748731","full_name":"Mongey/terraform-provider-kafka","owner":"Mongey","description":"Terraform provider for managing Apache Kafka Topics + ACLs","archived":false,"fork":false,"pushed_at":"2025-04-04T17:03:24.000Z","size":11367,"stargazers_count":539,"open_issues_count":86,"forks_count":138,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-04-06T21:09:18.220Z","etag":null,"topics":["kafka","kafka-acl","kafka-topic","terraform","terraform-provider"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mongey.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"Mongey"}},"created_at":"2017-09-16T11:59:37.000Z","updated_at":"2025-04-01T02:18:45.000Z","dependencies_parsed_at":"2023-02-13T01:05:14.564Z","dependency_job_id":"847f91e8-a693-4957-8751-c5806a0e2f8c","html_url":"https://github.com/Mongey/terraform-provider-kafka","commit_stats":{"total_commits":299,"total_committers":41,"mean_commits":"7.2926829268292686","dds":0.4782608695652174,"last_synced_commit":"810a0f002ad5cfeaf87c3dc8e476f4bdea5adaa8"},"previous_names":[],"tags_count":41,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mongey%2Fterraform-provider-kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mongey%2Fterraform-provider-kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mongey%2Fterraform-provider-kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mongey%2Fterraform-provider-kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mongey","download_url":"https://codeload.github.com/Mongey/terraform-provider-kafka/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248799922,"owners_count":21163403,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka","kafka-acl","kafka-topic","terraform","terraform-provider"],"created_at":"2024-10-23T11:37:14.357Z","updated_at":"2025-04-13T23:55:44.721Z","avatar_url":"https://github.com/Mongey.png","language":"Go","funding_links":["https://github.com/sponsors/Mongey"],"categories":[],"sub_categories":[],"readme":"# `terraform-provider-kafka`\n[![CircleCI](https://circleci.com/gh/Mongey/terraform-provider-kafka.svg?style=svg)](https://circleci.com/gh/Mongey/terraform-provider-kafka)\n\nA [Terraform][1] plugin for managing [Apache Kafka][2].\n\n## Contents\n\n* [Installation](#installation)\n  * [Developing](#developing)\n* [`kafka` Provider](#provider-configuration)\n* [Resources](#resources)\n  * [`kafka_topic`](#kafka_topic)\n  * [`kafka_acl`](#kafka_acl)\n  * [`kafka_quota`](#kafka_quota)\n* [Requirements](#requirements)\n\n## Installation\n\n`terraform-provider-kafka` is available on the terraform registry. To install, add\nthe below into your `main.tf` and execute `terraform init`\n\n```tf\nterraform {\n  required_providers {\n    kafka = {\n      source = \"Mongey/kafka\"\n    }\n  }\n}\n\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9092\"]\n  ca_cert           = file(\"../secrets/ca.crt\")\n  client_cert       = file(\"../secrets/terraform-cert.pem\")\n  client_key        = file(\"../secrets/terraform.pem\")\n  tls_enabled       = true\n}\n```\n\nOtherwise, install by downloading and extracting the [latest\nrelease](https://github.com/Mongey/terraform-provider-kafka/releases/latest) to\nyour [terraform plugin directory][third-party-plugins] (typically `~/.terraform.d/plugins/`)\n\n### Developing\n\n0. [Install go][install-go]\n0. Clone repository to: `$GOPATH/src/github.com/Mongey/terraform-provider-kafka`\n    ``` bash\n    mkdir -p $GOPATH/src/github.com/Mongey/terraform-provider-kafka; cd $GOPATH/src/github.com/Mongey/\n    git clone https://github.com/Mongey/terraform-provider-kafka.git\n    cd terraform-provider-kafka\n    ```\n0. Build the provider `make build`\n0. Run the tests `make test`\n0. Start a TLS enabled kafka-cluster `docker-compose up`\n0. Run the acceptance tests `make testacc`\n\n## Provider Configuration\n\n### Example\n\nExample provider with TLS client authentication.\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9092\"]\n  ca_cert           = file(\"../secrets/ca.crt\")\n  client_cert       = file(\"../secrets/terraform-cert.pem\")\n  client_key        = file(\"../secrets/terraform.pem\")\n  tls_enabled       = true\n}\n```\n\nExample provider with aws-iam(Assume role) client authentication.\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9098\"]\n  tls_enabled       = true\n  sasl_mechanism    = \"aws-iam\"\n  sasl_aws_region   = \"us-east-1\"\n  sasl_aws_role_arn = \"arn:aws:iam::account:role/role-name\"\n}\n```\n\nExample provider with aws-iam(Aws Profile) client authentication.\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9098\"]\n  tls_enabled       = true\n  sasl_mechanism    = \"aws-iam\"\n  sasl_aws_region   = \"us-east-1\"\n  sasl_aws_profile  = \"dev\"\n}\n```\n\nExample provider with aws-iam(Static Creds) client authentication using explicit credentials.\n```hcl\nprovider \"vault\" {\n  auth_login_jwt {\n    role = \"jwt-role-name\"\n  }\n}\n\ndata \"vault_aws_access_credentials\" \"creds\" {\n  backend = \"aws\"\n  type    = \"sts\"\n  role    = \"sts-role-name\"\n}\n\nprovider \"kafka\" {\n  bootstrap_servers   = [\"localhost:9098\"]\n  tls_enabled         = true\n  sasl_mechanism      = \"aws-iam\"\n  sasl_aws_region     = \"us-east-1\"\n  sasl_aws_access_key = data.vault_aws_access_credentials.creds.access_key\n  sasl_aws_secret_key = data.vault_aws_access_credentials.creds.secret_key\n  sasl_aws_token      = data.vault_aws_access_credentials.creds.security_token\n}\n```\n\nExample provider with aws-iam(Static Creds) client authentication. You have to export `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`(Optional if you are using temp creds)\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9098\"]\n  tls_enabled       = true\n  sasl_mechanism    = \"aws-iam\"\n  sasl_aws_region   = \"us-east-1\"\n}\n```\n\nExample provider with aws-iam(Container Creds) client authentication. You have to export `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` and `AWS_CONTAINER_CREDENTIALS_FULL_URI`\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9098\"]\n  tls_enabled       = true\n  sasl_mechanism    = \"aws-iam\"\n  sasl_aws_region   = \"us-east-1\"\n}\n```\n#### Compatibility with Redpanda\n\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9092\"]\n  kafka_version = \"2.1.0\"\n}\n```\n\nDue to Redpanda not implementing some Metadata APIs, we need to force the Kafka version to use when creating the provider.\n\n| Property                | Description                                                                                                           | Default    |\n| -------------------     | --------------------------------------------------------------------------------------------------------------------- | ---------- |\n| `bootstrap_servers`     | A list of host:port addresses that will be used to discover the full set of alive brokers                             | `Required` |\n| `ca_cert`               | The CA certificate or path to a CA certificate file in `PEM` format to validate the server's certificate.             | `\"\"`       |\n| `client_cert`           | The client certificate or path to a file containing the client certificate in `PEM` format. Use for Client authentication to Kafka.\u003cbr\u003eIf you have Intermediate CA certificate(s) append them to `client_cert`.| `\"\"`       |\n| `client_key`            | The private key or path to a file containing the private key that the client certificate was issued for.              | `\"\"`       |\n| `client_key_passphrase` | The passphrase for the private key that the certificate was issued for.                                               | `\"\"`       |\n| `kafka_version`         | The version of Kafka protocol to use in `$MAJOR.$MINOR.$PATCH` format. Some features may not be available on older versions.  | `\"\"`       |\n| `tls_enabled`           | Enable communication with the Kafka Cluster over TLS.                                                                 | `true`     |\n| `skip_tls_verify`       | Skip TLS verification.                                                                                                | `false`    |\n| `sasl_username`         | Username for SASL authentication.                                                                                     | `\"\"`       |\n| `sasl_password`         | Password for SASL authentication.                                                                                     | `\"\"`       |\n| `sasl_mechanism`        | Mechanism for SASL authentication. Allowed values are `plain`, `aws-iam`,  `scram-sha256`, `scram-sha512` or `oauthbearer`      | `plain`    |\n| `sasl_aws_region`       | AWS region for IAM authentication.                                                                                    | `\"\"`       |\n| `sasl_aws_container_authorization_token_file`       | Path to a file containing the AWS pod identity authorization token.                                                                                    | `\"\"`       |\n| `sasl_aws_container_credentials_full_uri`       | URI to retrieve AWS credentials from.                                                                                    | `\"\"`       |\n| `sasl_aws_role_arn`     | Arn of AWS IAM role to assume for IAM authentication.                                                                 | `\"\"`       |\n| `sasl_aws_profile`      | AWS profile to use for IAM authentication.                                                                            | `\"\"`       |\n| `sasl_aws_access_key`   | AWS access key.                                                                                                       | `\"\"`       |\n| `sasl_aws_secret_key`   | AWS secret key.                                                                                                       | `\"\"`       |\n| `sasl_aws_token`        | AWS session token.                                                                                                    | `\"\"`       |\n| `sasl_aws_creds_debug`  | Enable debug logging for AWS authentication.                                                                          | `false`    |\n| `sasl_token_url`        | The url to retrieve oauth2 tokens from, when using sasl mechanism `oauthbearer`                                         | `\"\"`    |\n\n\n## Resources\n### `kafka_topic`\n\nA resource for managing Kafka topics. Increases partition count without\ndestroying the topic.\n\n#### Example\n\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9092\"]\n}\n\nresource \"kafka_topic\" \"logs\" {\n  name               = \"systemd_logs\"\n  replication_factor = 2\n  partitions         = 100\n\n  config = {\n    \"segment.ms\"     = \"20000\"\n    \"cleanup.policy\" = \"compact\"\n  }\n}\n```\n\n#### Properties\n\n| Property             | Description                                    |\n| -------------------- | ---------------------------------------------- |\n| `name`               | The name of the topic                          |\n| `partitions`         | The number of partitions the topic should have |\n| `replication_factor` | The number of replicas the topic should have   |\n| `config`             | A map of string [K/V attributes][topic-config] |\n\n\n#### Importing Existing Topics\nYou can import topics with the following\n\n```sh\nterraform import kafka_topic.logs systemd_logs\n```\n\n\n### `kafka_acl`\nA resource for managing Kafka ACLs.\n\n#### Example\n\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9092\"]\n  ca_cert           = file(\"../secrets/ca.crt\")\n  client_cert       = file(\"../secrets/terraform-cert.pem\")\n  client_key        = file(\"../secrets/terraform.pem\")\n}\n\nresource \"kafka_acl\" \"test\" {\n  resource_name       = \"syslog\"\n  resource_type       = \"Topic\"\n  acl_principal       = \"User:Alice\"\n  acl_host            = \"*\"\n  acl_operation       = \"Write\"\n  acl_permission_type = \"Deny\"\n}\n```\n\n#### Properties\n\n| Property                       | Description                                                        | Valid values                                                                                                                                             |\n| ------------------------------ | ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `acl_principal`                | Principal that is being allowed or denied                          | `*`                                                                                                                                                      |\n| `acl_host`                     | Host from which principal listed in acl_principal will have access | `*`                                                                                                                                                      |\n| `acl_operation`                | Operation that is being allowed or denied                          | `Unknown`, `Any`, `All`, `Read`, `Write`, `Create`, `Delete`, `Alter`, `Describe`, `ClusterAction`, `DescribeConfigs`, `AlterConfigs`, `IdempotentWrite` |\n| `acl_permission_type`          | Type of permission                                                 | `Unknown`, `Any`, `Allow`, `Deny`                                                                                                                        |\n| `resource_name`                | The name of the resource                                           | `*`                                                                                                                                                      |\n| `resource_type`                | The type of resource                                               | `Unknown`, `Any`, `Topic`, `Group`, `Cluster`, `TransactionalID`                                                                                         |\n| `resource_pattern_type_filter` |                                                                    | `Prefixed`, `Any`, `Match`, `Literal`                                                                                                                    |\n\n\n#### Importing Existing ACLs\nFor import, use as a parameter the items separated by `|` character. Quote it to avoid shell expansion.\n\n```sh\n# Fields in shell notation are\n# ${acl_principal}|${acl_host}|${acl_operation}|${acl_permission_type}|${resource_type}|${resource_name}|${resource_pattern_type_filter}\nterraform import kafka_acl.admin 'User:12345|*|Describe|Allow|Topic|experimental-topic|Prefixed'\n```\n\n### `kafka_quota`\nA resource for managing Kafka Quotas.\n\n#### Example\n\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9092\"]\n  ca_cert           = file(\"../secrets/ca.crt\")\n  client_cert       = file(\"../secrets/terraform-cert.pem\")\n  client_key        = file(\"../secrets/terraform.pem\")\n}\n\nresource \"kafka_quota\" \"test\" {\n  entity_name       = \"client1\"\n  entity_type       = \"client-id\"\n  config = {\n    \"consumer_byte_rate\" = \"4000000\"\n    \"producer_byte_rate\" = \"3500000\"\n  }\n}\n\nresource \"kafka_quota\" \"default_user_quota\" {\n  entity_type = \"user\"\n  config = {\n    \"consumer_byte_rate\" = \"2000000\"\n    \"producer_byte_rate\" = \"1500000\"\n  }\n}\n```\n\n#### Properties\n\n| Property             | Description                                                                                         |\n| -------------------- | --------------------------------------------------------------------------------------------------- |\n| `entity_name`        | The name of the entity (if entity_name is not provided, it will create entity-default Kafka quota)  |\n| `entity_type`        | The entity type (client-id, user, ip)                                                               |\n| `config`             | A map of string attributes for the entity                                                           |\n\n### `kafka_user_scram_credential`\nA resource for managing Kafka SCRAM user credentials.\n\n#### Example\n\n```hcl\nprovider \"kafka\" {\n  bootstrap_servers = [\"localhost:9092\"]\n  ca_cert           = file(\"../secrets/ca.crt\")\n  client_cert       = file(\"../secrets/terraform-cert.pem\")\n  client_key        = file(\"../secrets/terraform.pem\")\n}\n\nresource \"kafka_user_scram_credential\" \"test\" {\n  username               = \"user1\"\n  scram_mechanism        = \"SCRAM-SHA-256\"\n  scram_iterations       = \"8192\"\n  password               = \"password\"\n}\n```\n\n#### Importing Existing SCRAM user credentials\nFor import, use as a parameter the items separated by `|` character. Quote it to avoid shell expansion.\n\n```sh\n# Fields in shell notation are\n# ${username}|${scram_mechanism}|${password}\nterraform import kafka_user_scram_credential.test 'user1|SCRAM-SHA-256|password'\n```\n\n#### Properties\n\n| Property             | Description                                    |\n| -------------------- | ---------------------------------------------- |\n| `username`        | The username                         |\n| `scram_mechanism`        | The SCRAM mechanism (SCRAM-SHA-256 or SCRAM-SHA-512)          |\n| `scram_iterations`             | The number of SCRAM iterations (must be \u003e= 4096). Default: 4096       |\n| `password` | The password for the user |\n\n## Requirements\n* [\u003e= Kafka 1.0.0][3]\n\n[1]: https://www.terraform.io\n[2]: https://kafka.apache.org\n[3]: https://cwiki.apache.org/confluence/display/KAFKA/KIP-117%3A+Add+a+public+AdminClient+API+for+Kafka+admin+operations\n[third-party-plugins]: https://www.terraform.io/docs/configuration/providers.html#third-party-plugins\n[install-go]: https://golang.org/doc/install#install\n[topic-config]: https://kafka.apache.org/documentation/#topicconfigs \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmongey%2Fterraform-provider-kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmongey%2Fterraform-provider-kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmongey%2Fterraform-provider-kafka/lists"}