{"id":25257539,"url":"https://github.com/michelin/kafkactl","last_synced_at":"2025-10-27T03:30:22.412Z","repository":{"id":65629086,"uuid":"564382700","full_name":"michelin/kafkactl","owner":"michelin","description":"Command-line interface for Ns4Kafka.","archived":false,"fork":false,"pushed_at":"2025-10-26T00:15:40.000Z","size":1014,"stargazers_count":15,"open_issues_count":11,"forks_count":7,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-10-26T02:28:13.428Z","etag":null,"topics":["cli","gitops","graalvm-native-image","kafka","namespace","ns4kafka"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/michelin.png","metadata":{"files":{"readme":"README.md","changelog":"changelog-builder.json","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-11-10T15:45:21.000Z","updated_at":"2025-10-20T15:03:23.000Z","dependencies_parsed_at":"2023-10-03T18:27:24.973Z","dependency_job_id":"56228c23-9e51-4a77-8a10-b43200407f86","html_url":"https://github.com/michelin/kafkactl","commit_stats":null,"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/michelin/kafkactl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michelin%2Fkafkactl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michelin%2Fkafkactl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michelin%2Fkafkactl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michelin%2Fkafkactl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/michelin","download_url":"https://codeload.github.com/michelin/kafkactl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michelin%2Fkafkactl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281211345,"owners_count":26462074,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-27T02:00:05.855Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","gitops","graalvm-native-image","kafka","namespace","ns4kafka"],"created_at":"2025-02-12T06:49:02.569Z","updated_at":"2025-10-27T03:30:22.405Z","avatar_url":"https://github.com/michelin.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\".readme/logo.svg\" alt=\"Kafkactl\"/\u003e\n\n# Kafkactl\n\n[![GitHub Build](https://img.shields.io/github/actions/workflow/status/michelin/kafkactl/build.yml?branch=main\u0026logo=github\u0026style=for-the-badge)](https://img.shields.io/github/actions/workflow/status/michelin/kafkactl/build.yml)\n[![GitHub release](https://img.shields.io/github/v/release/michelin/kafkactl?logo=github\u0026style=for-the-badge)](https://github.com/michelin/kafkactl/releases)\n[![GitHub Stars](https://img.shields.io/github/stars/michelin/kafkactl?logo=github\u0026style=for-the-badge)](https://github.com/michelin/kafkactl)\n[![Docker Pulls](https://img.shields.io/docker/pulls/michelin/kafkactl?label=Pulls\u0026logo=docker\u0026style=for-the-badge)](https://hub.docker.com/r/michelin/kafkactl/tags)\n[![SonarCloud Coverage](https://img.shields.io/sonar/coverage/michelin_kafkactl?logo=sonarcloud\u0026server=https%3A%2F%2Fsonarcloud.io\u0026style=for-the-badge)](https://sonarcloud.io/component_measures?id=michelin_kafkactl\u0026metric=coverage\u0026view=list)\n[![SonarCloud Tests](https://img.shields.io/sonar/tests/michelin_kafkactl/main?server=https%3A%2F%2Fsonarcloud.io\u0026style=for-the-badge\u0026logo=sonarcloud)](https://sonarcloud.io/component_measures?metric=tests\u0026view=list\u0026id=michelin_kafkactl)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg?logo=apache\u0026style=for-the-badge)](https://opensource.org/licenses/Apache-2.0)\n\n[Download](#download) • [Configuration](#configuration) • [Ns4Kafka](https://github.com/michelin/ns4kafka)\n\nCommand-line interface for Ns4Kafka.\n\nKafkactl enables the deployment of Kafka resources to Ns4Kafka using YAML descriptors.\n\n\u003c/div\u003e\n\n## Table of Contents\n\n* [Download](#download)\n* [Configuration](#configuration)\n  * [Contexts](#contexts)\n  * [Authentication](#authentication)\n  * [HTTP Client](#http-client)\n    * [Timeout](#timeout)\n    * [Retry](#retry)\n* [Usage](#usage)\n    * [Api Resources](#api-resources)\n    * [Apply](#apply)\n    * [Auth](#auth)\n        * [Info](#info)\n        * [Renew](#renew)\n    * [Config](#config)\n        * [Current Context](#current-context)\n        * [Get Contexts](#get-contexts)\n        * [Use Context](#use-context)\n    * [Connect Cluster](#connect-cluster)\n        * [Vault](#vault)\n    * [Connector](#connector)\n    * [Delete Records](#delete-records)\n    * [Delete](#delete)\n    * [Diff](#diff)\n    * [Get](#get)\n    * [Import](#import)\n    * [Reset Offsets](#reset-offsets)\n    * [Reset Password](#reset-password)\n    * [Schema](#schema)\n* [Resources](#resources)\n    * [User](#user)\n        * [Topic](#topic)\n        * [ACL](#acl)\n        * [Connector](#connector-2)\n        * [Connect Cluster](#connect-cluster-2)\n        * [Kafka Streams](#kafka-streams)\n        * [Schema](#schema-2)\n    * [Administrator](#administrator)\n        * [Namespace](#namespace)\n            * [Validation Constraints](#validation-constraints)\n                * [Range](#range)\n                * [ValidList](#validlist)\n                * [ValidString](#validstring)\n                * [NonEmptyString](#nonemptystring)\n                * [ContainsList](#containslist)\n                * [CompositeValidator](#compositevalidator)\n        * [ACL Owner](#acl-owner)\n        * [Role Binding](#role-binding)\n        * [Quota](#quota)\n* [CI/CD](#cicd)\n* [Contribution](#contribution)\n\n## Download\n\nYou can download Kafkactl from the [GitHub releases page](https://github.com/michelin/kafkactl/releases). It is available in the following formats:\n\n- JAR (requires Java 21)\n- Windows\n- Linux AMD64 (statically linked with Musl)\n- Linux ARM64\n- MacOS\n\nAdditionally, Docker images are available on [Docker Hub](https://hub.docker.com/repository/docker/michelin/kafkactl):\n\n- The `kafkactl:\u003cversion\u003e` images contains Kafkactl as a JAR.\n- The `kafkactl:\u003cversion\u003e-native` images contains Kafkactl as a native binary.\n\n## Configuration\n\n### Contexts\n\nTo use Kafkactl, you need to define three variables:\n\n- The URL of Ns4Kafka\n- Your namespace\n- Your security token (e.g., a GitLab token)\n\nThese variables can be defined in a dedicated configuration file `.kafkactl/config.yml`.\n\n- Windows: `C:\\Users\\Name\\.kafkactl\\config.yml`\n- Linux: `~/.kafkactl/config.yml`\n\nIt is possible to override this default location by setting the **KAFKACTL_CONFIG** environment variable:\n\n```console\nKAFKACTL_CONFIG=C:\\AnotherDirectory\\config.yml\nKAFKACTL_CONFIG=/anotherDirectory/config.yml\n```\n\nFill the `config.yml` file with the following content:\n\n```yaml\nkafkactl:\n  contexts:\n    - name: dev\n      context:\n        api: https://ns4kafka-dev-api.domain.com\n        user-token: my_gitlab_token\n        namespace: my_namespace\n    - name: prod\n      context:\n        api: https://ns4kafka-prod-api.domain.com\n        user-token: my_gitlab_token\n        namespace: my_namespace\n```\n\nFor each context, define your token and your namespace.\n\nTo check all available contexts, use the following command:\n\n```command\nkafkactl config get-contexts\n```\n\nTo set yourself on a given context, use the following command:\n\n```command\nkafkactl config use-context dev\n```\n\nTo check your current context, use the following command:\n\n```command\nkafkactl config current-context\n```\n\n### Authentication\n\nKafkactl supports only Bearer JWT authentication. It requires a token to be set in the `user-token` field of the context\nwithin the `config.yml` file.\n\n- [GitLab](https://github.com/michelin/ns4kafka?tab=readme-ov-file#gitlab): A personal access token can be used.\n- [Local Users](https://github.com/michelin/ns4kafka?tab=readme-ov-file#local-users): The local user's password must be\n  used as the token. The username should be set to `gitlab` in the\n  Ns4Kafka configuration.\n\nUpon successful authentication, a JWT token signed by Ns4Kafka is stored in the `~/.kafkactl` directory.\n\n### HTTP Client\n\nKafkactl uses an HTTP client to communicate with the Ns4Kafka API.\n\n##### Timeout\n\nHTTP client timeouts can be configured using the following properties:\n\n```yaml\nmicronaut:\n  http:\n    client:\n      connect-timeout: '60s'\n      read-idle-timeout: '60s'\n      read-timeout: '60s'\n```\n\nTimeouts are set to 60 seconds by default, which is suitable to allow the Ns4Kafka server to perform retries if needed.\n\n##### Retry\n\nThe Kafkactl HTTP client is configured to retry requests in case of errors. The retry behavior is controlled by the following properties:\n\n```yaml\nkafkactl:\n  retry:\n    attempt: '5'\n    delay: '2s'\n    multiplier: '2.0'\n```\n\n## Usage\n\n```console\nUsage: kafkactl [-hV] [COMMAND]\n\nDescription: These are common Kafkactl commands.\n\nOptions:\n  -h, --help      Show this help message and exit.\n  -V, --version   Print version information and exit.\n\nCommands:\n  api-resources    Print the supported API resources on the server.\n  apply            Create or update a resource.\n  auth             Interact with authentication.\n  config           Manage configuration.\n  connect-cluster  Interact with connect clusters.\n  connector        Interact with connectors.\n  delete-records   Delete all records within a topic.\n  delete           Delete a resource.\n  diff             Get differences between a new resource and a old resource.\n  get              Get resources by resource type for the current namespace.\n  import           Import non-synchronized resources.\n  reset-offsets    Reset consumer group offsets.\n  schema           Interact with schemas.\n  reset-password   Reset a Kafka password.\n```\n\n### Api Resources\n\nThe `api-resources` command allows you to check which resources can be accessed through the API.\n\n```console\nUsage: kafkactl api-resources [-hv] [-n=\u003coptionalNamespace\u003e]\n\nDescription: Print the supported API resources on the server.\n\nOptions:\n  -h, --help      Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                  Override namespace defined in config or YAML resources.\n  -v, --verbose   Enable the verbose mode.\n```\n\nExample(s):\n\n```console\nkafkactl api-resources\n```\n\n### Auth\n\nThe `auth` command allows you to interact with authentication.\n\n```console\nUsage: kafkactl auth [-h] COMMAND\n\nDescription: Interact with authentication.\n\nOptions:\n  -h, --help   Show this help message and exit.\n\nCommands:\n  info   Get the JWT token information.\n  renew  Renew the JWT token.\n```\n\n#### Info\n\nThe `info` command allows you to get the JWT token information.\n\n```console\nUsage: kafkactl auth info [-h] [-o=\u003coutput\u003e]\n\nDescription: Get the JWT token information.\n\nOptions:\n  -h, --help              Show this help message and exit.\n  -o, --output=\u003coutput\u003e   Output format (yaml, yml, table).\n```\n\nExample(s):\n\n```console\nkafkactl auth info\n```\n\n#### Renew\n\nThe `renew` command allows you to renew the JWT token.\n\n```console\nUsage: kafkactl auth renew [-hv]\n\nDescription: Renew the JWT token.\n\nOptions:\n  -h, --help      Show this help message and exit.\n  -v, --verbose   Enable the verbose mode.\n```\n\nExample(s):\n\n```console\nkafkactl auth renew\n```\n\n### Apply\n\nThe `apply` command allows you to deploy a resource.\n\n```console\nUsage: kafkactl apply [-hRv] [--dry-run] [-f=\u003cfile\u003e] [-n=\u003coptionalNamespace\u003e]\n\nDescription: Create or update a resource.\n\nOptions:\n      --dry-run       Does not persist resources. Validate only.\n  -f, --file=\u003cfile\u003e   YAML file or directory containing resources.\n  -h, --help          Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                      Override namespace defined in config or YAML resources.\n  -R, --recursive     Search file recursively.\n  -v, --verbose       Enable the verbose mode.\n```\n\nExample(s):\n\n```console\nkafkactl apply -f directoryOfResources\nkafkactl apply -f resource.yml\n```\n\nThe resources have to be described in YAML manifests.\n\n### Config\n\nThe `config` command allows you to manage your Kafka contexts.\n\n```console\nUsage: kafkactl config [-h] COMMAND\n\nDescription: Manage configuration.\n\nOptions:\n  -h, --help   Show this help message and exit.\n\nCommands:\n  get-contexts     Get all contexts.\n  use-context      Use a context.\n  current-context  Get the current context.\n```\n\n#### Current Context\n\nThe `current-context` command allows you to check the current context.\n\n```console\nUsage: kafkactl config current-context [-hu]\n\nDescription: Get the current context.\n\nOptions:\n  -h, --help            Show this help message and exit.\n  -u, --unmask-tokens   Unmask tokens.\n```\n\nExample(s):\n\n```console\nkafkactl config current-context\n```\n\n#### Get Contexts\n\nThe `get-contexts` command allows you to list all the contexts defined in your configuration file.\n\n```console\nUsage: kafkactl config get-contexts [-hu]\n\nDescription: Get all contexts.\n\nOptions:\n  -h, --help            Show this help message and exit.\n  -u, --unmask-tokens   Unmask tokens.\n```\n\nExample(s):\n\n```console\nkafkactl config get-contexts\n```\n\n#### Use Context\n\nThe `use-context` command allows you to switch to a different context.\n\n```console\nUsage: kafkactl config use-context [-h] \u003ccontext\u003e\n\nDescription: Use a context.\n\nParameters:\n      \u003ccontext\u003e   Context to use.\n\nOptions:\n  -h, --help      Show this help message and exit.\n```\n\nExample(s):\n\n```console\nkafkactl config use-context local\n```\n\n### Connect Cluster\n\nThe `connect-cluster` command allows you to interact with Kafka Connect clusters.\n\n```console\nUsage: kafkactl connect-cluster [-h] [COMMAND]\n\nDescription: Interact with connect clusters.\n\nOptions:\n  -h, --help   Show this help message and exit.\n\nCommands:\n  vault  Vault secrets for a connect cluster.\n```\n\n#### Vault\n\nThe `vault` command allows you to vault sensitive connector configuration.\n\n```console\nUsage: kafkactl connect-cluster vault [-hv] [-n=\u003coptionalNamespace\u003e] \u003cconnectClusterName\u003e [\u003csecrets\u003e...]\n\nDescription: Vault secrets for a connect cluster.\n\nParameters:\n      \u003cconnectClusterName\u003e   Connect cluster name that will vault the secrets.\n      [\u003csecrets\u003e...]         Secrets to vaults separated by space.\n\nOptions:\n  -h, --help                 Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                             Override namespace defined in config or YAML resources.\n  -v, --verbose              Enable the verbose mode.\n```\n\n- `connectClusterName`: If defined, this option specifies the name of a Connect cluster to use to vault sensitive\n  connector configuration.\n- `secrets`: This option specifies the clear text to encrypt. You can specify one or more secrets as command arguments.\n\nExample(s):\n\n```console\nkafkactl connect-cluster vault\nkafkactl connect-cluster vault myConnectCluster someClearText\n```\n\n### Connector\n\nThe `connector` command allows you to interact with Kafka Connect connectors.\n\n```console\nUsage: kafkactl connector [-hv] [-n=\u003coptionalNamespace\u003e] \u003caction\u003e \u003cconnectors\u003e...\n\nDescription: Interact with connectors.\n\nParameters:\n      \u003caction\u003e          Action to perform (pause, resume, restart).\n      \u003cconnectors\u003e...   Connector names separated by space or \"all\" for all connectors.\n\nOptions:\n  -h, --help            Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                        Override namespace defined in config or YAML resources.\n  -v, --verbose         Enable the verbose mode.\n```\n\n- `action`: This option specifies the action to execute, which can be `pause`, `resume`, `restart`\n- `connectors`: This option specifies the list of connector names separated by space or \"all\" for all connectors.\n\nExample(s):\n\n```console\nkafkactl connector pause myConnector\nkafkactl connector resume myConnector\nkafkactl connector restart myConnector\n```\n\n### Delete Records\n\nThe `delete-records` command allows you to delete all records within \"delete\" typed topics.\n\n```console\nUsage: kafkactl delete-records [-hv] [--dry-run] [-n=\u003coptionalNamespace\u003e] \u003ctopic\u003e\n\nDescription: Delete all records within a topic.\n\nParameters:\n      \u003ctopic\u003e     Name of the topic.\n\nOptions:\n      --dry-run   Does not persist resources. Validate only.\n  -h, --help      Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                  Override namespace defined in config or YAML resources.\n  -v, --verbose   Enable the verbose mode.\n```\n\nExample(s):\n\n```console\nkafkactl delete-records myTopic\n```\n\n### Delete\n\nThe `delete` command allows you to delete a resource.\n\nPlease note that the resources are deleted instantly and cannot be recovered once deleted. Any data or access associated\nwith the resource is permanently lost.\n\n```console\nUsage: kafkactl delete [-hv] [--dry-run] [-n=\u003coptionalNamespace\u003e] ([\u003cresourceType\u003e \u003cname\u003e [-V[=\u003cversion\u003e]]] | [[-f=\u003cfile\u003e] [-R]])\nDescription: Delete a resource.\n\nParameters:\n      \u003cresourceType\u003e   Resource type.\n      \u003cresourceName\u003e   Resource name or wildcard matching resource names.\n\nOptions:\n      --dry-run        Does not persist resources. Validate only.\n  -f, --file=\u003cfile\u003e    YAML file or directory containing resources to delete.\n  -h, --help           Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                       Override namespace defined in config or YAML resources.\n  -R, --recursive      Search file recursively.\n  -v, --verbose        Enable the verbose mode.\n  -V, --version=\u003cversion\u003e\n                       Version to delete. Only with schema resource and name parameter.\n      --execute        This option is mandatory to delete resources with wildcard.\n```\n\nExample(s):\n\n```console\nkafkactl delete -f directoryOfResources\nkafkactl delete -f resource.yml\nkafkactl delete topic myTopic\nkafkactl delete topic *-test\nkafkactl delete schema *\nkafkactl delete schema mySchema -V latest\n```\n\n### Diff\n\nThe `diff` command allows you to compare a new YAML descriptor with the current one deployed in Ns4Kafka, allowing you\nto easily identify any differences.\n\n```console\nUsage: kafkactl diff [-hRv] [-f=\u003cfile\u003e] [-n=\u003coptionalNamespace\u003e]\n\nDescription: Get differences between a new resource and a old resource.\n\nOptions:\n  -f, --file=\u003cfile\u003e   YAML file or directory containing resources to compare.\n  -h, --help          Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                      Override namespace defined in config or YAML resources.\n  -R, --recursive     Search file recursively.\n  -v, --verbose       Enable the verbose mode.\n```\n\nExample(s):\n\n```console\nkafkactl diff -f resource.yml\n```\n\n### Get\n\nThe `get` command allows you to retrieve information about one or multiple resources.\n\n```console\nUsage: kafkactl get [-hv] [-n=\u003coptionalNamespace\u003e] [-o=\u003coutput\u003e] \u003cresourceType\u003e [\u003cresourceName\u003e]\n\nDescription: Get resources by resource type for the current namespace.\n\nParameters:\n      \u003cresourceType\u003e      Resource type or 'all' to display resources of all types.\n      [\u003cresourceName\u003e]    Resource name or wildcard matching resource names.\n\nOptions:\n  -h, --help              Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                          Override namespace defined in config or YAML\n                            resources.\n  -o, --output=\u003coutput\u003e   Output format (yaml, yml, table).\n      --search[=\u003cString=String\u003e[,\u003cString=String\u003e...]]\n                          Search resources based on parameters.\n  -v, --verbose           Enable the verbose mode.\n```\n\n- `resourceType`: This option specifies one of the managed resources: `topic`, `connector`, `acl`, `schema`, `stream`\n  or `all` to fetch all the resources.\n- `resourceName`: This option specifies the name of the resource to consult.\n\nExample(s):\n\n```console\nkafkactl get all\nkafkactl get topic\nkafkactl get topic myTopic\nkafkactl get topic myTopic-test?\nkafkactl get connector *jdbc-*\nkafkactl get schema mySchema\nkafkactl get schema *-value\n```\n\nAdministrators can filter namespace resources using the `--search` option, which currently only supports search on\nnamespace topics.\n\nExample(s):\n\n```console\nkafkactl get namespace --search topic=myTopic\n```\n\n### Import\n\nThe `import` command allows you to import unsynchronized resources between Ns4Kafka and the Kafka broker/Kafka Connect\ncluster.\n\n```console\nUsage: kafkactl import [-hv] [--dry-run] [-n=\u003coptionalNamespace\u003e] \u003cresourceType\u003e [\u003cresourceName\u003e]\n\nDescription: Import non-synchronized resources.\n\nParameters:\n      \u003cresourceType\u003e     Resource type.\n      [\u003cresourceName\u003e]   Resource name or wildcard matching resource names.\n\nOptions:\n      --dry-run          Does not persist resources. Validate only.\n  -h, --help             Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                         Override namespace defined in config or YAML resources.\n  -v, --verbose          Enable the verbose mode.\n```\n\n- `resourceType`: This option specifies the type of resource that you want to import, which can be either `topics`\n  or `connectors`.\n- `resourceName`: This option specifies the name of the resource to import.\n\nExample(s):\n\n```console\nkafkactl import topics\nkafkactl import topic myTopicName\nkafkactl import connectors\nkafkactl import connector myConnectorName\n```\n\n### Reset Offsets\n\nThe `reset-offsets` command allows you to reset the offsets of consumer groups and topics.\n\n```console\nUsage: kafkactl reset-offsets [-hv] [--dry-run] --group=\u003cgroup\u003e[-n=\u003coptionalNamespace\u003e] (--topic=\u003ctopic\u003e | --all-topics) (--to-earliest | --to-latest |\n                        --to-datetime=\u003cdatetime\u003e | --shift-by=\u003cshiftBy\u003e | --by-duration=\u003cduration\u003e | --to-offset=\u003coffset\u003e)\n\nDescription: Reset consumer group offsets.\n\nOptions:\n      --all-topics           All topics.\n      --by-duration=\u003cduration\u003e\n                             Shift offset by a duration format (PnDTnHnMnS).\n      --dry-run              Does not persist resources. Validate only.\n      --group=\u003cgroup\u003e        Consumer group name.\n  -h, --help                 Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                             Override namespace defined in config or YAML resources.\n      --shift-by=\u003cshiftBy\u003e   Shift offset by a number. Negative to reprocess or positive to skip.\n      --to-datetime=\u003cdatetime\u003e\n                             Set offset to a specific ISO8601 date time with time zone (yyyy-MM-ddTHH:mm:ssZ).\n      --to-earliest          Set offset to its earliest value (reprocess all).\n      --to-latest            Set offset to its latest value (skip all).\n      --to-offset=\u003coffset\u003e   Set offset to a specific index.\n      --topic=\u003ctopic\u003e        Topic name or topic:partition.\n  -v, --verbose              Enable the verbose mode.\n```\n\n- `--group`: This option specifies one of your consumer group to reset.\n- `--topic/--all-topics`: This option specifies a given topic or all the topics to reset.\n- `method`: This option specifies the reset method, which can\n  be `--to-earliest`, `--to-latest`, `--to-offset`, `--to-datetime`, `--shift-by`.\n\nExample(s):\n\n```console\nkafkactl reset-offsets --group myConsumerGroup --topic myTopic --to-earliest\n```\n\n### Reset Password\n\nThe `reset-password` command allows you to reset the password of a user.\n\n```console\nUsage: kafkactl reset-password [-hv] [--execute] [-n=\u003coptionalNamespace\u003e] [-o=\u003coutput\u003e] \u003cuser\u003e\n\nDescription: Reset a Kafka password.\n\nParameters:\n      \u003cuser\u003e              The user to reset password.\n\nOptions:\n      --execute           This option is mandatory to change the password\n  -h, --help              Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                          Override namespace defined in config or YAML resources.\n  -o, --output=\u003coutput\u003e   Output format (yaml, yml, table).\n  -v, --verbose           Enable the verbose mode.\n```\n\nExample(s):\n\n```console\nkafkactl reset-password myUser\n```\n\n### Schema\n\nThe `schema` command allows you to modify the schema compatibility.\n\n```console\nUsage: kafkactl schema [-hv] [-n=\u003coptionalNamespace\u003e] \u003ccompatibility\u003e \u003csubjects\u003e...\n\nDescription: Interact with schemas.\n\nParameters:\n      \u003ccompatibility\u003e   Compatibility to set (global, backward, backward-transitive, forward, forward-transitive, full, full-transitive, none).\n      \u003csubjects\u003e...     Subject names separated by space.\n\nOptions:\n  -h, --help            Show this help message and exit.\n  -n, --namespace=\u003coptionalNamespace\u003e\n                        Override namespace defined in config or YAML resources.\n  -v, --verbose         Enable the verbose mode.\n```\n\n- `compatibility`: This option specifies the compatibility mode to apply.\n- `subject`: This option specifies the subject to update the compatibility.\n\nExample(s):\n\n```console\nkafkactl schema forward-transitive mySubject-value\n```\n\n## Resources\n\n### User\n\nThis is the list of resources a simple Ns4Kafka user can manage.\n\n#### Topic\n\n```yaml\n---\napiVersion: v1\nkind: Topic\nmetadata:\n  name: myPrefix.topic\nspec:\n  replicationFactor: 3\n  partitions: 3\n  configs:\n    min.insync.replicas: '2'\n    cleanup.policy: delete\n    retention.ms: '60000'\n  tags:\n    - tag1\n    - tag2\n    - tag3\n  description: \"myDescription\"\n```\n\n- The `metadata.name` field must be part of your allowed ACLs. Visit your namespace's ACLs to understand which topics\n  you are allowed to manage.\n- The validation of `spec` properties, and especially `spec.config` properties, depends on the topic validation rules\n  associated with your namespace.\n- `spec.replicationFactor` and `spec.partitions` are immutable and cannot be modified once the topic is created.\n- The `spec.tags` field represents a list of tags associated with the topic within a Confluent Cloud cluster.\n  You can learn more about using tags in\n  the [Confluent Cloud documentation](https://docs.confluent.io/cloud/current/stream-governance/stream-catalog.html).\n\n#### ACL\n\nTo provide access to your topics to another namespace, you can add an Access Control List (ACL) using the following\nexample, where \"daaagbl0\" is your namespace and \"dbbbgbl0\" is the namespace that needs access to your topics:\n\n```yaml\n---\napiVersion: v1\nkind: AccessControlEntry\nmetadata:\n  name: acl-topic-a-b\n  namespace: daaagbl0\nspec:\n  resourceType: TOPIC\n  resource: aaa.\n  resourcePatternType: PREFIXED\n  permission: READ\n  grantedTo: dbbbgbl0\n```\n\nHere are some points to keep in mind:\n\n- `spec.resourceType` can be `TOPIC`, `GROUP`, `CONNECT`, or `CONNECT_CLUSTER`.\n- `spec.resourcePatternType` can be `PREFIXED` or `LITERAL`.\n- `spec.permission` can be `READ` or `WRITE`.\n- `spec.grantedTo` must reference a namespace on the same Kafka cluster as yours.\n- `spec.resource` must reference any “sub-resource” that you own. For example, if you are owner of the prefix “aaa”, you\n  can grant READ or WRITE access to:\n    - the whole prefix: “aaa”\n    - a sub prefix: “aaa_subprefix”\n    - a literal topic name: “aaa_myTopic”\n\n\u003ch4 id=\"connector-2\"\u003eConnector\u003c/h4\u003e\n\n```yaml\n---\napiVersion: v1\nkind: Connector\nmetadata:\n  name: myPrefix.myConnector\nspec:\n  connectCluster: myConnectCluster\n  config:\n    connector.class: myConnectorClass\n    tasks.max: '1'\n    topics: myPrefix.myTopic\n    file: /tmp/output.out\n    consumer.override.sasl.jaas.config: o.a.k.s.s.ScramLoginModule required username=\"\u003cuser\u003e\" password=\"\u003cpassword\u003e\";\n```\n\n- `spec.connectCluster` must refer to one of the Kafka Connect clusters authorized in your namespace. It can also refer\n  to a Kafka Connect cluster that you have self-deployed or have been granted access to.\n- Everything else depend on the connect validation rules associated to your namespace.\n\n\u003ch4 id=\"connect-cluster-2\"\u003eConnect Cluster\u003c/h4\u003e\n\nThe `Connect Cluster` resource declares a Connect cluster that has been self-deployed, so namespaces are autonomous to\ndeploy connectors on it without any Ns4Kafka outage.\n\n```yaml\n---\napiVersion: v1\nkind: ConnectCluster\nmetadata:\n  name: myPrefix.myConnectCluster\nspec:\n  url: http://localhost:8083\n  username: myUsername\n  password: myPassword\n  aes256Key: myKey\n  aes256Salt: mySalt\n  aes256Format: \"%s\"\n```\n\n- `metadata.name` should not collide with the name of a Connect cluster declared in the Ns4Kafka configuration.\n  Otherwise, an error message will be thrown.\n- `metadata.aes256Key` and `metadata.aes256Salt` are the AES256 key and salt used to encrypt connector-sensitive\n  configuration, if needed. You can use\n  the [AES256 Config Provider](https://github.com/michelin/michelin-connect-plugins/blob/main/doc/config-providers/aes256-config-provider.md)\n  to encrypt connector-sensitive configuration (such as username, password, etc.) at rest. This provides the ability for\n  your Connect cluster to decrypt it by itself.\n- `metadata.aes256Format` is the AES256 format used to display encrypted connector-sensitive configuration, if needed.\n  The default format is \"${aes256:%s}\".\n- Owners of Connect clusters can authorize other namespaces to deploy connectors on their Connect clusters by giving an\n  ACL with the WRITE permission to the grantees.\n\n#### Kafka Streams\n\nThe `Kafka Streams` resource grants the necessary ACLs for your Kafka Streams to work properly if you have internal\ntopics.\n\n```yaml\n---\napiVersion: v1\nkind: KafkaStream\nmetadata:\n  name: myKafkaStreamsApplicationId\n```\n\n- `metadata.name` must correspond to your Kafka Streams `application.id`.\n\n\u003ch4 id=\"schema-2\"\u003eSchema\u003c/h4\u003e\n\nThe `Schema` resource allows you to declare subjects for your schemas. You can either reference a local `avsc` file\nwith `spec.schemaFile`, or define your schema directly inline with `spec.schema`.\n\n##### Local file\n\n```yml\n---\napiVersion: v1\nkind: Schema\nmetadata:\n  name: myPrefix.topic-value # your subject name\nspec:\n  schemaFile: schemas/topic.avsc # relative to kafkactl binary\n```\n\n##### Inline\n\n```yml\n---\napiVersion: v1\nkind: Schema\nmetadata:\n  name: myPrefix.topic-value\nspec:\n  schema: |\n    {\n      \"type\": \"long\"\n    }\n```\n\n##### Reference\n\nIf your schema references a type that is already stored in the Schema Registry, you can do the following:\n\n```yml\n---\napiVersion: v1\nkind: Schema\nmetadata:\n  name: myPrefix.topic-value\nspec:\n  schema: |\n    {\n      \"type\": \"record\",\n      \"namespace\": \"com.schema.avro\",\n      \"name\": \"Client\",\n      \"fields\": [\n        {\n          \"name\": \"name\",\n          \"type\": \"string\"\n        },\n        {\n          \"name\": \"address\",\n          \"type\": \"com.schema.avro.Address\"\n        }\n      ]\n    }\n  references:\n    - name: com.schema.avro.Address\n      subject: commons.address-value\n      version: 1\n```\n\nThis example assumes that a subject named `commons.address-value` with version 1 is already available in the Schema\nRegistry.\n\nYour schema's ACLs are the same as your topic's ACLs. If you are allowed to create a topic `myPrefix.topic`, then you\nare automatically allowed to create the subjects `myPrefix.topic-key` and `myPrefix.topic-value`.\n\n### Administrator\n\nHere is the list of resources a Ns4Kafka administrator can manage.\n\n#### Namespace\n\nThe `Namespace` resource is the core of Ns4Kafka.\n\n```yml\n---\napiVersion: v1\nkind: Namespace\nmetadata:\n  name: myNamespace\n  cluster: myCluster\n  labels:\n    contacts: namespace.owner@example.com\nspec:\n  kafkaUser: kafkaServiceAccount\n  protectionEnabled: false\n  transactionsEnabled: true\n  connectClusters:\n    - myConnectCluster\n  topicValidator:\n    validationConstraints:\n      partitions:\n        validation-type: Range\n        min: 1\n        max: 6\n      replication.factor:\n        validation-type: Range\n        min: 3\n        max: 3\n      min.insync.replicas:\n        validation-type: Range\n        min: 2\n        max: 2\n      retention.ms:\n        optional: true\n        validation-type: Range\n        min: 60000\n        max: 604800000\n      cleanup.policy:\n        validation-type: ValidList\n        validStrings:\n          - delete\n          - compact\n  connectValidator:\n    validationConstraints:\n      key.converter:\n        validation-type: NonEmptyString\n      value.converter:\n        validation-type: NonEmptyString\n      connector.class:\n        validation-type: ValidString\n        validStrings:\n          - io.confluent.connect.jdbc.JdbcSinkConnector\n          - io.confluent.connect.jdbc.JdbcSourceConnector\n    sourceValidationConstraints:\n      producer.override.sasl.jaas.config:\n        validation-type: NonEmptyString\n    sinkValidationConstraints:\n      consumer.override.sasl.jaas.config:\n        validation-type: NonEmptyString\n    classValidationConstraints:\n      io.confluent.connect.jdbc.JdbcSourceConnector:\n        db.timezone:\n          validation-type: NonEmptyString\n      io.confluent.connect.jdbc.JdbcSinkConnector:\n        db.timezone:\n          validation-type: NonEmptyString\n```\n\n- `metadata.cluster` is the name of the Kafka cluster. It should refer to a cluster defined in the Ns4Kafka\n  configuration.\n- `spec.kafkaUser` is the Kafka principal. It should refer to an Account ID. It will be used to create ACLs on this\n  service account.\n- `spec.protectionEnabled` is a boolean that defines whether the namespace is protected. Protected namespaces cannot grant public ACLs or grant ACLs to public namespaces. \n  Protected namespaces can only consume or produce records in protected namespace resources. This field is optional; by default, namespaces are public.\n- `spec.transactionsEnabled` is a boolean that defines whether transactions are allowed for the namespace. If enabled, the transactional ID ACLs will be created based on GROUP ACLs.\n  This field is optional; by default, transactions are enabled.\n- `spec.connectClusters` is a list of Kafka Connect clusters. It should refer to a Kafka Connect cluster declared in the\n  Ns4Kafka configuration.\n- `spec.topicValidator` is a list of constraints for topics.\n- `spec.connectValidator` is a list of constraints for connectors.\n\n##### Validation Constraints\n\nValidations constraints enforce rules on the configuration of topics using `topicValidator` and connectors using\n`connectValidator`.\n\nFor topics, the following constraints are available:\n\n- `validationConstraints` applies to all topics.\n\nFor connectors, the following constraints are available:\n\n- `validationConstraints` applies to all connectors.\n- `sourceValidationConstraints` applies to source connectors.\n- `sinkValidationConstraints` applies to sink connectors.\n- `classValidationConstraints` applies to connectors of a specific class.\n\nValidation constraints define a list of properties that must adhere to specific rules set by the `validation-type`.\nConstraints can be made optional by setting the `optional` attribute to `true`.\nIf the field is present, it will be validated; otherwise, it can be omitted without causing an error.\n\n###### Range\n\nEnsures that the property falls within the specified range, defined by optional `min` and `max` limits.\n\n```yml\ntopicValidator:\n  validationConstraints:\n    partitions:\n      validation-type: Range\n      min: 1\n      max: 6\n```\n\n###### ValidList\n\nEnsures that the property is a comma-separated list of strings, where each string must be part of the predefined\n`validStrings` list.\n\n```yml\ntopicValidator:\n  validationConstraints:\n    cleanup.policy:\n      validation-type: ValidList\n      validStrings:\n        - delete\n        - compact\n```\n\n###### ValidString\n\nEnsures that the property is a string that matches one of the values specified in the `validStrings` list.\n\n```yml\nconnectValidator:\n  validationConstraints:\n    connector.class:\n      validation-type: ValidString\n      validStrings:\n        - io.confluent.connect.jdbc.JdbcSinkConnector\n        - io.confluent.connect.jdbc.JdbcSourceConnector\n```\n\n###### NonEmptyString\n\nEnsures that the property is a non-empty string.\n\n```yml\nconnectValidator:\n  validationConstraints:\n    key.converter:\n      validation-type: NonEmptyString\n```\n\n###### ContainsList\n\nEnsures that the property is a comma-separated list of strings that contains all the items defined in the `mandatoryStrings` list.\n\n```yml\nconnectValidator:\n  validationConstraints:\n    topic.blacklist:\n      validation-type: ContainsList\n      mandatoryStrings:\n        - ns1.topic1\n        - ns1.topic2\n```\n\n###### CompositeValidator\n\nEnsures that the property satisfies multiple validation rules. The property is valid only if it meets all specified\nvalidators.\n\n```yml\nconnectValidator:\n  validationConstraints:\n    connector.class:\n      validation-type: CompositeValidator\n      validators:\n        - validation-type: NonEmptyString\n        - validation-type: ValidString\n          validStrings:\n            - io.confluent.connect.jdbc.JdbcSinkConnector\n            - io.confluent.connect.jdbc.JdbcSourceConnector\n```\n\n#### ACL Owner\n\nACLs with owner permission can only be deployed by administrators.\n\n```yml\n---\napiVersion: v1\nkind: AccessControlEntry\nmetadata:\n  name: acl-topic-myNamespace\n  namespace: myNamespace\nspec:\n  resourceType: TOPIC\n  resource: myPrefix.\n  resourcePatternType: PREFIXED\n  permission: OWNER\n  grantedTo: myNamespace\n```\n\n- With this ACL, the namespace \"myNamespace\" will be the owner of topics prefixed by \"myPrefix.\". No one else is able to\n  modify these resources.\n- `resourceType` can be `topic`, `connect`, `connect_cluster` or `group`.\n\n#### Role Binding\n\nThe `Role Binding` resource links a namespace to a project team.\n\n```yaml\n---\napiVersion: v1\nkind: RoleBinding\nmetadata:\n  name: rb-myNamespace\n  namespace: myNamespace\nspec:\n  role:\n    resourceTypes:\n      - schemas\n      - schemas/config\n      - topics\n      - topics/import\n      - topics/delete-records\n      - connectors\n      - connectors/import\n      - connectors/change-state\n      - connect-clusters\n      - connect-clusters/vaults\n      - acls\n      - consumer-groups/reset\n      - streams\n    verbs:\n      - GET\n      - POST\n      - PUT\n      - DELETE\n  subject:\n    subjectType: GROUP\n    subjectName: myGitLabGroup\n---\napiVersion: v1\nkind: RoleBinding\nmetadata:\n  name: rb2-myNamespace\n  namespace: myNamespace\nspec:\n  role:\n    resourceTypes:\n      - quota\n    verbs:\n      - GET\n  subject:\n    subjectType: GROUP\n    subjectName: myGitLabGroup\n```\n\n- With this role binding, members of the group \"myGitLabGroup\" can use Ns4Kafka to manage topics starting with \"\n  myPrefix.\" on the \"myCluster\" Kafka cluster.\n\n#### Quota\n\nIt is possible to define quotas on a namespace.\n\n```yml\napiVersion: v1\nkind: ResourceQuota\nmetadata:\n  name: quota-myNamespace\n  namespace: myNamespace\nspec:\n  count/topics: 10\n  count/partitions: 60\n  count/connectors: 5\n  disk/topics: 500MiB\n  user/consumer_byte_rate: 204800\n  user/producer_byte_rate: 204800\n```\n\n- `count/topics` is the maximum number of deployable topics.\n- `count/partitions` is the maximum number of deployable partitions.\n- `count/connectors` is the maximum number of deployable connectors.\n- `disk/topics** is the maximum size of all topics. It is computed from the sum of _retention.bytes_ * _number of\n  partitions_ of all topics.\n  Unit of measure accepted is byte (B), kibibyte (KiB), mebibyte (MiB), gibibyte (GiB).\n- `user/consumer_byte_rate` is the consumer network bandwith quota before throttling. Expressed in bytes/sec.\n- `user/producer_byte_rate` is the producer network bandwith quota before throttling. Expressed in bytes/sec.\n\n## CI/CD\n\nKafkactl can be easily integrated into a CI/CD pipeline using\nthe [Docker images](https://hub.docker.com/repository/docker/michelin/kafkactl) available on Docker Hub.\n\nHere are two examples of how you can use the Kafkactl Docker images in your pipeline, one for\nthe `kafkactl:\u003cversion\u003e-native` image and one for the kafkactl:\u003cversion\u003e image:\n\n```yaml\nkafkactl:\n  stage: kafkactl\n  image:\n    name: michelin/kafkactl:\u003cversion\u003e-native\n    entrypoint: [ '/bin/sh', '-c' ]\n  before_script:\n    - export KAFKACTL_CURRENT_NAMESPACE=test\n    - export KAFKACTL_API=https://ns4kafka-dev-api.domain.com\n    - export KAFKACTL_USER_TOKEN=${GITLAB_TOKEN}\n  script:\n    - kafkactl get all\n```\n\n```yaml\nkafkactl:\n  stage: kafkactl\n  image:\n    name: michelin/kafkactl:\u003cversion\u003e\n    entrypoint: [ '/bin/sh', '-c' ]\n  before_script:\n    - export KAFKACTL_CURRENT_NAMESPACE=test\n    - export KAFKACTL_API=https://ns4kafka-dev-api.domain.com\n    - export KAFKACTL_USER_TOKEN=${GITLAB_TOKEN}\n  script:\n    - java -jar /home/app/application.jar get all\n```\n\n- `KAFKACTL_CURRENT_NAMESPACE` specifies the namespace to use.\n- `KAFKACTL_API` is the URL of Ns4Kafka in which to deploy.\n- `KAFKACTL_USER_TOKEN` contains the GitLab token.\n\n## Contribution\n\nWe welcome contributions from the community! Before you get started, please take a look at\nour [contribution guide](https://github.com/michelin/kafkactl/blob/main/CONTRIBUTING.md) to learn about our guidelines\nand best practices. We appreciate your help in making Kafkactl a better tool for everyone.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichelin%2Fkafkactl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmichelin%2Fkafkactl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichelin%2Fkafkactl/lists"}