{"id":18804194,"url":"https://github.com/scholzj/kafka-kubernetes-authenticator","last_synced_at":"2025-04-13T18:36:33.980Z","repository":{"id":54928734,"uuid":"197277844","full_name":"scholzj/kafka-kubernetes-authenticator","owner":"scholzj","description":"Kafka Kubernetes Authenticator and Authorizer","archived":false,"fork":false,"pushed_at":"2023-09-05T22:01:25.000Z","size":55,"stargazers_count":12,"open_issues_count":5,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-27T09:13:52.253Z","etag":null,"topics":["authentication","authenticator","authorization","authorizer","kafka","kubernetes","oauth","rbac"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scholzj.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-16T22:44:13.000Z","updated_at":"2024-10-23T21:00:15.000Z","dependencies_parsed_at":"2022-08-14T06:50:31.378Z","dependency_job_id":null,"html_url":"https://github.com/scholzj/kafka-kubernetes-authenticator","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scholzj%2Fkafka-kubernetes-authenticator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scholzj%2Fkafka-kubernetes-authenticator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scholzj%2Fkafka-kubernetes-authenticator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scholzj%2Fkafka-kubernetes-authenticator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scholzj","download_url":"https://codeload.github.com/scholzj/kafka-kubernetes-authenticator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248761747,"owners_count":21157607,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["authentication","authenticator","authorization","authorizer","kafka","kubernetes","oauth","rbac"],"created_at":"2024-11-07T22:38:34.546Z","updated_at":"2025-04-13T18:36:33.958Z","avatar_url":"https://github.com/scholzj.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kafka Kubernetes Authenticator and Authorizer\n\nThis project provides Kafka Authenticator and Authorizer which are based on Kubernetes Service Accounts and Kubernetes RBAC.\nIt is using the service account tokens and the SASL OAUTHBEARER authentication mehcnaism provided by Kafka.\n\nYou can watch my demo video on YouTube:\n\n[![Demo of a Kafka Authenticator and Authorizer based on Kubernetes ServiceAccounts and RBAC](http://img.youtube.com/vi/AF4abbVlocc/0.jpg)](http://www.youtube.com/watch?v=AF4abbVlocc \"Demo of a Kafka Authenticator and Authorizer based on Kubernetes ServiceAccounts and RBAC\")\n\n## Building the project\n\nRun `mvn clean install` to build the project.\nThe module subdirectories will contain the binaries which should be used.\n\n## Authenticator\n\nThe authenticator consists of two separate parts:\n* Client part which should be used with Kubernetes clients\n* Server part which should be used in the Kafka brokers\n\n### Clients\n\nAdd the `authenticator-client` module as dependency into your application using the Kafka client libraries (in Java, provided by the Apache Kafka project).\nIf needed, you can find the JAR in `authenticator-client/target` and the required libraries in `authenticator-client/target`.\nThis project is currently not available on Maven Central, so you cannot get it from there.\n\nIn your client you have to configure the use of SASL and the SASL OAUTHBEARER mechanism.\nFirst set the `security.protocol` option to `SASL_PLAINTEXT` (or to `SASL_SSL` if you use SSL)\nNext set the `sasl.mechanism` to `OAUHTBEARER` mechanism.\n\nSASL authentication is implemented using JAAS.\nSo you have to configure the Kafka's OAuthBearer login module.\nYou can do that for example by setting the `sasl.jaas.config` to `org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required`;\nAdditionally you have to configure the login callback provided by this project.\nIn `sasl.login.callback.handler.class` set `io.strimzi.kafka.kubernetes.authenticator.KubernetesTokenLoginCallbackHandler`.\n\nYour complete configuration might look something like this:\n\n```java\nProperties props = new Properties();\nprops.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, \"localhost:9092\");\nprops.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, \"org.apache.kafka.common.serialization.StringSerializer\");\nprops.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, \"org.apache.kafka.common.serialization.StringSerializer\");\n\nprops.put(\"sasl.jaas.config\", \"org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required;\");\nprops.put(\"security.protocol\",\"SASL_PLAINTEXT\");\nprops.put(\"sasl.mechanism\",\"OAUTHBEARER\");\nprops.put(\"sasl.login.callback.handler.class\",\"io.strimzi.kafka.kubernetes.authenticator.KubernetesTokenLoginCallbackHandler\");\n```\n\nAll you need to do next is just open the connection and start sending or receiving messages.\nThe login callback assumes that it will run inside a Kubernetes Pod and will by default try to get the Service Account token from the default location which is the file `/var/run/secrets/kubernetes.io/serviceaccount/token` inside the Pod.\nYou can use the environment variable `OAUTHBEARER_TOKEN_FILE` to have the file read from a different file or the environment variable `OAUTHBEARER_TOKEN` to pass the token directly.\nYou can also pass the token in the `token` option in the JAAS file configuration like this:\n\n```java\nprops.put(\"sasl.jaas.config\", \"org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required token=XXX.YYY.ZZZ;\");\n```  \n\n### Brokers\n\nCopy the JAR from `authenticator-server/target` and the required libraries in `authenticator-server/target` into the `libs` directory of the broker.\nIn your server properties file, configure your listener as usually to use the SASL OAUTHBEARER mechanism.\nSet the callback to the class provided by this project: \n\n```properties\nsasl.server.callback.handler.class=io.strimzi.kafka.kubernetes.authenticator.KubernetesTokenValidatorCallbackHandler\n```\n\nOnce the client connects to the broker using the OAUTHBEARER protocol it will pass the broker its Service Account token.\nThe broker callback will take the token and use the Kubernetes Token Review API to validate that the token is valid.\nIt expects the broker to run either inside a Kubernetes Pod or to have a configured Kubernets context (either in the default path, configured using the `KUBECONFIG` environment variable etc.).\n\nAn example of the principal under which will the accounts be authenticated is `system:serviceaccount:mynamespace:mysa`.\n\n#### Required RBAC rights\n\nThe Authenticator requires the right to post the Token Review requests.\nYou have to give it a role similar to this:\n\n```yaml\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n  name: token-review-role\nrules:\n- apiGroups:\n  - authentication.k8s.io\n  resources:\n  - tokenreviews\n  verbs:\n  - create\n```\n\n## Authorizer\n\nThe Authorizer provided by this project is authorizing the Kafka clients based on the Kubernetes Service Accounts and Kubernetes RBAC.\nIt uses the `Kafka` and `KafkaTopic` resources from the [Strimzi](https://strimzi.io) project to authorize on.\n\nSince Kafka allows only one Authorizer to exist, if will check the format of the user name.\nAnd if it matches Kubernetes service account, it will authorize it based on RBAC.\nIf it doesn't, it will fall  back to the `SimpleAclAuthorizer` class shipped with Apache Kafka.\nThe authorizer also respects the `super.users` field to define users who will be allowed everything.\n\nTo enable the authorizer, copy the Authorizer JAR from `authorizer/target` to the `libs` directoty on your Kafka broker and configure it:\n\n```properties\nauthorizer.class.name=io.strimzi.kafka.kubernetes.authorizer.KubernetesAuthorizer\n``` \n\nThe following table shows how the RBAC rights map against the Kafka ACLs.\nThe match here is not perfect. \nBut might work for most regular consumers and producers.\n\n|                 | Cluster                 | Topic                        | Group | DelegationToken | TransactionalId |\n| --------------- | ----------------------- | ---------------------------- | ----- | --------------- | --------------- |\n| Alter           | Patch `Kafka` resource  | Patch `KafkaTopic` resource  | n/a   | n/a             | n/a             |\n| AlterConfigs    | Patch `Kafka` resource  | Patch `KafkaTopic` resource  | n/a   | n/a             | n/a             |\n| ClusterAction   | Patch `Kafka` resource  | Patch `KafkaTopic` resource  | n/a   | n/a             | n/a             |\n| Create          | Create `Kafka` resource | Create `KafkaTopic` resource | n/a   | n/a             | n/a             |\n| Describe        | List `Kafka` resource   | List `KafkaTopic` resource   | n/a   | n/a             | n/a             |\n| DescribeConfigs | List `Kafka` resource   | List `KafkaTopic` resource   | n/a   | n/a             | n/a             |\n| IdempotentWrite | Update `Kafka` resource | Update `KafkaTopic` resource | n/a   | n/a             | n/a             |\n| Read            | Get `Kafka` resource    | Get `KafkaTopic` resource    | n/a   | n/a             | n/a             |\n| Write           | Update `Kafka` resource | Update `KafkaTopic` resource | n/a   | n/a             | n/a             |\n\nWhen the service account has the given right, it will be allwoed the action.\nOtherwise it will be denied.\n\nThere is no resource for consumer groups or transactional IDs.\nThe authorizer currently handles them in a way that if they start with the ID of the user, they will be always allowed.\nAnd example of the service account based username is `system:serviceaccount:mynamespace:mysa`.\n\n**The authorizer is currently only experimental and does not caching of the authorization results.\nTherefore it might have significant performance impact on the Kubernetes and Kafka clusters.**\n\n#### Required RBAC rights\n\nThe Authorizer requires the RBAC rights to post Subject Access Review API calls.\nYou have to give the broker the rights similar to this:\n\n```yaml\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRole\nmetadata:\n  name: subject-access-review\nrules:\n- apiGroups:\n  - authorization.k8s.io\n  resources:\n  - subjectaccessreviews\n  verbs:\n  - create\n```\n\n## Trying it with Strimzi\n\nYou can try this with specially modified Strimzi images.\nFirst, create a namespace `myproject` which will be used for the demo:\n\n```sh\nkubectl create ns myproject\n```\n\nNext deploy the modified Strimzi Kafka Operator:\n\n```sh\nkubectl apply -f examples/strimzi -n myproject\n```\n\nAnd deploy the Kafka cluster:\n\n```sh\nkubectl apply -f examples/kafka-cluster.yaml -n myproject\n```\n\nAfterwards you can deplyo two sets of clients.\n`allowed.yaml` contain simple producer and consumer which are authorized and should work properly.\n\n```sh\nkubectl apply -f examples/allowed.yaml -n myproject\n```\n\n`denied.yaml` contain two clients which will be failing.\nOne is configured with a token from a different cluster and should not pass authentication.\nThe second is using a valid service account to authenticate, but should be unauthorized.\n\n```sh\nkubectl apply -f examples/denied.yaml -n myproject\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscholzj%2Fkafka-kubernetes-authenticator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscholzj%2Fkafka-kubernetes-authenticator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscholzj%2Fkafka-kubernetes-authenticator/lists"}