Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/scholzj/kafka-x509-config-provider
Apache Kafka configuration provider for automatically converting X509 certificates to PKCS12 truststores or keystores
https://github.com/scholzj/kafka-x509-config-provider
config-provider kafka security tls
Last synced: 8 days ago
JSON representation
Apache Kafka configuration provider for automatically converting X509 certificates to PKCS12 truststores or keystores
- Host: GitHub
- URL: https://github.com/scholzj/kafka-x509-config-provider
- Owner: scholzj
- License: apache-2.0
- Created: 2019-08-11T19:12:25.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2022-08-20T18:57:41.000Z (about 2 years ago)
- Last Synced: 2023-05-25T23:26:53.711Z (over 1 year ago)
- Topics: config-provider, kafka, security, tls
- Language: Java
- Homepage:
- Size: 49.8 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# KeyStore and TrustStore Configuration Providers for Apache Kafka
`X509KeystoreConfigProvider` and `X509TruststoreConfigProvider` are [Apache Kafka®](https://kafka.apache.org) configuration providers which can be used to generate a PKCS12 keystore or truststore from PEM files with 509 certificates _on the fly_.
The PKCS12 files will be generated in a temporary directory and the path to them will be passed to Kafka as the output from the provider.
The providers currently supports unencrypted RSA Private keys and public keys.
They are passed to the provider as file paths.
The certificate files should be in the text format with the markers and look like this:_Public key example_
```
-----BEGIN CERTIFICATE-----
...
...
...
-----END CERTIFICATE-----
```_Private key example_
```
-----BEGIN PRIVATE KEY-----
...
...
...
-----END PRIVATE KEY-----
```## Installation
You can download the archives with the binaries on GitHub release page.
You can download it, unpack it and copy the JAR file into the classpath of your Apache Kafka installation.
If you want to use it with Apache Kafka clients, you can also get the project directly from [Maven repositories](https://mvnrepository.com/artifact/cz.scholz/kafka-x509-config-provider):```xml
cz.scholz
kafka-x509-config-provider
0.1.0
```## Configuration
First, you need to initialize the providers:
```properties
config.providers=truststore,keystore
config.providers.keystore.class=cz.scholz.kafka.x509configprovider.X509KeystoreConfigProvider
config.providers.truststore.class=cz.scholz.kafka.x509configprovider.X509TruststoreConfigProvider
```And then you can use them:
```properties
ssl.keystore.type=PKCS12
ssl.keystore.location=${keystore:/my-path-to-certs/user.key:/my-path-to-certs/user.crt}
ssl.keystore.password=
ssl.key.password=
ssl.truststore.type=PKCS12
ssl.truststore.location=${truststore:/my-path-to-certs/ca.crt}
ssl.truststore.password=
```You should always configure the format to `PKCS12` and the passwords to empty strings.
When Kafka applies the configuration providers, you should see a configuration like this in the Kafka logs:```
2022-08-20 15:41:30,575 INFO [echo-sink-connector|task-0] ConsumerConfig values:
ssl.key.password = [hidden]
ssl.keystore.location = /tmp/cz.scholz.kafka.x509configprovider.X509KeystoreConfigProvider18094880232725639415.p12
ssl.keystore.password = [hidden]
ssl.keystore.type = PKCS12
ssl.truststore.location = /tmp/cz.scholz.kafka.x509configprovider.X509TruststoreConfigProvider7460901468812493428.p12
ssl.truststore.password = [hidden]
ssl.truststore.type = PKCS12
```## Use cases
Apache Kafka can use PEM files directly.
So in most cases, you do not need to use this configuration provider, and you can use the PEM files directly.
But there are some use-cases when this is not possible.When configuring Kafka Connect connectors, you often need to configure the TLS configuration not only for connecting to the Kafka brokers.
You usually need also to configure TLS for the connection to other system which your connector integrates with - from where it read the data or where it forwards the data from Apache Kafka.
This is typically done using some third party library which might not support using PEM files directly.
Sometimes, they support only the JKS or PKCS12 stores supported by Java by default.
In such case, you can use these config providers to create the keystore or truststore automaticallyKafka Connect also allows you to override the configuration of the consumers or producers created for given connector.
Overriding the configs can be done in the connector configuration.
But when the underlying Connect cluster uses PKCS12 or JKS stores, it is not always easy to override the configuration from the connector with PEM files.
You can again use these configuration providers to dynamically create the PKCS12 stores and use them to override the default settings.