https://github.com/gustavomcarmo/kafka-proxy-test
kafka-proxy test project
https://github.com/gustavomcarmo/kafka-proxy-test
kafka kafka-proxy ldap ldap-authentication tls-termination
Last synced: 2 months ago
JSON representation
kafka-proxy test project
- Host: GitHub
- URL: https://github.com/gustavomcarmo/kafka-proxy-test
- Owner: gustavomcarmo
- License: mit
- Created: 2020-06-20T09:49:56.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2020-08-25T15:53:14.000Z (over 4 years ago)
- Last Synced: 2025-01-18T02:44:02.354Z (4 months ago)
- Topics: kafka, kafka-proxy, ldap, ldap-authentication, tls-termination
- Language: Shell
- Homepage:
- Size: 28.3 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# kafka-proxy-test
[](https://opensource.org/licenses/MIT) [](https://travis-ci.org/gustavomcarmo/kafka-proxy-test)
[kafka-proxy](https://github.com/grepplabs/kafka-proxy) test project.
## Requirements
For setting up the local Kafka environment (localhost):
- [Docker](https://www.docker.com)
- [Docker Compose](https://docs.docker.com/compose)
For testing:
- [JDK 1.8](https://openjdk.java.net)
- Java client embedded in [Kafka](https://kafka.apache.org)
For bootstraping a local Kafka VM (kafka.example.org):
- [Virtual Box](https://www.virtualbox.org)
- [Vagrant](https://www.vagrantup.com)
- [Ansible](https://www.ansible.com)
## Tests
`curl -Ls https://mirrors.up.pt/pub/apache/kafka/2.5.0/kafka_2.12-2.5.0.tgz | tar xz`
### No auth test
Comment in [docker-compose.yml](docker-compose.yml):
```yml
# - --auth-local-enable
# - --auth-local-command=/auth-ldap
# - --auth-local-param=--start-tls=false
# - --auth-local-param=--url=ldap://openldap:389
# - --auth-local-param=--bind-dn=cn=admin,dc=example,dc=org
# - --auth-local-param=--bind-passwd=admin
# - --auth-local-param=--user-search-base=ou=people,dc=example,dc=org
# - --auth-local-param=--user-filter=(&(objectClass=inetOrgPerson)(uid=%u)(memberOf=cn=kafka-users,ou=groups,dc=example,dc=org))
```
Then:
`docker-compose up -d`
To list the topics:
`kafka_2.12-2.5.0/bin/kafka-topics.sh --list --bootstrap-server localhost:32400,localhost:32401,localhost:32402`
To create the test topic:
`kafka_2.12-2.5.0/bin/kafka-topics.sh --create --bootstrap-server localhost:32400,localhost:32401,localhost:32402 --replication-factor 3 --partitions 1 --topic test`
To produce a single message:
`echo "Hello, World!" | kafka_2.12-2.5.0/bin/kafka-console-producer.sh --bootstrap-server localhost:32400,localhost:32401,localhost:32402 --topic test`
To consume the message:
`kafka_2.12-2.5.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:32400,localhost:32401,localhost:32402 --topic test --from-beginning --max-messages 1`
### LDAP auth test
Uncomment in [docker-compose.yml](docker-compose.yml):
```yml
- --auth-local-enable
- --auth-local-command=/auth-ldap
- --auth-local-param=--start-tls=false
- --auth-local-param=--url=ldap://openldap:389
- --auth-local-param=--bind-dn=cn=admin,dc=example,dc=org
- --auth-local-param=--bind-passwd=admin
- --auth-local-param=--user-search-base=ou=people,dc=example,dc=org
- --auth-local-param=--user-filter=(&(objectClass=inetOrgPerson)(uid=%u)(memberOf=cn=kafka-users,ou=groups,dc=example,dc=org))
```
Then:
`docker-compose up -d`
Set the `KAFKA_OPTS` environment variable:
`export KAFKA_OPTS="-Djava.security.auth.login.config=$(pwd)/jaas.conf"`
To list the topics:
`kafka_2.12-2.5.0/bin/kafka-topics.sh --list --bootstrap-server localhost:32400,localhost:32401,localhost:32402 --command-config client-sasl.properties`
To create the test topic:
`kafka_2.12-2.5.0/bin/kafka-topics.sh --create --bootstrap-server localhost:32400,localhost:32401,localhost:32402 --replication-factor 3 --partitions 1 --topic test --command-config client-sasl.properties`
To produce a single message:
`echo "Hello, World!" | kafka_2.12-2.5.0/bin/kafka-console-producer.sh --broker-list localhost:32400,localhost:32401,localhost:32402 --topic test --producer.config client-sasl.properties`
To consume the message:
`kafka_2.12-2.5.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:32400,localhost:32401,localhost:32402 --topic test --from-beginning --max-messages 1 --consumer.config client-sasl.properties`
:information_source: Alternatively, you can run `./test.sh`.
### LDAP auth/TLS termination test
`vagrant up`
Set the `KAFKA_OPTS` environment variable:
`export KAFKA_OPTS="-Djava.security.auth.login.config=$(pwd)/jaas.conf"`
To list the topics:
`kafka_2.12-2.5.0/bin/kafka-topics.sh --list --bootstrap-server kafka.example.org:32400,kafka.example.org:32401,kafka.example.org:32402 --command-config client-sasl-ssl.properties`
To create the test topic:
`kafka_2.12-2.5.0/bin/kafka-topics.sh --create --bootstrap-server kafka.example.org:32400,kafka.example.org:32401,kafka.example.org:32402 --replication-factor 3 --partitions 1 --topic test --command-config client-sasl-ssl.properties`
To produce a single message:
`echo "Hello, World!" | kafka_2.12-2.5.0/bin/kafka-console-producer.sh --broker-list kafka.example.org:32400,kafka.example.org:32401,kafka.example.org:32402 --topic test --producer.config client-sasl-ssl.properties`
To consume the message:
`kafka_2.12-2.5.0/bin/kafka-console-consumer.sh --bootstrap-server kafka.example.org:32400,kafka.example.org:32401,kafka.example.org:32402 --topic test --from-beginning --max-messages 1 --consumer.config client-sasl-ssl.properties`
:information_source: You can use `192.168.33.10` instead of `kafka.example.org`.