https://github.com/ksingh7/amq_streams_ocp
AMQ_Streams_OCP 4.x
https://github.com/ksingh7/amq_streams_ocp
Last synced: 3 months ago
JSON representation
AMQ_Streams_OCP 4.x
- Host: GitHub
- URL: https://github.com/ksingh7/amq_streams_ocp
- Owner: ksingh7
- Created: 2019-09-03T19:47:01.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2019-09-08T18:08:05.000Z (over 5 years ago)
- Last Synced: 2025-01-08T20:15:37.445Z (4 months ago)
- Language: Shell
- Size: 37.1 KB
- Stars: 1
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Red Hat AMQ Streams on OCP 4.x
This directory contains instructions to
- Install RHT AMQ Streams (Apache Kafka) 1.2.0 on OpenShift Container Platform 4.x
- Deploy Kafka / Zookeeper /
- Test and validate sample consumers and applications
- Benchmark Apache Kafka and gather performance results
## Deploying the Cluster Operator to OpenShift
```
oc whoami
oc new-project amq
scp amq-streams-1.2.0-ocp-install-examples.zip [email protected]:/home/karasing-redhat.com
unzip amq-streams-1.2.0-ocp-install-examples.zip
```
- MacOS
```
sed -i '' 's/namespace: .*/namespace: amq/' install/cluster-operator/*RoleBinding*.yaml
```
- For Linux
```
sed -i 's/namespace: .*/namespace: amq/' install/cluster-operator/*RoleBinding*.yaml
```
```
oc apply -f install/cluster-operator -n amq
oc apply -f examples/templates/cluster-operator -n amq
watch oc get all
```
```
[karasing-redhat.com@clientvm 0 ~]$ oc get all
NAME READY STATUS RESTARTS AGE
pod/strimzi-cluster-operator-76bcf9bc76-k24bx 1/1 Running 0 5m21s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/strimzi-cluster-operator 1/1 1 1 5m21s
NAME DESIRED CURRENT READY AGE
replicaset.apps/strimzi-cluster-operator-76bcf9bc76 1 1 1 5m21s
[karasing-redhat.com@clientvm 0 ~]$
```
## Clone Repository
```
git clone https://github.com/ksingh7/AMQ_Streams_OCP.git ; cd AMQ_Streams_OCP
```
## Deploying the Kafka Cluster to OpenShift
```
oc apply -f 01-kafka-ephemeral.yaml
```
```
[karasing-redhat.com@clientvm 130 ~/AMQ_Streams_OCP master ⭑|✔]$ oc get all
NAME READY STATUS RESTARTS AGE
pod/my-cluster-entity-operator-5bbcbb859c-6sp5l 3/3 Running 0 2m19s
pod/my-cluster-kafka-0 2/2 Running 0 2m54s
pod/my-cluster-kafka-1 2/2 Running 0 2m54s
pod/my-cluster-kafka-2 2/2 Running 0 2m54s
pod/my-cluster-zookeeper-0 2/2 Running 0 4m58s
pod/my-cluster-zookeeper-1 2/2 Running 0 4m58s
pod/my-cluster-zookeeper-2 2/2 Running 0 4m58s
pod/strimzi-cluster-operator-76bcf9bc76-k24bx 1/1 Running 0 12m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/my-cluster-kafka-bootstrap ClusterIP 172.30.40.218 9091/TCP,9092/TCP,9093/TCP 2m55s
service/my-cluster-kafka-brokers ClusterIP None 9091/TCP,9092/TCP,9093/TCP 2m55s
service/my-cluster-zookeeper-client ClusterIP 172.30.33.211 2181/TCP 4m59s
service/my-cluster-zookeeper-nodes ClusterIP None 2181/TCP,2888/TCP,3888/TCP 4m59s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/my-cluster-entity-operator 1/1 1 1 2m19s
deployment.apps/strimzi-cluster-operator 1/1 1 1 12m
NAME DESIRED CURRENT READY AGE
replicaset.apps/my-cluster-entity-operator-5bbcbb859c 1 1 1 2m19s
replicaset.apps/strimzi-cluster-operator-76bcf9bc76 1 1 1 12m
NAME READY AGE
statefulset.apps/my-cluster-kafka 3/3 2m54s
statefulset.apps/my-cluster-zookeeper 3/3 4m58s
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$
```
## Setting up Prometheus and Grafana Monitoring
```
oc apply -f 06-prometheus.yaml
oc apply -f 07-grafana.yaml
oc get route
```
- Add Grafana datasource, edit ``08-grafana-datasource.sh`` with the correct grafana endpoint
```
sh 08-grafana-datasource.sh
```
## Creating a Kafka Topic
```
oc apply -f 02-kafka-topic.yaml
```
```
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$ oc get kt
No resources found.
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$ oc apply -f 02-kafka-topic.yaml
kafkatopic.kafka.strimzi.io/my-topic created
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$ oc get kt
NAME PARTITIONS REPLICATION FACTOR
my-topic 1 1
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$
```
## Creating a Kafka User
```
oc apply -f 03-kafka-user.yaml
```
```
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$ oc get kafkauser
No resources found.
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$ oc apply -f 03-kafka-user.yaml
kafkauser.kafka.strimzi.io/my-user created
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$ oc get kafkauser
NAME AUTHENTICATION AUTHORIZATION
my-user tls simple
[karasing-redhat.com@clientvm 0 ~/AMQ_Streams_OCP master ⭑|✔]$
```
## Create a sample Kafka Producer and Consumer Application
- Kafka Producer
```
oc apply -f 04-hello-world-producer.yaml
```
```
oc logs -f hello-world-producer-b8987b5d-b9d5q
```
- Kafka Consumer
```
oc apply -f 05-hello-world-consumer.yaml
```
```
oc logs -f hello-world-consumer-bbfd5d66b-nlz8x
```
## Cleanup sample Kafka Producer / Consumer Apps
```
oc delete -f 04-hello-world-producer.yaml
oc delete -f 05-hello-world-consumer.yaml
```
# Kafka Performance Tests