https://github.com/guilhermecamposo/demo_streams-monitoring

Demonstration on how to configure Prometheus/Grafana monitoring for AMQ Streams
https://github.com/guilhermecamposo/demo_streams-monitoring

ansible grafana kafka prometheus

Last synced: 5 days ago
JSON representation

Demonstration on how to configure Prometheus/Grafana monitoring for AMQ Streams

• Host: GitHub
• URL: https://github.com/guilhermecamposo/demo_streams-monitoring
• Owner: GuilhermeCamposo
• Created: 2022-08-19T19:10:00.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-01-04T15:25:30.000Z (11 months ago)
• Last Synced: 2024-01-04T19:31:16.436Z (11 months ago)
• Topics: ansible, grafana, kafka, prometheus
• Language: HTML
• Homepage:
• Size: 77.1 KB
• Stars: 0
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.adoc

Awesome Lists containing this project

README

        
= Streams Monitoring

Automation to deploy the AMQ Streams Monitoring Demo.

== Tools available

* OpenShift User Workload Monitoring

* AMQ Streams Operator

* Grafana Operator

* https://github.com/provectus/kafka-ui[Kafka-ui]

== Install The Demo Using Ansible

=== Ansible prerequisites

Make sure you have Ansible installed and with all dependencies before running the playbook.

----

sudo yum install ansible

sudo pip3 install openshift

ansible-galaxy collection install kubernetes.core

----

=== Parameters

[options="header"]

|=======================

| Parameter | Example Value                                      | Definition

| token     | sha256~vFanQbthlPKfsaldJT3bdLXIyEkd7ypO_XPygY1DNtQ | access token for a user with cluster-admin privileges

| server    | https://api.mycluster.opentlc.com:6443             | OpenShift cluster API URL

|=======================

=== Deploying the demo

----

token=REPLACE_ME

server=REPLACE_ME

ansible-playbook -e token=${token} -e server=${server} playbook.yml

----

== Using the applications

This demo comes with to applications that simulates a consumer and producer relationship. Those applications can be run using a terminal. Follow the instructions below:

=== Get the certificate for TLS connection

To connect the applications from outside the OpenShift cluster you first need the certificate for the TLS connection:

----

PROJECT=kafka-cluster

oc extract -n $PROJECT secret/my-cluster-cluster-ca-cert --keys ca.crt --to /tmp/

keytool -keystore /tmp/client-truststore.jks -alias CARoot -import -file /tmp/ca.crt -storepass kafka1 -noprompt

----

The keystore will be used by the applications to trust the certificate in the connection.

=== Starting the Producer

----

PROJECT=kafka-cluster

PRODUCER_PWD=$(oc get secret producer -n $PROJECT  -o jsonpath='{.data.password}' | base64 -d )

BOOTSTRAP_URL=$(oc get route my-cluster-kafka-bootstrap -n $PROJECT -o jsonpath='{.spec.host}'):443

mvn clean quarkus:dev -Dbootstrap.url=$BOOTSTRAP_URL  -Dproducer.pwd=$PRODUCER_PWD

----

=== Starting the Consumer

----

PROJECT=kafka-cluster

CONSUMER_PWD=$(oc get secret consumer -n $PROJECT -o jsonpath='{.data.password}' | base64 -d )

BOOTSTRAP_URL=$(oc get route my-cluster-kafka-bootstrap -n $PROJECT -o jsonpath='{.spec.host}'):443

mvn clean quarkus:dev -Dbootstrap.url=$BOOTSTRAP_URL  -Dconsumer.pwd=$CONSUMER_PWD

----

=== Testing

First monitor the stream output

----

curl  http://localhost:8080/quotes

----

Then start sending requests

----

curl  http://localhost:8080/quotes/request -X POST

----

== Additional information

To add more dashboards for other Strimzi's components, check this https://github.com/strimzi/strimzi-kafka-operator/tree/main/examples/metrics/grafana-dashboards[git repo]