https://github.com/harisekhon/nagios-plugin-kafka
Kafka Scala API CLI / Advanced Nagios Plugin, with Kerberos support (uses Kafka 0.9+ native Java API)
https://github.com/harisekhon/nagios-plugin-kafka
hacktoberfest kafka kafka-api kafka-broker kafka-consumer kafka-producer kerberos nagios-plugin nagios-plugins scala
Last synced: 2 months ago
JSON representation
Kafka Scala API CLI / Advanced Nagios Plugin, with Kerberos support (uses Kafka 0.9+ native Java API)
- Host: GitHub
- URL: https://github.com/harisekhon/nagios-plugin-kafka
- Owner: HariSekhon
- License: mit
- Created: 2016-06-07T15:52:24.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2025-03-14T19:41:24.000Z (7 months ago)
- Last Synced: 2025-06-13T02:06:48.726Z (4 months ago)
- Topics: hacktoberfest, kafka, kafka-api, kafka-broker, kafka-consumer, kafka-producer, kerberos, nagios-plugin, nagios-plugins, scala
- Language: Shell
- Homepage: https://www.linkedin.com/in/HariSekhon
- Size: 844 KB
- Stars: 11
- Watchers: 2
- Forks: 6
- Open Issues: 21
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Kafka Scala API - Advanced Nagios Plugin / CLI Tool with Kerberos support
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/stargazers)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/network)
[](https://github.com/boyter/scc/)
[](https://github.com/boyter/scc/)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/blob/master/LICENSE)
[](https://www.linkedin.com/in/HariSekhon/)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/commits/master)
[](https://www.codacy.com/gh/HariSekhon/Nagios-Plugin-Kafka/dashboard)
[](https://www.codefactor.io/repository/github/harisekhon/Nagios-Plugin-Kafka)
[](https://sonarcloud.io/dashboard?id=HariSekhon_Nagios-Plugin-Kafka)
[](https://sonarcloud.io/dashboard?id=HariSekhon_Nagios-Plugin-Kafka)
[](https://sonarcloud.io/dashboard?id=HariSekhon_Nagios-Plugin-Kafka)
[](https://sonarcloud.io/dashboard?id=HariSekhon_Nagios-Plugin-Kafka)
[](https://sonarcloud.io/summary/new_code?id=HariSekhon_Nagios-Plugin-Kafka)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka)
[](https://hub.docker.com/r/harisekhon/nagios-plugin-kafka)
[](https://github.com/HariSekhon/Dockerfiles)
[](https://hub.docker.com/r/harisekhon/nagios-plugin-kafka)
[](https://hub.docker.com/r/harisekhon/nagios-plugin-kafka)
[](https://harisekhon.github.io/CI-CD/)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/blob/master/Jenkinsfile)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/blob/master/cicd/.concourse.yml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/blob/master/cicd/.gocd.yml)
[](https://github.com/HariSekhon/TeamCity-CI)
[](https://circleci.com/gh/HariSekhon/Nagios-Plugin-Kafka)
[](https://buildkite.com/hari-sekhon/nagios-plugin-kafka)
[](https://ci.appveyor.com/project/HariSekhon/Nagios-Plugin-Kafka/branch/master)
[](https://cloud.drone.io/HariSekhon/Nagios-Plugin-Kafka)
[](https://g.codefresh.io/pipelines/edit/new/builds?id=5e58e3573953b779e04b7907&pipeline=Nagios%20Plugin%20Kafka&projects=GitHub&projectId=5e52ca8ea284e00f882ea992&context=github&filter=page:1;pageSize:10;timeFrameStart:week)
[](https://cirrus-ci.com/github/HariSekhon/Nagios-Plugin-Kafka)
[](https://harisekhon.semaphoreci.com/projects/Nagios-Plugin-Kafka)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/blob/master/buddy.yml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/blob/master/shippable.yml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/blob/master/.travis.yml)
[](https://dev.azure.com/harisekhon/GitHub/_build/latest?definitionId=11&branchName=master)
[](https://gitlab.com/HariSekhon/Nagios-Plugin-Kafka/pipelines)
[](https://bitbucket.org/harisekhon/nagios-plugin-kafka/addon/pipelines/home#!/)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/blob/master/cicd/buildspec.yml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/blob/master/cicd/cloudbuild.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka)
[](https://gitlab.com/HariSekhon/Nagios-Plugin-Kafka)
[](https://dev.azure.com/harisekhon/GitHub/_git/Nagios-Plugin-Kafka)
[](https://bitbucket.org/HariSekhon/Nagios-Plugin-Kafka)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/shellcheck.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/json.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/yaml.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/xml.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/markdown.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/validate.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/kics.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/grype.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/semgrep.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/semgrep-cloud.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/trivy.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22GitHub+Actions+Ubuntu%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/mac.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/mac_11.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions/workflows/mac_12.yaml)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Ubuntu%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Ubuntu+20.04%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Ubuntu+22.04%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Debian%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Debian+10%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Debian+11%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Debian+12%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Fedora%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Alpine%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Alpine+3%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Maven%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22SBT%22)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka/actions?query=workflow%3A%22Gradle%22)
[git.io/nagios-plugin-kafka](https://git.io/nagios-plugin-kafka)
Kafka 0.9+ API CLI Tester & Advanced Nagios Plugin with Kerberos support, written in Scala.
Tested on Hortonworks HDP 2.4.0 with Kerberos + Ranger ACLs and Apache Kafka 0.8.x / 0.9.0.1 [docker images](https://hub.docker.com/r/harisekhon/kafka) with regular ACLs.
You may need to change the Kafka library version in `pom.xml` / `build.sbt` / `build.gradle` before building to match your deployed Kafka server / cluster otherwise it may hang when run due to version / protocol mismatch.
See [The Advanced Nagios Plugins Collection](https://github.com/HariSekhon/Nagios-Plugins#advanced-nagios-plugins-collection) for many more related enterprise monitoring programs.
Hari Sekhon
Cloud & Big Data Contractor, United Kingdom
(ex-Cloudera, former Hortonworks Consultant)
[](https://www.linkedin.com/in/HariSekhon/)
*(you're welcome to connect with me on LinkedIn)*
## Intro
This project builds a single self-contained Java jar file with all dependencies included and can simply be run on the command line with full switch option support:
```shell
java -jar check_kafka.jar --help
```
and there is an optional convenience shell wrapper script at the top level to make commands shorter:
```shell
./check_kafka --help
```
Run against one or more Kafka brokers, comma separated:
```shell
$ ./check_kafka --brokers localhost:9092 --topic test
OK: Kafka broker successfully returned unique message via topic 'test' partition '0', write time = 0.185s, read time = 0.045s, total time = 1.729s | write_time=0.185s read_time=0.045s total_time=1.729s
```
Use the ```--verbose``` switch to also show the brokers list that were tested. If you have specified one of the kerberos switches (or edited the consumer/producer properties files to do so) then the output will additionally contain the marker ```with sasl authentication``` to let you know that it was a secure configuration that was tested (originally I called this ```with kerberos``` but technically it may not be in future).
```none
OK: Kafka broker ':6667' successfully returned unique message via topic 'topic3' partition '0' with sasl authentication, write time = 0.148s, read time = 0.043s, total time = 0.691s | write_time=0.148s read_time=0.043s total_time=0.691s
```
### Kafka 0.9+ API Caveats
This program only supports Kafka 0.9+ as the API changed (again) and Kerberos security was only added in the 0.9 API. For Kafka versions before 0.9 you can find Python and Perl versions of this program in the [Advanced Nagios Plugins Collection](https://github.com/HariSekhon/Nagios-Plugins#advanced-nagios-plugins-collection) that support 0.8 onwards (they dosn't support Kafka <= 0.7 as the API changed in 0.8 too and the underlying libraries in those languages don't support Kafka <= 0.7).
It appears that several errors are caught too early in the new Kafka Java API and result in embedded looping retry behaviour on encountering errors (visible in debug level logging of the base library).
I haven't found a great way of handle that behaviour as it's not exposed to the client code so it ends up being handled via my generic default self timeout mechanism that I apply to all my tools. Hence if you specify an incorrect ```--brokers :``` or the Kafka brokers are down or you fail to negotiate the protocol due to security settings you will only receive a generic ```UNKNOWN: self timed out after 10 secs``` message as the code self terminates.
Otherwise the Kafka API would just hang there indefintely as it keeps retrying deeper in the library. I've tried various settings to get it to time out but nothing worked and I even posted to the Kafka users mailing list without an answer. If you know of a setting that will make the Kafka Client library time out and return the more specific error then please let me know and I'll update this code accordingly.
#### Kerberos Support
See the ```conf/``` directory for JAAS kerberos configurations.
If you're running the code on a Hortonworks Kafka broker it'll auto-detect the HDP configuration and use that.
### Build
#### Quick Start - Docker
A Dockerized pre-built version is available on [DockerHub](https://hub.docker.com/r/harisekhon/nagios-plugin-kafka).
If you have docker installed this one command will download and run it:
```shell
docker run harisekhon/nagios-plugin-kafka check_kafka --help
```
#### Automated Build from Source
```shell
curl -L https://git.io/nagios-plugin-kafka-bootstrap | sh
```
OR
Maven, Gradle and SBT automated builds are all provided.
A self-contained jar file with all dependencies will be created and symlinked to ```check_kafka.jar``` at the top level.
The Maven and Gradle builds are best as they will auto bootstap and run with no prior installed dependencies other than Java and ```make``` to kick it off.
The default ```make``` build will trigger a Gradle bootstrap from scratch with an embedded checksum for security:
```shell
make
```
You can call any one of the 3 major build systems explicitly instead, which will recurse to build the library submodule using the same mechanism:
Maven:
```shell
make mvn
```
Gradle:
```shell
make gradle
```
SBT:
```shell
make sbt
```
#### Custom TLDs
If using bespoke internal domains such as `.local`, `.intranet`, `.vm`, `.cloud` etc. that aren't part of the official IANA TLD list then this is additionally supported via a custom configuration file [lib/resources/custom_tlds.txt](https://github.com/HariSekhon/lib-java/blob/master/src/main/resources/tlds-alpha-by-domain.txt) containing one TLD per line, with support for # comment prefixes. Just add your bespoke internal TLD to the file and it will then pass the host/domain/fqdn validations.
#### Testing
[Continuous Integration](https://travis-ci.org/HariSekhon/nagios-plugin-kafka) is run on this repo with tests for success and failure scenarios:
- unit tests for the custom supporting [java library](https://github.com/HariSekhon/lib-java)
- integration tests of the top level programs using the libraries for things like option parsing
- [functional tests](https://github.com/HariSekhon/Nagios-Plugin-Kafka/tree/master/tests) for the top level programs using [Docker containers](https://hub.docker.com/u/harisekhon/)
To trigger all tests run:
```shell
make test
```
which will start with the underlying libraries, then move on to top level integration tests and functional tests using docker containers if docker is available.
### Kafka 0.8 support - Alternative Perl & Python Kafka API Nagios Plugins
The [Advanced Nagios Plugins Collection](https://github.com/HariSekhon/Nagios-Plugins#advanced-nagios-plugins-collection) has both Perl and Python predecessors to this program which work with Kafka 0.8+. The main differenitator with this Scala version is that it uses the new native 0.9+ Java API which has Kerberos support (the dynamic language versions were built on libraries for Kafka 0.8).
[git.io/nagios-plugin-kafka](https://git.io/nagios-plugin-kafka)
## More Core Repos
### Knowledge
[](https://github.com/HariSekhon/Knowledge-Base)
[](https://github.com/HariSekhon/Diagrams-as-Code)
### DevOps Code
[](https://github.com/HariSekhon/DevOps-Bash-tools)
[](https://github.com/HariSekhon/DevOps-Python-tools)
[](https://github.com/HariSekhon/DevOps-Perl-tools)
[](https://github.com/HariSekhon/DevOps-Golang-tools)
### Containerization
[](https://github.com/HariSekhon/Kubernetes-configs)
[](https://github.com/HariSekhon/Dockerfiles)
### CI/CD
[](https://github.com/HariSekhon/GitHub-Actions)
[](https://github.com/HariSekhon/Jenkins)
### DBA - SQL
[](https://github.com/HariSekhon/SQL-scripts)
### DevOps Reloaded
[](https://github.com/HariSekhon/Nagios-Plugins)
[](https://github.com/HariSekhon/HAProxy-configs)
[](https://github.com/HariSekhon/Terraform)
[](https://github.com/HariSekhon/Packer-templates)
[](https://github.com/HariSekhon/Nagios-Plugin-Kafka)
### Templates
[](https://github.com/HariSekhon/Templates)
[](https://github.com/HariSekhon/Template-repo)
### Misc
[](https://github.com/HariSekhon/Spotify-tools)
[](https://github.com/HariSekhon/Spotify-playlists)
The rest of my original source repos are
[here](https://github.com/HariSekhon?tab=repositories&q=&type=source&language=&sort=stargazers).
Pre-built Docker images are available on my [DockerHub](https://hub.docker.com/u/harisekhon/).
