https://github.com/farmdawgnation/kafka-hawk
An application that records stats about consumer group offset commits and reports them as prometheus metrics
https://github.com/farmdawgnation/kafka-hawk
java java11 kafka
Last synced: about 1 year ago
JSON representation
An application that records stats about consumer group offset commits and reports them as prometheus metrics
- Host: GitHub
- URL: https://github.com/farmdawgnation/kafka-hawk
- Owner: farmdawgnation
- License: apache-2.0
- Created: 2018-11-04T19:20:22.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2019-04-27T14:01:25.000Z (about 7 years ago)
- Last Synced: 2025-02-15T17:38:42.444Z (over 1 year ago)
- Topics: java, java11, kafka
- Language: Java
- Size: 142 KB
- Stars: 14
- Watchers: 5
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# kafka-hawk
An application that records stats about consumer group offset commits and
reports them as prometheus metrics.
This application is useful for determining how frequently each consumer group
in your Kafka cluster is committing the offsets and, for the chattiest ones,
it can calculate what the offset deltas are over time.
## Building
If you wish to build the project you'll need JDK 11.
We also recommend Docker for running the local Kafka and Zookeeper instance
for manual testing purposes.
## Deployment
This application is packaged and designed to be deployed using a Docker image.
It's available on Docker Hub as `farmdawgnation/kafka-hawk`.
## Configuration
The application can be configured by environment variables for most cases.
* `BOOTSTRAP_SERVERS` — (required) The bootstrap servers setting for hawk
* `GROUP_ID` - (optional) The group id, defaults to "hawk"
* `SASL_CONFIG` - (optional) The SASL config if needed
* `CLUSTER_NAME` - (optional) The custom name for this cluster (if any).
Useful for situations where multiple instances of Hawk are deployed to
monitor different clusters. Defaults to "kafka".
* `JSON_LOGGING` - (optional) If this environment variable is present, logback
will configure itself to use Logstash compatible JSON logs.
The application also has an optional feature that can report on the offset
commit deltas. You have to enable this per consumer group because it requires
more resources to track the offset diffs in memory. It can be enabled with
and configured with the following environment variables:
* `FEATURES_DELTAS_ENABLED` - Set to "true" to enable the feature
* `FEATURES_DELTAS_GROUPS` - A comma separated list of groups to report deltas on
# About the Author
This tool was written by Matt Farmer who by day slings code and architecture
for [Mailchimp](https://mailchimp.com)'s Data Application Development Team.