https://github.com/smallnest/kafka_monitor
A kafka monitor to send alarm emails
https://github.com/smallnest/kafka_monitor
kafka lag monitor sarama
Last synced: 28 days ago
JSON representation
A kafka monitor to send alarm emails
- Host: GitHub
- URL: https://github.com/smallnest/kafka_monitor
- Owner: smallnest
- License: apache-2.0
- Created: 2017-02-04T02:58:36.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2017-02-06T08:20:20.000Z (about 8 years ago)
- Last Synced: 2025-03-19T06:51:32.588Z (about 1 month ago)
- Topics: kafka, lag, monitor, sarama
- Language: Go
- Homepage:
- Size: 2.28 MB
- Stars: 14
- Watchers: 3
- Forks: 6
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## KafkaMonitor
the design and implementation reference to [kt](https://github.com/fgeller/kt) and [kafkatools](https://github.com/jurriaan/kafkatools)
### Install
```
go get github.com/smallnest/kafka_monitor
```
### Run
```
# change the below options with your environments
[email protected]
smtpHost=xxxxxx
smtpPort=25
[email protected]
smtpPassword=xxxxxx
brokers=10.77.11.1:9092,10.77.11.2:9092,10.77.11.3:9092,10.77.11.4:9092
version=0.9.0.1
zkAddr=10.77.0.1:2181,10.77.0.2:2181,10.77.0.3:2181,10.77.0.4:2181,10.77.0.5:2181
./kafka_monitor -brokers ${brokers} -version ${version} -email=${informEmail} -smtp.host ${smtpHost} -smtp.port ${smtpPort) -smtp.user ${smtpUser} -smtp.password ${smtpPassword} -zkAddr ${zkAddr} -basePath "/kafka"
```
### Output
Once some partitions are not writable, down or some consumers don't work, an email will be sent to your email with details.
Normal kafka cluster info is output in console:
```
Brokers: 10.77.11.1:9092,10.77.11.2:9092,10.77.11.4:9092:10.77.11.3:9092
Version: 0.9.0.1
Topic: products, Group: default, Partitions: 32
+-----------+-------------------+--------+---------------+---------------+
| PARTITION | LEADER ADDRESS | LEADER | REPLICAS | ISR |
+-----------+-------------------+--------+---------------+---------------+
| 0 | 10.77.11.1:9092 | 1 | [1 2 3] | [1 2 3] |
| 1 | 10.77.11.2:9092 | 2 | [1 2 4] | [1 2 4] |
| 2 | 10.77.11.4:9092 | 4 | [2 4 3] | [2 4 3] |
| 3 | 10.77.11.3:9092 | 3 | [1 4 3] | [1 4 3] |
| 4 | 10.77.11.1:9092 | 1 | [1 2 4] | [1 2 4] |
| 5 | 10.77.11.2:9092 | 2 | [2 4 3] | [2 4 3] |
| 6 | 10.77.11.4:9092 | 4 | [1 4 3] | [1 4 3] |
| 7 | 10.77.11.3:9092 | 3 | [1 2 3] | [1 2 3] |
| 8 | 10.77.11.1:9092 | 1 | [1 4 3] | [1 4 3] |
| 9 | 10.77.11.2:9092 | 2 | [1 2 3] | [1 2 3] |
| 10 | 10.77.11.4:9092 | 4 | [1 2 4] | [1 2 4] |
| 11 | 10.77.11.3:9092 | 3 | [2 4 3] | [2 4 3] |
| 12 | 10.77.11.1:9092 | 1 | [1 2 3] | [1 2 3] |
| 13 | 10.77.11.2:9092 | 2 | [1 2 4] | [1 2 4] |
| 14 | 10.77.11.4:9092 | 4 | [2 4 3] | [2 4 3] |
| 15 | 10.77.11.3:9092 | 3 | [1 4 3] | [1 4 3] |
| 16 | 10.77.11.1:9092 | 1 | [1 2 4] | [1 2 4] |
| 17 | 10.77.11.2:9092 | 2 | [2 4 3] | [2 4 3] |
| 18 | 10.77.11.4:9092 | 4 | [1 4 3] | [1 4 3] |
| 19 | 10.77.11.3:9092 | 3 | [1 2 3] | [1 2 3] |
| 20 | 10.77.11.1:9092 | 1 | [1 4 3] | [1 4 3] |
| 21 | 10.77.11.2:9092 | 2 | [1 2 3] | [1 2 3] |
| 22 | 10.77.11.4:9092 | 4 | [1 2 4] | [1 2 4] |
| 23 | 10.77.11.3:9092 | 3 | [2 4 3] | [2 4 3] |
| 24 | 10.77.11.1:9092 | 1 | [1 2 3] | [1 2 3] |
| 25 | 10.77.11.2:9092 | 2 | [1 2 4] | [1 2 4] |
| 26 | 10.77.11.4:9092 | 4 | [2 4 3] | [2 4 3] |
| 27 | 10.77.11.3:9092 | 3 | [1 4 3] | [1 4 3] |
| 28 | 10.77.11.1:9092 | 1 | [1 2 4] | [1 2 4] |
| 29 | 10.77.11.2:9092 | 2 | [2 4 3] | [2 4 3] |
| 30 | 10.77.11.4:9092 | 4 | [1 4 3] | [1 4 3] |
| 31 | 10.77.11.3:9092 | 3 | [1 2 3] | [1 2 3] |
+-----------+-------------------+--------+---------------+---------------+
+-----------+------------+--------------+-----+
| PARTITION | END OF LOG | GROUP OFFSET | LAG |
+-----------+------------+--------------+-----+
| 0 | 1106509557 | 1106509542 | 15 |
| 1 | 1106517793 | 1106517779 | 14 |
| 2 | 1106573067 | 1106573047 | 20 |
| 3 | 1106531253 | 1106531253 | 0 |
| 4 | 1106529394 | 1106529385 | 9 |
| 5 | 11065504 | 1106550146 | 4 |
| 6 | 1106467253 | 1106467248 | 5 |
| 7 | 1106479964 | 1106479964 | 0 |
| 8 | 1106464607 | 1106464594 | 13 |
| 9 | 1106503601 | 1106503589 | 12 |
| 10 | 1106543568 | 1106543552 | 16 |
| 11 | 1106558780 | 1106558768 | 12 |
| 12 | 1106494251 | 1106494238 | 13 |
| 13 | 1106561496 | 1106561490 | 6 |
| 14 | 1106521678 | 1106521673 | 5 |
| 15 | 1106569720 | 1106569686 | 34 |
| 16 | 1106573067 | 1106573049 | 18 |
| 17 | 1106521317 | 1106521305 | 12 |
| 18 | 1106534494 | 1106534484 | 10 |
| 19 | 1106481325 | 1106481268 | 57 |
| 20 | 1106530486 | 1106530465 | 21 |
| 21 | 1106500650 | 1106500624 | 26 |
| 22 | 1106552135 | 1106552113 | 22 |
| 23 | 1106551799 | 1106551771 | 28 |
| 24 | 1106564761 | 1106564750 | 11 |
| 25 | 1106537078 | 1106537064 | 14 |
| 26 | 1106573795 | 1106573781 | 14 |
| 27 | 1106493569 | 1106493554 | 15 |
| 28 | 1106500029 | 1106500015 | 14 |
| 29 | 1106528095 | 1106528078 | 17 |
| 30 | 1106493651 | 1106493629 | 22 |
| 31 | 1106562315 | 1106562312 | 3 |
+-----------+------------+--------------+-----+
| TOTAL | 482 |
+-----------+------------+--------------+-----+
```