Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/ri-nat/kafka-proto-monitor

🛠️ A simple CLI utility for monitoring protobuf-encoded messages in Kafka topics
https://github.com/ri-nat/kafka-proto-monitor

debug kafka logger monitor protobuf tool utility

Last synced: about 17 hours ago
JSON representation

🛠️ A simple CLI utility for monitoring protobuf-encoded messages in Kafka topics

Host: GitHub
URL: https://github.com/ri-nat/kafka-proto-monitor
Owner: ri-nat
License: mit
Created: 2022-10-18T15:07:09.000Z (about 2 years ago)
Default Branch: main
Last Pushed: 2023-03-06T14:12:20.000Z (almost 2 years ago)
Last Synced: 2024-06-20T17:39:22.280Z (7 months ago)
Topics: debug, kafka, logger, monitor, protobuf, tool, utility
Language: Go
Homepage:
Size: 62.5 KB
Stars: 2
Watchers: 1
Forks: 0
Open Issues: 2
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# kafka-proto-monitor

[![Lints](https://github.com/ri-nat/kafka-proto-monitor/actions/workflows/lints.yml/badge.svg)](https://github.com/ri-nat/kafka-proto-monitor/actions/workflows/lints.yml)
[![Tests](https://github.com/ri-nat/kafka-proto-monitor/actions/workflows/tests.yml/badge.svg)](https://github.com/ri-nat/kafka-proto-monitor/actions/workflows/tests.yml)

🛠️ A simple CLI utility for monitoring protobuf-encoded messages in Kafka topics.

Under the hood, `kafka-proto-monitor` relies on [protoreflect](https://github.com/jhump/protoreflect) in order to use raw (not compiled) proto files in runtime for unmarshaling Kafka messages.

## Installation

```bash
go install github.com/ri-nat/kafka-proto-monitor@latest
```

## Usage

```
Usage:
kafka-proto-monitor [OPTIONS]

Application Options:
-b, --broker= Kafka broker URL (you can use this option multiple times) (default: localhost:9092)
-t, --topic= Kafka topic to read from (you can use this option multiple times)
-a, --read-earliest Read topic starting from the beginning (false by default)
-r, --print-headers Print message headers (false by default)
-p, --proto-file= Path to proto file
-m, --proto-message= Proto message to use
-e, --proto-message-header= Name of Kafka message header, that contains message's proto name

Help Options:
-h, --help Show this help message
```

## Examples

```bash
kafka-proto-monitor -p ./service.proto -m 'app.users.UserCreated' -t users -a -r
```

* Use `app.users.UserCreated` message from `service.proto`
* Connect to Kafka on `localhost:9092` (default)
* Read from topic `users`
* Read messages from the beginning of the topic (`-a`)
* Print message headers (`-r`)

---

```bash
kafka-proto-monitor -p ./service.proto -e proto-name -b kafka:9092 -t users
```

* Use all messages from `service.proto`
* Use `proto-name` header content as proto message name
* Connect to Kafka on `kafka:9092`
* Read from topic `users`
* Read only new messages (no `-a` flag)
* Do not print message headers (no `-r` flag)

## TODO

* Display help message on startup if no arguments provided
* Filter messages by header value
* Filter messages by parsed object attributes
* Construct proto name programmatically
* Be more verbose about message unmarshaling errors
* Use multiple proto files at once
* Filter printable attributes out
* Write more tests

## Licensing

This software is licensed under the MIT License. See [LICENSE](https://github.com/ri-nat/kafka-proto-monitor/blob/master/LICENSE) for the full license text.