https://github.com/mkroli/kafka-proxy

Service to proxy from various protocols to Kafka
https://github.com/mkroli/kafka-proxy

coap file kafka nanomsg nng posixmq proxy rest stdin tcp udp unix unix-dgram

Last synced: 3 months ago
JSON representation

Service to proxy from various protocols to Kafka

• Host: GitHub
• URL: https://github.com/mkroli/kafka-proxy
• Owner: mkroli
• License: apache-2.0
• Created: 2022-12-22T20:30:32.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-09-22T20:23:05.000Z (3 months ago)
• Last Synced: 2024-10-03T10:46:16.762Z (3 months ago)
• Topics: coap, file, kafka, nanomsg, nng, posixmq, proxy, rest, stdin, tcp, udp, unix, unix-dgram
• Language: Rust
• Homepage:
• Size: 492 KB
• Stars: 5
• Watchers: 3
• Forks: 0
• Open Issues: 5
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# kafka-proxy

[![Build](https://github.com/mkroli/kafka-proxy/actions/workflows/build.yml/badge.svg)](https://github.com/mkroli/kafka-proxy/actions/workflows/build.yml)

Sidecar service to proxy from various protocols to Kafka.

It supports proxying text messages, binary messages (on stream based protocols via newline terminated base64 strings) and [Avro](https://avro.apache.org/) with [Schema Registry](https://docs.confluent.io/platform/current/schema-registry/index.html).

For Avro support, JSON messages are expected and converted according to the schema.

## Installation

### Download Binaries

[Latest Release](https://github.com/mkroli/kafka-proxy/releases/latest)

### Using Docker

```bash

docker run --rm -it ghcr.io/mkroli/kafka-proxy --help

```

### Using Cargo

```bash

cargo install --locked --bin kafka-proxy --git https://github.com/mkroli/kafka-proxy

```

## Usage

```

Service to proxy from various protocols to Kafka

Usage: kafka-proxy [OPTIONS] --topic  

Commands:

  stdin       Read one message per line from stdin

  file        Read one message per line from a file

  unix-dgram  Receive messages from a Unix Datagram socket

  unix        Receive one message per line from a Unix socket

  udp         Receive messages from a UDP socket

  tcp         Receive one message per line from a TCP socket

  coap        Receive messages via CoAP

  rest        Receive messages via HTTP

  posixmq     Receive messages via Posix MQ

  nng         Receive messages via NNG

  help        Print this message or the help of the given subcommand(s)

Options:

      --prometheus 
  [env: KAFKA_PROXY_PROMETHEUS_ADDRESS=]

  -h, --help                  Print help

  -V, --version               Print version


Kafka Options:

  -b, --bootstrap-server 

          [env: KAFKA_PROXY_BOOTSTRAP_SERVER=] [default: 127.0.0.1:9092]

  -t, --topic 

          [env: KAFKA_PROXY_TOPIC=]

      --producer-config 

          [env: KAFKA_PROXY_PRODUCER_=]

      --dead-letters 

          [env: KAFKA_PROXY_DEAD_LETTERS=]

Schema Registry Options:

      --schema-registry-url 

          [env: KAFKA_PROXY_SCHEMA_REGISTRY_URL=]

      --schema-id 

          Use a specific schema id rather than the latest version [env: KAFKA_PROXY_SCHEMA_ID=]

      --topic-name

          Use TopicNameStrategy to derive the subject name (default)

      --record-name 

          Use RecordNameStrategy to derive the subject name [env: KAFKA_PROXY_SCHEMA_REGISTRY_RECORD_NAME=]

      --topic-record-name 

          Use TopicRecordNameStrategy to derive the subject name [env: KAFKA_PROXY_SCHEMA_REGISTRY_TOPIC_RECORD_NAME=]

```