https://github.com/dayitv89/kafka-schema-patcher

https://github.com/dayitv89/kafka-schema-patcher

Last synced: 18 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/dayitv89/kafka-schema-patcher
• Owner: dayitv89
• Created: 2022-07-21T15:28:44.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2022-07-22T15:31:11.000Z (over 2 years ago)
• Last Synced: 2023-03-24T08:42:28.085Z (almost 2 years ago)
• Language: Go
• Size: 11.7 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# kafka-schema-patcher

A go lang based tool to fix the kafka avro message with old schema id to new schema id. This is useful if you migrated the kafka and schema servers and something unwanted happen as message has old schema server id and new message has new schema server id.

So this tool is helpful to transform message which has wrong schema id(old shema server id) to correct schema id and publish it to the same topic again.

See the env

```env

LOG_IN_JSON=false

KAFKA_GROUP_NAME=kafka-schema-patcher-tool

KAFKA_MESSAGE_CONCURRENCY=100

KAFKA_FETCH_MESSAGE_TIMEOUT_IN_SECOND=10

KAFKA_BROKERS=127.0.0.1:9092

KAFKA_TOPIC=kafka.topic.has_wrong_schema_id_in_few_messages

KAFKA_SCHEMA_REGISTRY_URL=http://127.0.0.1:8081

KAFKA_SCHEMA_OLD_ID=60

KAFKA_SCHEMA_NEW_ID=4

```

## Links:

**Github:** https://github.com/dayitv89/kafka-schema-patcher

**Docker:** https://hub.docker.com/r/gauravds/kafka-schema-patcher

## Build docker image locally:

`docker buildx build --platform=linux/amd64 . --no-cache -t gauravds/kafka-schema-patcher:1.2`

## Run docker image

`docker run -it --env-file .env gauravds/kafka-schema-patcher:1.2`

## Publish docker image:

`docker push gauravds/kafka-schema-patcher:1.2`