https://github.com/nodefluent/anon-kafka-mirror

Mirror Kafka topics (JSON) while anonymising certain fields with (deterministic) random values
https://github.com/nodefluent/anon-kafka-mirror

anonymize kafka mirror nodefluent random

Last synced: about 1 year ago
JSON representation

Mirror Kafka topics (JSON) while anonymising certain fields with (deterministic) random values

• Host: GitHub
• URL: https://github.com/nodefluent/anon-kafka-mirror
• Owner: nodefluent
• License: mit
• Created: 2018-01-18T14:47:25.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2022-12-10T15:01:27.000Z (over 3 years ago)
• Last Synced: 2024-10-10T13:18:17.026Z (over 1 year ago)
• Topics: anonymize, kafka, mirror, nodefluent, random
• Language: TypeScript
• Homepage:
• Size: 908 KB
• Stars: 3
• Watchers: 9
• Forks: 2
• Open Issues: 7
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# anon-kafka-mirror - consume, anon, produce

[![npm version](https://badge.fury.io/js/anon-kafka-mirror.svg)](https://badge.fury.io/js/anon-kafka-mirror)

[![Docker Repository on Quay](https://quay.io/repository/nodefluent/anon-kafka-mirror/status "Docker Repository on Quay")](https://quay.io/repository/nodefluent/anon-kafka-mirror)

## Intro

Anonymize Kafka topics while mirroring them on the fly.

Just pass a config file describing your topics and schema (and Kafka cluster connections) and start the service / container.

## Example

- Start docker compose setup: `yarn setup`

- Produce some message to **input** topic: `echo '{"test": 2, "abc":"cba", "id": "1", "mail": "real@email.com", "a":["test"], "b":[{"c":0},{"d":1}, {"c":1}], "z": [["test"],[]], "d": [[{"e": 0}, {"ex": 1}]], "f": [{"g": "g", "t": 1},"test"], "y": [1,2,3,4,5], "x":[{"x": 1, "y": 2}, {"g":2}] }' | kafka-console-producer --topic input --broker-list localhost:9092`

- Show output message in **output topic** : `kafka-console-consumer --topic output --from-beginning --bootstrap-server localhost:9092`

### Maintainer

Build with :heart: :pizza: and :coffee: by [nodefluent](https://github.com/nodefluent)