https://github.com/tsmx/kafkajs-file-consumer
File-consumer project for the "Using Apache Kafka for transferring files" tutorial
https://github.com/tsmx/kafkajs-file-consumer
apache-kafka file-transfer kafka kafkajs
Last synced: 3 days ago
JSON representation
File-consumer project for the "Using Apache Kafka for transferring files" tutorial
- Host: GitHub
- URL: https://github.com/tsmx/kafkajs-file-consumer
- Owner: tsmx
- License: mit
- Created: 2025-04-14T20:16:18.000Z (6 months ago)
- Default Branch: master
- Last Pushed: 2025-04-28T19:13:18.000Z (5 months ago)
- Last Synced: 2025-07-19T13:52:52.955Z (3 months ago)
- Topics: apache-kafka, file-transfer, kafka, kafkajs
- Language: JavaScript
- Homepage: https://tsmx.net/using-apache-kafka-for-transferring-files/
- Size: 12.7 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# KafkaJS file consumer
Consumer example for the [Using Apache Kafka for transferring files](https://tsmx.net/using-apache-kafka-for-transferring-files/) article.
Uses the [KafkaJS](https://kafka.js.org/) library for Kafka client functionality.
## Usage
First, [get Kafka up & running](https://tsmx.net/using-apache-kafka-for-transferring-files/#Getting_Kafka_up_running) and [create an appropriate topic](https://tsmx.net/using-apache-kafka-for-transferring-files/#Creating_a_topic) as described in the tutorial.
To consume file messages, simply start the consumer app. Received files from Kafka will be written to the `output/` subfolder.
```bash
$ node app.js
{"level":"INFO","timestamp":"2025-04-16T20:24:47.647Z","logger":"kafkajs","message":"[Consumer] Starting","groupId":"kafkajs-group"}
{"level":"INFO","timestamp":"2025-04-16T20:25:07.791Z","logger":"kafkajs","message":"[ConsumerGroup] Consumer has joined the group","groupId":"kafkajs-group","memberId":"kafkajs-file-consumer-fe505a61-5d2a-47ca-bbbc-07ec4df8c3d0","leaderId":"kafkajs-file-consumer-fe505a61-5d2a-47ca-bbbc-07ec4df8c3d0","isLeader":true,"memberAssignment":{"filetransfer":[0]},"groupProtocol":"RoundRobinAssigner","duration":20140}
File example.jpg written. (2690215 Bytes)
```
To produce file messages, use the [producer example](https://github.com/tsmx/kafkajs-file-producer).