https://github.com/oslabs-beta/kafkasaur
Apache Kafka client for Deno
https://github.com/oslabs-beta/kafkasaur
apache-kafka deno denoland kafka-client
Last synced: about 2 months ago
JSON representation
Apache Kafka client for Deno
- Host: GitHub
- URL: https://github.com/oslabs-beta/kafkasaur
- Owner: oslabs-beta
- License: mit
- Created: 2021-08-06T16:56:24.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2021-12-02T00:25:08.000Z (over 3 years ago)
- Last Synced: 2025-04-08T10:53:19.013Z (3 months ago)
- Topics: apache-kafka, deno, denoland, kafka-client
- Language: TypeScript
- Homepage:
- Size: 2.37 MB
- Stars: 45
- Watchers: 4
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
A native Deno client for Apache Kafka
## Table of Contents
- [About the project](#about)
- [Features](#features)
- [Getting Started](#getting-started)
- [Usage](#usage)
- [Developers](#developers)
- [Acknowledgements](#acknowledgements)
- [License](#license)
## About the Project
KafkaSaur an Apache Kafka client for deno
###
Getting Started
```sh
https://deno.land/x/[email protected]
```
### Prerequisites
>Deno - https://deno.land/manual/getting_started/installation
>
>Apache Kafka - https://kafka.apache.org/
>
>Docker (for examples) -https://www.docker.com/
#### Usage
To run examples initiate Docker containers included in yaml file:
```sh
docker-compose up
```
Then run example producer/consumer files, in seperate terminals, with the following commands:
```sh
deno run --allow-all --unstable examples/example_producer.ts
deno run --allow-all --unstable examples/example_consumer.ts
```
Your two terminals (one Consuming, and one Producing) will now interact with the Broker and begin consuming and producing respectively.
With the Client imported into your application, you can write the producer/consumer logic like this:
```typescript
//producer example
import {Kafkasaur} from "https://deno.land/x/kafkasaur/index.ts"
const kafka = new Kafkasaur({
clientId: 'example-producer',
brokers: ['localhost:9092']
})
const topic = 'topic-test';
const producer = kafka.producer();
const testmessage = {
key: 'key',
value: 'hello there',
headers: {'correlation-id': `${Date.now()}`}
}
const messages: object[] = [];
messages.push(testmessage)
const sendMessage = () => {
producer.send({
topic,
messages
})
}
const run = async() => {
await producer.connect();
sendMessage();
}
run()
```
```typescript
//consumer example
import {Kafkasaur} from "https://deno.land/x/kafkasaur/index.ts"
const kafka = new Kafkasaur({
clientId: 'example-consumer',
brokers: ['localhost:9092']
})
const topic = 'topic-test';
const consumer = kafka.consumer({ groupId: 'test-group' })
const run = async () => {
await consumer.connect()
await consumer.subscribe({ topic, fromBeginning: true })
await consumer.run({
eachMessage: async (message: any) => {
console.log(message.value.toString())
},
})
}
run()
```
To run the instances of your Consumer/Producer be sure to pass the flags
```sh
--allow-all
```
and
```sh
--unstable
```
when issuing your deno run command. This ensures that Deno as the proper configuration to communicate with the Broker, and to log any errors.
## Features
- 🛠 Built with [TypeScript][Deno]
- 🎬 Producer
- 🍴 Consumer
- 🤝 interactive producer with consumer
- 💂 deno's built in security; No file, network, or environment access, unless explicitly enabled
## Want to Contribute?
If you'd like to contribute and help grow the Deno community, just reach out to one of us via LinkedIn or write some code, and make a PR here! We're super excited about getting the conversation started, and working to bring Kafka to Deno!
Sam Arnold
Wesley Appleget
Adam Blackwell
Stephanie Benitez
- [Tommy Brunn](https://github.com/Nevon) - for his guidance and for trailblazing with KafkaJS
- [Ryan Dahl](https://github.com/ry) - for building an awesome community with Node.js and then leveling it up even further with Deno
- [Franz Kafka](https://en.wikipedia.org/wiki/Franz_Kafka) - for making us all remember, we could just be cockroaches.
This product is licensed under the MIT License - see the LICENSE.md file for details.
This is an open source product.
This product is accelerated by [OS Labs](https://opensourcelabs.io/).
Apache Kafka and Kafka are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries. KafkaJS has no affiliation with the Apache Software Foundation.