https://github.com/arjun-1/kafkagosaur
Kafka client for Deno binding to kafka-go using WebAssembly
https://github.com/arjun-1/kafkagosaur
deno golang kafka typescript wasm webassembly
Last synced: over 1 year ago
JSON representation
Kafka client for Deno binding to kafka-go using WebAssembly
- Host: GitHub
- URL: https://github.com/arjun-1/kafkagosaur
- Owner: arjun-1
- License: mit
- Created: 2021-12-05T17:18:04.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2023-02-25T00:13:59.000Z (over 3 years ago)
- Last Synced: 2025-02-17T06:17:43.814Z (over 1 year ago)
- Topics: deno, golang, kafka, typescript, wasm, webassembly
- Language: TypeScript
- Homepage:
- Size: 13.3 MB
- Stars: 15
- Watchers: 2
- Forks: 1
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: security/sasl.ts
Awesome Lists containing this project
README
# kafkagosaur
[](https://deno.land/x/kafkagosaur)
[](https://doc.deno.land/https/deno.land/x/kafkagosaur/mod.ts)
[](https://github.com/arjun-1/kafkagosaur/actions/workflows/CI.yaml)
[](https://codecov.io/gh/arjun-1/kafkagosaur)
Kafkagosaur is a Kafka client for Deno built using WebAssembly. The project
binds to the [kafka-go](https://github.com/segmentio/kafka-go) library.
See [this article](https://medium.com/@arjun.dhawan/kafkagosaur-eac3c063388) for
an overview of its inner workings.
## Supported features
- [x] Writer
- [x] Reader
- [x] SASL
- [x] TLS
- [ ] Deno streams
## Examples
For comprehensive examples on how to use kafkagosaur, head over to the
[provided examples](#provided-examples).
#### KafkaWriter
To write a message, first create `KafkaWriter` instance using the `createWriter`
function on the `KafkaGoSaur` instance:
```typescript
import KafkaGoSaur from "https://deno.land/x/kafkagosaur/mod.ts";
const kafkaGoSaur = new KafkaGoSaur();
const writer = await kafkaGoSaur.createWriter({
broker: "localhost:9092",
topic: "test-0",
});
const enc = new TextEncoder();
const msgs = [{ value: enc.encode("value") }];
await writer.writeMessages(msgs);
```
#### KafkaReader
To read a message, first create `KafkaReader` instance using the `createReader`
function on the `KafkaGoSaur` instance:
```typescript
import KafkaGoSaur from "https://deno.land/x/kafkagosaur/mod.ts";
const kafkaGoSaur = new KafkaGoSaur();
const reader = await kafkaGoSaur.createReader({
brokers: ["localhost:9092"],
topic: "test-0",
});
const readMsg = await reader.readMessage();
```
### Provided examples
To run the [provided examples](examples), ensure you have docker up and running.
Then start the kafka broker using
```bash
make docker-up
```
To run the writer example
```bash
deno run --allow-read --allow-net --unstable examples/writer.ts
```
To run the reader example
```bash
deno run --allow-read --allow-net --unstable examples/reader.ts
```
## Documentation
The API documentation is hosted
[here](https://doc.deno.land/https/deno.land/x/kafkagosaur/mod.ts).
## Development
To build the WebAssembly module, first run
```bash
make build-wasm
```
To run the tests, ensure first you have docker up and running. Then start the
kafka broker using
```bash
make docker-up
```
Then run
```bash
make test
```
## Performance benchmarks
The Deno benchmarks are located in [bench](bench) and can be run via
```bash
deno run --allow-read --allow-net --allow-env --unstable bench/reader.ts
```
```bash
deno run --allow-read --allow-net --allow-env --unstable bench/writer.ts
```
### Results
| | kafka-go[^2] | kafkagosaur (`DialBackend.Node`) | kafkagosaur (`DialBackend.Deno`) |
| ----------------- | ------------ | -------------------------------- | -------------------------------- |
| fetchMessage | 5236 ± 743 | 2760 ± 376 | 2678 ± 361 |
| writeMessages[^1] | 5333 ± 148 | 4455 ± 159 | N/A[^3] |
[^1]: Batching 10.000 messages.
[^2]: Using a single goroutine.
[^3]: `DialBackend.Deno` not yet functional for writing.
#### Environment
- 2,6 GHz 6-Core Intel Core i7
- [Confluent Cloud Basic cluster](https://docs.confluent.io/cloud/current/clusters/cluster-types.html#basic-clusters);
6 partitions
## Contributing
Kafkgagosaur is in early stage of development. Nevertheless your contributions
are highly valued and welcomed! Feel free to ask for new features, report bugs,
or submit your code.