https://github.com/scristobal/rxjs-kafka

A no fuss rxjs wrapper for kafkajs, focused on ease of use.
https://github.com/scristobal/rxjs-kafka

Last synced: about 5 hours ago
JSON representation

A no fuss rxjs wrapper for kafkajs, focused on ease of use.

• Host: GitHub
• URL: https://github.com/scristobal/rxjs-kafka
• Owner: scristobal
• License: gpl-3.0
• Created: 2021-10-09T19:37:33.000Z (about 3 years ago)
• Default Branch: main
• Last Pushed: 2022-07-17T07:14:23.000Z (over 2 years ago)
• Last Synced: 2024-10-06T02:15:20.404Z (about 1 month ago)
• Language: TypeScript
• Size: 615 KB
• Stars: 9
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# rxjs-kafka or rxkfk

A no fuss rxjs wrapper for kafkajs, focused on ease of use.

## Who?

Anyone who just wants to read/write Kafka topics from the comfort of reactive javascript.

## Getting started

> This section follows the same example as in KakfaJS docs ()

Install rxkfk using yarn:

```bash

yarn add rxjs-kafka

```

Or npm:

```bash

npm install rxjs-kafka

```

Let's start by creating our RxJS subjects, using our Kafka brokers, topic and consumer details:

```typescript

import fromKafkaTopic from 'rxjs-kafka';

import { of, first } from 'rxjs';

const { message$, send } = fromKafkaTopic(

    {

        clientId: 'my-app',

        brokers: ['kafka1:9092', 'kafka2:9092']

    },

    { topic: 'test-topic', fromBeginning: true },

    { groupId: 'test-group' }

);

```

Now to produce a message to a topic, we'll subscribe `send` to an observable:

```typescript

of('Hello KafkaJS user!').subscribe(send);

```

Finally, to verify that our message has indeed been produced to the topic, let's subscribe an observer to the `message$` subject:

```typescript

message$.pipe(first()).subscribe(console.log);

```

Congratulations, you just produced and consumed your first Kafka message using RxJS and a bit of help from rxkfk!

## Limitations

-   only a single topic per subject (intended)

-   only JSON payloads (intended)

-   only pushing messages to a single topic (intended)

-   no transactions (intended, there is no equivalent mechanisms in RXJS and probably never will)

## Roadmap

-   expose `batches` to allow consume a whole batch from Kafka.

-   return an observable containing the operation result when producing messages, eg. `RecordMetadata[]` returned by `producer.send()`

-   using `eachBatchAutoResolve: true` might be problematic with `take()` or similar, since the whole batch is consumed but maybe fewer elements are delivered