https://github.com/perbu/mkafka

https://github.com/perbu/mkafka

Last synced: about 2 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/perbu/mkafka
• Owner: perbu
• License: bsd-3-clause
• Created: 2023-03-06T13:01:16.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2023-03-06T13:14:51.000Z (almost 2 years ago)
• Last Synced: 2024-06-21T04:51:11.168Z (7 months ago)
• Language: Go
• Size: 8.79 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.txt

Awesome Lists containing this project

README

        
# mkafka

This package implements a buffered writer for Kafka. It'll try to write to Kafka asynchronously, 

and if it fails, it'll retry until it succeeds. It'll persist the in the buffer until it succeeds.

It uses the amazing franz-go package to do the heavy lifting.

## Things to note:

Writing should never fail. If kafka fails, the module will buffer.

Run() will block until the context is cancelled. On context cancel, it'll flush the buffer, if this fails then

it'll return an error, so be sure to check the error.

## Usage

```go

package main

import (

	"context"

	"github.com/perbu/mkafka"

	"log"

	"os"

	"os/signal"

	"sync"

)

func main() {

	// create a client.

	client, err := mkafka.New()

	if err != nil {

		panic(err)

	}

	ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt)

	defer cancel()

	wg := &sync.WaitGroup{}

	wg.Add(1)

	go func() {

		defer wg.Done()

		// write to kafka, will always succeed

		err := client.Run(ctx)

		if err != nil {

			log.Println("client.Run() failed:", err)

		}

	}()

	// write to kafka, will always succeed

	err = client.Write(ctx, "test", []byte("hello world"))

	if err != nil {

        log.Println("client.Write() failed:", err)

    }

	cancel()

	wg.Wait()

}

```