https://github.com/jaskaranbir/go-kafkaproxy

Go library providing abstraction over Shopify's Sarama library.
https://github.com/jaskaranbir/go-kafkaproxy

golang kafka sarama

Last synced: 20 days ago
JSON representation

Go library providing abstraction over Shopify's Sarama library.

• Host: GitHub
• URL: https://github.com/jaskaranbir/go-kafkaproxy
• Owner: Jaskaranbir
• License: mit
• Created: 2018-07-20T19:15:25.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2018-08-05T15:07:13.000Z (over 6 years ago)
• Last Synced: 2023-03-02T17:11:43.547Z (almost 2 years ago)
• Topics: golang, kafka, sarama
• Language: Go
• Size: 17.6 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
## KafkaProxy for Sarama library

This is just a simple Go library providing abstraction over Shopify's [Sarama][0] library.

### Usage:

* Install [dep][1] dependencies:

```

dep ensure

```

* Import/Use in your code!

```Go

import github.com/Jaskaranbir/go-kafkaproxy/pkg/consumer // Import Consumer

import github.com/Jaskaranbir/go-kafkaproxy/pkg/producer // Import Producer

```

### Docs:

* **[Consumer][2]**

* **[Producer][3]**

* **[ProxyError][4]**

### Minimal examples:

#### Consumer:

```Go

msgHandler := func(msg *sarama.ConsumerMessage, c *consumer.Consumer) {

  // Convert from []byte to string

  println("Received message: ", string(msg.Value))

  consumer := c.Get()

  if !c.IsClosed() {

    consumer.MarkOffset(msg, "")

  } else {

    log.Fatalln("Consumer was closed before offsets could be marked.")

  }

}

errHandler := func(e *error) {

  log.Fatalln((*e).Error())

}

config := consumer.Config{

  ConsumerGroup: "test",

  ErrHandler:    errHandler,

  KafkaBrokers:  []string{"localhost:9092"},

  MsgHandler:    msgHandler,

  Topics:        []string{"test"},

}

proxyconsumer, _ := consumer.New(&config)

proxyconsumer.EnableLogging()

// Temporary hack for simplicity. Use channels/app-logic in actual application.

time.Sleep(100000 * time.Millisecond)

```

#### Producer:

```Go

errHandler := func(err *sarama.ProducerError) {

  errs := *err

  fmt.Println(errs.Error())

}

config := producer.Config{

  ErrHandler:   errHandler,

  KafkaBrokers: []string{"localhost:9092"},

}

asyncProducer, err := producer.New(&config)

asyncProducer.EnableLogging()

strTime := strconv.Itoa(int(time.Now().Unix()))

msg := asyncProducer.CreateKeyMessage("test", strTime, "testValue")

if err != nil {

  panic(err)

}

input, _ := asyncProducer.Input()

input <- msg

// Temporary hack for simplicity. Use channels/app-logic in actual application.

time.Sleep(2000 * time.Millisecond)

```

  [0]: https://github.com/Shopify/sarama

  [1]: https://github.com/golang/dep

  [2]: https://godoc.org/github.com/Jaskaranbir/go-kafkaproxy/pkg/consumer

  [3]: https://godoc.org/github.com/Jaskaranbir/go-kafkaproxy/pkg/producer

  [4]: https://godoc.org/github.com/Jaskaranbir/go-kafkaproxy/pkg/proxyerror