Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/optiopay/kafka
Go driver for Kafka
https://github.com/optiopay/kafka
driver kafka open-source
Last synced: 4 months ago
JSON representation
Go driver for Kafka
- Host: GitHub
- URL: https://github.com/optiopay/kafka
- Owner: optiopay
- License: mit
- Created: 2015-02-09T09:36:50.000Z (almost 10 years ago)
- Default Branch: master
- Last Pushed: 2019-11-04T15:58:33.000Z (about 5 years ago)
- Last Synced: 2024-07-21T17:46:53.764Z (4 months ago)
- Topics: driver, kafka, open-source
- Language: Go
- Homepage: https://godoc.org/github.com/optiopay/kafka
- Size: 711 KB
- Stars: 216
- Watchers: 27
- Forks: 57
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: CODEOWNERS
Awesome Lists containing this project
- awesome-kafka - Go
README
[](https://github.com/optiopay/kafka/actions?query=workflow%3AKafka)
[](https://godoc.org/github.com/optiopay/kafka/v2)
# Kafka
Kafka is Go client library for [Apache Kafka](https://kafka.apache.org/)
server, released under [MIT license](LICENSE]).
Kafka provides minimal abstraction over wire protocol, support for transparent
failover and easy to use blocking API.
* [godoc](https://godoc.org/github.com/optiopay/kafka/v2) generated documentation,
* [code examples](https://godoc.org/github.com/optiopay/kafka/v2#pkg-examples)
## Example
Write all messages from stdin to kafka and print all messages from kafka topic
to stdout.
```go
package main
import (
"bufio"
"log"
"os"
"strings"
"github.com/optiopay/kafka/v2"
"github.com/optiopay/kafka/v2/proto"
)
const (
topic = "my-messages"
partition = 0
)
var kafkaAddrs = []string{"localhost:9092", "localhost:9093"}
// printConsumed read messages from kafka and print them out
func printConsumed(broker kafka.Client) {
conf := kafka.NewConsumerConf(topic, partition)
conf.StartOffset = kafka.StartOffsetNewest
consumer, err := broker.Consumer(conf)
if err != nil {
log.Fatalf("cannot create kafka consumer for %s:%d: %s", topic, partition, err)
}
for {
msg, err := consumer.Consume()
if err != nil {
if err != kafka.ErrNoData {
log.Printf("cannot consume %q topic message: %s", topic, err)
}
break
}
log.Printf("message %d: %s", msg.Offset, msg.Value)
}
log.Print("consumer quit")
}
// produceStdin read stdin and send every non empty line as message
func produceStdin(broker kafka.Client) {
producer := broker.Producer(kafka.NewProducerConf())
input := bufio.NewReader(os.Stdin)
for {
line, err := input.ReadString('\n')
if err != nil {
log.Fatalf("input error: %s", err)
}
line = strings.TrimSpace(line)
if line == "" {
continue
}
msg := &proto.Message{Value: []byte(line)}
if _, err := producer.Produce(topic, partition, msg); err != nil {
log.Fatalf("cannot produce message to %s:%d: %s", topic, partition, err)
}
}
}
func main() {
conf := kafka.NewBrokerConf("test-client")
conf.AllowTopicCreation = true
// connect to kafka cluster
broker, err := kafka.Dial(kafkaAddrs, conf)
if err != nil {
log.Fatalf("cannot connect to kafka cluster: %s", err)
}
defer broker.Close()
go printConsumed(broker)
produceStdin(broker)
}
```