{"id":21577945,"url":"https://github.com/wesleysbmartins/go_kafka","last_synced_at":"2026-05-20T14:03:52.542Z","repository":{"id":250069776,"uuid":"832343389","full_name":"wesleysbmartins/go_kafka","owner":"wesleysbmartins","description":"Este é um exemplo de como uma aplicação Golang pode trabalhar integrada ao Kafka.","archived":false,"fork":false,"pushed_at":"2024-07-27T16:08:20.000Z","size":23,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-24T14:16:04.631Z","etag":null,"topics":["adapters","entities","factory","golang","kafka","kafka-client","kafka-connect","kafka-consumer","kafka-consumer-group","kafka-producer","kafka-topic","sarama","singleton","usecases"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wesleysbmartins.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-22T20:39:44.000Z","updated_at":"2024-07-27T16:11:00.000Z","dependencies_parsed_at":"2024-07-25T02:07:33.158Z","dependency_job_id":"84bd3803-562a-46c3-9ecc-77a7484567d0","html_url":"https://github.com/wesleysbmartins/go_kafka","commit_stats":null,"previous_names":["wesleysbmartins/go_kafka"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wesleysbmartins%2Fgo_kafka","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wesleysbmartins%2Fgo_kafka/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wesleysbmartins%2Fgo_kafka/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wesleysbmartins%2Fgo_kafka/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wesleysbmartins","download_url":"https://codeload.github.com/wesleysbmartins/go_kafka/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244175947,"owners_count":20410780,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adapters","entities","factory","golang","kafka","kafka-client","kafka-connect","kafka-consumer","kafka-consumer-group","kafka-producer","kafka-topic","sarama","singleton","usecases"],"created_at":"2024-11-24T13:09:04.080Z","updated_at":"2026-05-20T14:03:52.427Z","avatar_url":"https://github.com/wesleysbmartins.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Golang [![My Skills](https://skillicons.dev/icons?i=golang)](https://skillicons.dev) Kafka Integration [![My Skills](https://skillicons.dev/icons?i=kafka)](https://skillicons.dev)\nEste é um exemplo de como uma aplicação Golang pode trabalhar integrada ao Kafka.\n\n## Kafka\n\n\nApache Kafka é uma plataforma de streaming de eventos distribuída e altamente escalável, projetada para processar e gerenciar grandes volumes de dados em tempo real. Originalmente desenvolvido pela LinkedIn e posteriormente aberto como um projeto de código aberto pela Apache Software Foundation, o Kafka é amplamente utilizado em diversas indústrias para várias aplicações.\n\n\u003e **OBS:** Para entender melhor o que é o Kafka, como instalar e rodar em seu ambiente, acesse o meu [repositório](https://github.com/wesleysbmartins/kafka) onde registrei meus estudos sobre o tema.\n\n## Hands-On\nNeste momento iremos abordar de forma simples como integra o Kafka a sua aplicação Golang usando a biblioteca [Sarama](https://pkg.go.dev/github.com/IBM/sarama), utilizando padrões de desenvolvimento como **Singleton**, **Factory**, **Entities** e **Usecases**.\n\nApós iniciar sua aplicação Go e instalar a biblioteca do Sarama, você pode criar sua conexão, criando seu Producer ou seu Consumer.\n\n### Client Kafka\nApós inicar sua aplicação podemos iniciar o desenvolvimento da aplicação, neste primeiro momento vamos criar o client do Kafka para sua aplicação e instancia-lo utilizando um Singleton.\n\n```go\npackage kafka\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/IBM/sarama\"\n)\n\ntype Kafka struct {\n\thost string\n\tport string\n}\n\ntype IKafka interface {\n\tConnect()\n}\n\nvar Client sarama.Client\n\nfunc (k *Kafka) Connect() {\n\tif Client == nil {\n\t\tcredentials := Kafka{\n\t\t\thost: \"localhost\",\n\t\t\tport: \"9092\",\n\t\t}\n\n\t\tconfig := sarama.NewConfig()\n\t\tconfig.Version = sarama.MaxVersion\n\t\tconfig.Producer.Return.Successes = true\n\t\tconfig.Consumer.Return.Errors = true\n\t\tconfig.Consumer.Offsets.AutoCommit.Enable = true\n\t\tconfig.Consumer.Offsets.AutoCommit.Interval = 1 * time.Second\n\n\t\tbroker := []string{fmt.Sprintf(\"%s:%s\", credentials.host, credentials.port)}\n\n\t\tclient, err := sarama.NewClient(broker, config)\n\n\t\tif err != nil {\n\t\t\tfmt.Println(\"Client Intance Error!\\n\", err)\n\t\t} else {\n\t\t\tClient = client\n\t\t}\n\t}\n}\n```\n\n### Factory Kafka\nPara a criação dos Producers e Consumers do Kafka vamos utilizar a abordagem de Factory.\n\n### Producer Factory\nPara o factory dos producers retemos os métodos de criação do producer e o de envio de mensagens, onde esperam os parametros necessários para realizar tais operações como tópico, chave da mensagem e valor da mensagem.\n```go\npackage factory\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"go_kafka/internal/adapters/kafka\"\n\n\t\"github.com/IBM/sarama\"\n)\n\ntype ProducerFactory struct {\n\tkey      string\n\ttopic    string\n\tinstance sarama.SyncProducer\n}\n\ntype IProducerFactory interface {\n\tCreate(key string, topic string)\n\tSendMessage(message string) error\n}\n\nfunc (p *ProducerFactory) Create(key string, topic string) {\n\tp.key = key\n\tp.topic = topic\n\tproducer, err := sarama.NewSyncProducerFromClient(kafka.Client)\n\n\tif err != nil {\n\t\tpanic(\"Error to create new Producer!\")\n\t} else {\n\t\tp.instance = producer\n\t}\n}\n\nfunc (p *ProducerFactory) Send(message interface{}) error {\n\tvalue, _ := json.Marshal(message)\n\n\tmsg := \u0026sarama.ProducerMessage{\n\t\tTopic: p.topic,\n\t\tKey:   sarama.StringEncoder(p.key),\n\t\tValue: sarama.StringEncoder(value),\n\t}\n\n\t_, _, err := p.instance.SendMessage(msg)\n\n\tif err != nil {\n\t\tfmt.Println(\"Send message ERROR: \", err)\n\t} else {\n\t\tfmt.Println(\"Send message SUCCESS!\")\n\t}\n\n\treturn err\n}\n```\n\n### Consumers\nTemos dois tipos de implementações de consumers, um consumer de um único tópico e os consumers de grupos de tópicos.\n\nAlém disse foi criado interfaces de handlers para lidarem com as mensagens recebidas, o que representaria sua regra de negócio.\n\n### Consumer Handlers Interfaces\n```go\npackage factory\n\nimport \"github.com/IBM/sarama\"\n\ntype IConsumerHandler interface {\n\tRun(message sarama.ConsumerMessage)\n}\n\ntype IConsumerGroupHandler interface {\n\tSetup(session sarama.ConsumerGroupSession) error\n\tCleanup(session sarama.ConsumerGroupSession) error\n\tConsumeClaim(session sarama.ConsumerGroupSession, claim sarama.ConsumerGroupClaim) error\n}\n```\n\n### Consumer Factory\n```go\npackage factory\n\nimport (\n\t\"fmt\"\n\t\"go_kafka/internal/adapters/kafka\"\n\t\"os\"\n\t\"os/signal\"\n\t\"syscall\"\n\n\t\"github.com/IBM/sarama\"\n)\n\ntype ConsumerFactory struct {\n\ttopic     string\n\tpartition int32\n\tinstance  sarama.Consumer\n}\n\ntype IConsumerFactory interface {\n\tCreate(topic string, partition int32)\n\tListen() error\n}\n\nfunc (c *ConsumerFactory) Create(topic string, partition int32) {\n\tc.topic = topic\n\tc.partition = partition\n\tconsumer, err := sarama.NewConsumerFromClient(kafka.Client)\n\n\tif err != nil {\n\t\tpanic(\"Error to create new Consumer!\")\n\t} else {\n\t\tc.instance = consumer\n\t}\n}\n\nfunc (c *ConsumerFactory) Listen(handler IConsumerHandler) {\n\tconsumerPartition, err := c.instance.ConsumePartition(c.topic, c.partition, sarama.OffsetOldest)\n\n\tif err != nil {\n\t\tfmt.Println(\"Consumer ERROR: \", err)\n\t}\n\n\tdefer consumerPartition.Close()\n\n\tch := make(chan os.Signal, 1)\n\n\tsignal.Notify(ch, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)\n\n\tmessages := consumerPartition.Messages()\n\n\tfor {\n\t\tselect {\n\t\tcase msg := \u003c-ch:\n\t\t\tfmt.Printf(\"Reveived message SIGNALL: %v\\n\", msg)\n\t\t\treturn\n\t\tcase msg := \u003c-messages:\n\t\t\thandler.Run(*msg)\n\t\t}\n\t}\n}\n```\n\n### Consumer Group Factory\n```go\npackage factory\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"go_kafka/internal/adapters/kafka\"\n\t\"os\"\n\t\"os/signal\"\n\t\"syscall\"\n\n\t\"github.com/IBM/sarama\"\n)\n\ntype ConsumerGroupFactory struct {\n\tgroupId  string\n\ttopics   []string\n\tinstance sarama.ConsumerGroup\n}\n\ntype IConsumerGroupfactory interface {\n\tCreate(groupId string, topics []string)\n\tListen(handler IConsumerGroupHandler)\n}\n\nfunc (c *ConsumerGroupFactory) Create(groupId string, topics []string) {\n\tc.groupId = groupId\n\tc.topics = topics\n\tconsumerGroup, err := sarama.NewConsumerGroupFromClient(groupId, kafka.Client)\n\tif err != nil {\n\t\tpanic(\"Error to create Consumer Group!\")\n\t} else {\n\t\tc.instance = consumerGroup\n\t}\n}\n\nfunc (c *ConsumerGroupFactory) Listen(handler IConsumerGroupHandler) {\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\n\tgo func() {\n\t\tfor {\n\n\t\t\tif err := c.instance.Consume(ctx, c.topics, handler); err != nil {\n\t\t\t\tpanic(fmt.Sprintf(\"Consumer Group Error!\\n%s\", err.Error()))\n\t\t\t}\n\n\t\t\tif ctx.Err() != nil {\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}()\n\n\tch := make(chan os.Signal, 1)\n\n\tsignal.Notify(ch, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)\n\n\tselect {\n\tcase \u003c-ctx.Done():\n\t\tfmt.Println(\"Context Cancelled!\")\n\tcase signal := \u003c-ch:\n\t\tfmt.Printf(\"Signal Event: %v\\n\", signal)\n\t}\n}\n```\n\n## Entities\nPara exemplificar uma mensagem foi criado uma struct **Activity**.\n```go\npackage entities\n\ntype Activity struct {\n\tId          string `json:\"id\"`\n\tTitle       string `json:\"title\"`\n\tDescription string `json:\"description\"`\n}\n```\n\n### Usecases\nStructs e métodos responsáveis por implementar as interfaces esperadas pelos consumers como handler e também, sua regra de negócio.\n\n### Consumer Usecase\n```go\npackage usecases\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/IBM/sarama\"\n)\n\ntype ConsumerUsecase struct{}\n\nfunc (c *ConsumerUsecase) Run(message sarama.ConsumerMessage) {\n\tfmt.Printf(\"Consumer Usecase Received Message - Topic: %q - Value: %s\\n\", message.Topic, message.Value)\n}\n```\n\n### Consumer Group Usecase\n```go\npackage usecases\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/IBM/sarama\"\n)\n\ntype ConsumerGroupUsecase struct{}\n\nfunc (ConsumerGroupUsecase) Setup(_ sarama.ConsumerGroupSession) error {\n\treturn nil\n}\n\nfunc (ConsumerGroupUsecase) Cleanup(_ sarama.ConsumerGroupSession) error {\n\treturn nil\n}\n\nfunc (h ConsumerGroupUsecase) ConsumeClaim(session sarama.ConsumerGroupSession, claim sarama.ConsumerGroupClaim) error {\n\tfor msg := range claim.Messages() {\n\n\t\tfmt.Printf(\"Consumer Group Usecase Received Message - Topic: %q - Value: %s\\n\", msg.Topic, msg.Value)\n\n\t\tsession.MarkMessage(msg, \"Readed\")\n\t}\n\n\treturn nil\n}\n```\n### Main\nSendo assim, basta brincar com o código desenvolvido até aqui, use isto no seu main, execute e veja a mágica acontecer:\n```go\npackage main\n\nimport (\n\t\"go_kafka/internal/adapters/kafka\"\n\tfactory_consumer \"go_kafka/internal/adapters/kafka/factory/consumer\"\n\tfactory_producer \"go_kafka/internal/adapters/kafka/factory/producer\"\n\t\"go_kafka/internal/entities\"\n\t\"go_kafka/internal/usecases\"\n)\n\nfunc init() {\n\tkafka_client := \u0026kafka.Kafka{}\n\tkafka_client.Connect()\n}\n\nconst (\n\tpartition = 0\n\tgroupId   = \"activity-group\"\n\ttopic     = \"activity-topic\"\n\tkey       = \"activity\"\n)\n\nfunc main() {\n\tactivityProducer := \u0026factory_producer.ProducerFactory{}\n\tactivityProducer.Create(key, topic)\n\n\t// activityConsumer := \u0026factory_consumer.ConsumerFactory{}\n\t// activityConsumer.Create(topic, partition)\n\n\tconsumerGroup := \u0026factory_consumer.ConsumerGroupFactory{}\n\tconsumerGroup.Create(groupId, []string{topic})\n\n\tconsumerGrouphandler := \u0026usecases.ConsumerGroupUsecase{}\n\n\tconsumerGroup.Listen(consumerGrouphandler)\n\n\tactivities := []entities.Activity{\n\t\t{\n\t\t\tId:          \"1\",\n\t\t\tTitle:       \"Example 01\",\n\t\t\tDescription: \"KAFKA IMPLEMENTATION DESCRIPTION 01\",\n\t\t}, {\n\t\t\tId:          \"2\",\n\t\t\tTitle:       \"Example 02\",\n\t\t\tDescription: \"KAFKA IMPLEMENTATION DESCRIPTION 02\",\n\t\t},\n\t\t{\n\t\t\tId:          \"3\",\n\t\t\tTitle:       \"Example 03\",\n\t\t\tDescription: \"KAFKA IMPLEMENTATION DESCRIPTION 03\",\n\t\t},\n\t}\n\n\tfor _, activity := range activities {\n\t\tactivityProducer.Send(activity)\n\t}\n\n\t// consumerUsecase := \u0026usecases.ConsumerUsecase{}\n\n\t// activityConsumer.Listen(consumerUsecase)\n}\n```\n\n### Resultado\nExecutando o projeto, você deve obter um resultado semelhante a este:\n```shell\nConsumer Group Usecase Received Message - Topic: \"activity-topic\" - Value: {\"id\":\"1\",\"title\":\"Example 01\",\"description\":\"KAFKA IMPLEMENTATION DESCRIPTION 01\"}\nConsumer Group Usecase Received Message - Topic: \"activity-topic\" - Value: {\"id\":\"2\",\"title\":\"Example 02\",\"description\":\"KAFKA IMPLEMENTATION DESCRIPTION 02\"}\nConsumer Group Usecase Received Message - Topic: \"activity-topic\" - Value: {\"id\":\"3\",\"title\":\"Example 03\",\"description\":\"KAFKA IMPLEMENTATION DESCRIPTION 03\"}\nConsumer Group Usecase Received Message - Topic: \"activity-topic\" - Value: {\"id\":\"1\",\"title\":\"Example 01\",\"description\":\"KAFKA IMPLEMENTATION DESCRIPTION 01\"}\nConsumer Group Usecase Received Message - Topic: \"activity-topic\" - Value: {\"id\":\"2\",\"title\":\"Example 02\",\"description\":\"KAFKA IMPLEMENTATION DESCRIPTION 02\"}\nConsumer Group Usecase Received Message - Topic: \"activity-topic\" - Value: {\"id\":\"3\",\"title\":\"Example 03\",\"description\":\"KAFKA IMPLEMENTATION DESCRIPTION 03\"}\nSignal Event: interrupt\nSend message SUCCESS!\nSend message SUCCESS!\nSend message SUCCESS!\n```\n\nPara parar a aplicação basta pressionar **CTRL + C** em seu terminal:\n```shell\nReveived message SIGNALL: interrupt\n```\n\nAssim voce tem uma aplicação Golang integrada ao Kafka, sendo capaz de realizar operações de envio ou leitura de mensagens ou eventos.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwesleysbmartins%2Fgo_kafka","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwesleysbmartins%2Fgo_kafka","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwesleysbmartins%2Fgo_kafka/lists"}