{"id":13413389,"url":"https://github.com/confluentinc/confluent-kafka-go","last_synced_at":"2026-01-05T14:26:40.692Z","repository":{"id":37256669,"uuid":"63196359","full_name":"confluentinc/confluent-kafka-go","owner":"confluentinc","description":"Confluent's Apache Kafka Golang client","archived":false,"fork":false,"pushed_at":"2025-05-12T01:17:19.000Z","size":522469,"stargazers_count":4862,"open_issues_count":281,"forks_count":683,"subscribers_count":381,"default_branch":"master","last_synced_at":"2025-05-12T02:39:01.190Z","etag":null,"topics":["confluent","consumer","golang","golang-bindings","golang-library","kafka-client","librdkafka","producer"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/confluentinc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2016-07-12T22:23:34.000Z","updated_at":"2025-05-11T18:57:27.000Z","dependencies_parsed_at":"2024-11-07T02:35:10.854Z","dependency_job_id":"fec970da-2a97-4674-bed0-6ec64332a78b","html_url":"https://github.com/confluentinc/confluent-kafka-go","commit_stats":{"total_commits":498,"total_committers":79,"mean_commits":6.30379746835443,"dds":0.7389558232931727,"last_synced_commit":"3f6ce098c04c5e7d37f77e59e755822c7affbdc0"},"previous_names":[],"tags_count":54,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/confluentinc%2Fconfluent-kafka-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/confluentinc%2Fconfluent-kafka-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/confluentinc%2Fconfluent-kafka-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/confluentinc%2Fconfluent-kafka-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/confluentinc","download_url":"https://codeload.github.com/confluentinc/confluent-kafka-go/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253672696,"owners_count":21945480,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["confluent","consumer","golang","golang-bindings","golang-library","kafka-client","librdkafka","producer"],"created_at":"2024-07-30T20:01:39.217Z","updated_at":"2026-01-05T14:26:40.687Z","avatar_url":"https://github.com/confluentinc.png","language":"HTML","funding_links":[],"categories":["HTML","消息","Drivers","Messaging","Libraries","消息系统","Relational Databases","Clients","机器学习"],"sub_categories":["检索及分析资料库","Search and Analytic Databases","Go","SQL 查询语句构建库","Advanced Console UIs"],"readme":"Confluent's Golang Client for Apache Kafka\u003csup\u003eTM\u003c/sup\u003e\n=====================================================\n\n**confluent-kafka-go** is Confluent's Golang client for [Apache Kafka](http://kafka.apache.org/) and the\n[Confluent Platform](https://www.confluent.io/product/compare/).\n\n\nFeatures:\n\n- **High performance** - confluent-kafka-go is a lightweight wrapper around\n[librdkafka](https://github.com/confluentinc/librdkafka), a finely tuned C\nclient.\n\n- **Reliability** - There are a lot of details to get right when writing an Apache Kafka\nclient. We get them right in one place (librdkafka) and leverage this work\nacross all of our clients (also [confluent-kafka-python](https://github.com/confluentinc/confluent-kafka-python)\nand [confluent-kafka-dotnet](https://github.com/confluentinc/confluent-kafka-dotnet)).\n\n- **Supported** - Commercial support is offered by\n[Confluent](https://confluent.io/).\n\n- **Future proof** - Confluent, founded by the\noriginal creator/co-creator of Kafka, is building a [streaming platform](https://www.confluent.io/product/compare/)\nwith Apache Kafka at its core. It's high priority for us that client features keep\npace with core Apache Kafka and components of the [Confluent Platform](https://www.confluent.io/product/compare/).\n\n\nThe Golang bindings provides a high-level Producer and Consumer with support\nfor the balanced consumer groups of Apache Kafka 0.9 and above.\n\nSee the [API documentation](http://docs.confluent.io/current/clients/confluent-kafka-go/index.html) for more information.\n\nFor a step-by-step guide on using the client see [Getting Started with Apache Kafka and Golang](https://developer.confluent.io/get-started/go/).\n\n\n\nExamples\n========\n\nHigh-level balanced consumer\n\n```golang\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\t\"github.com/confluentinc/confluent-kafka-go/v2/kafka\"\n)\n\nfunc main() {\n\n\tc, err := kafka.NewConsumer(\u0026kafka.ConfigMap{\n\t\t\"bootstrap.servers\": \"localhost\",\n\t\t\"group.id\":          \"myGroup\",\n\t\t\"auto.offset.reset\": \"earliest\",\n\t})\n\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\terr = c.SubscribeTopics([]string{\"myTopic\", \"^aRegex.*[Tt]opic\"}, nil)\n\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\t// A signal handler or similar could be used to set this to false to break the loop.\n\trun := true\n\n\tfor run {\n\t\tmsg, err := c.ReadMessage(time.Second)\n\t\tif err == nil {\n\t\t\tfmt.Printf(\"Message on %s: %s\\n\", msg.TopicPartition, string(msg.Value))\n\t\t} else if !err.(kafka.Error).IsTimeout() {\n\t\t\t// The client will automatically try to recover from all errors.\n\t\t\t// Timeout is not considered an error because it is raised by\n\t\t\t// ReadMessage in absence of messages.\n\t\t\tfmt.Printf(\"Consumer error: %v (%v)\\n\", err, msg)\n\t\t}\n\t}\n\n\tc.Close()\n}\n```\n\nProducer\n\n```golang\nimport (\n\t\"fmt\"\n\t\"github.com/confluentinc/confluent-kafka-go/v2/kafka\"\n)\n\nfunc main() {\n\n\tp, err := kafka.NewProducer(\u0026kafka.ConfigMap{\"bootstrap.servers\": \"localhost\"})\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\tdefer p.Close()\n\n\t// Delivery report handler for produced messages\n\tgo func() {\n\t\tfor e := range p.Events() {\n\t\t\tswitch ev := e.(type) {\n\t\t\tcase *kafka.Message:\n\t\t\t\tif ev.TopicPartition.Error != nil {\n\t\t\t\t\tfmt.Printf(\"Delivery failed: %v\\n\", ev.TopicPartition)\n\t\t\t\t} else {\n\t\t\t\t\tfmt.Printf(\"Delivered message to %v\\n\", ev.TopicPartition)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}()\n\n\t// Produce messages to topic (asynchronously)\n\ttopic := \"myTopic\"\n\tfor _, word := range []string{\"Welcome\", \"to\", \"the\", \"Confluent\", \"Kafka\", \"Golang\", \"client\"} {\n\t\tp.Produce(\u0026kafka.Message{\n\t\t\tTopicPartition: kafka.TopicPartition{Topic: \u0026topic, Partition: kafka.PartitionAny},\n\t\t\tValue:          []byte(word),\n\t\t}, nil)\n\t}\n\n\t// Wait for message deliveries before shutting down\n\tp.Flush(15 * 1000)\n}\n```\n\nMore elaborate examples are available in the [examples](examples) directory,\nincluding [how to configure](examples/confluent_cloud_example) the Go client\nfor use with [Confluent Cloud](https://www.confluent.io/confluent-cloud/).\n\n\nGetting Started\n===============\n\nSupports Go 1.17+ and librdkafka 2.13.0+.\n\nUsing Go Modules\n----------------\n\nYou can use [Go Modules](https://blog.golang.org/using-go-modules) to install\nconfluent-kafka-go.\n\nImport the `kafka` package from GitHub in your code:\n\n```golang\nimport \"github.com/confluentinc/confluent-kafka-go/v2/kafka\"\n```\n\nBuild your project:\n\n```bash\ngo build ./...\n```\n\nIf you are building for Alpine Linux (musl), `-tags musl` must be specified.\n\n```bash\ngo build -tags musl ./...\n```\n\nA dependency to the latest stable version of confluent-kafka-go should be automatically added to\nyour `go.mod` file.\n\nInstall the client\n------------------\n\nManual install:\n```bash\ngo get -u github.com/confluentinc/confluent-kafka-go/v2/kafka\n```\n\nGolang import:\n```golang\nimport \"github.com/confluentinc/confluent-kafka-go/v2/kafka\"\n```\n\nlibrdkafka\n----------\n\nPrebuilt librdkafka binaries are included with the Go client and librdkafka\ndoes not need to be installed separately on the build or target system.\nThe following platforms are supported by the prebuilt librdkafka binaries:\n\n * Mac OSX x64 and arm64\n * glibc-based Linux x64 and arm64 (e.g., RedHat, Debian, CentOS, Ubuntu, etc) - without GSSAPI/Kerberos support\n * musl-based Linux amd64 and arm64 (Alpine) - without GSSAPI/Kerberos support\n * Windows amd64 - without GSSAPI/Kerberos support\n\nWhen building your application for Alpine Linux (musl libc) you must pass\n`-tags musl` to `go get`, `go build`, etc.\n\n`CGO_ENABLED` must NOT be set to `0` since the Go client is based on the\nC library librdkafka.\n\nIf GSSAPI/Kerberos authentication support is required you will need\nto install librdkafka separately, see the **Installing librdkafka** chapter\nbelow, and then build your Go application with `-tags dynamic`.\n\nInstalling librdkafka\n---------------------\n\nIf the bundled librdkafka build is not supported on your platform, or you\nneed a librdkafka with GSSAPI/Kerberos support, you must install librdkafka\nmanually on the build and target system using one of the following alternatives:\n\n- For Debian and Ubuntu based distros, install `librdkafka-dev` from the standard\nrepositories or using [Confluent's Deb repository](http://docs.confluent.io/current/installation.html#installation-apt).\n- For Redhat based distros, install `librdkafka-devel` using [Confluent's YUM repository](http://docs.confluent.io/current/installation.html#rpm-packages-via-yum).\n- For MacOS X, install `librdkafka` from Homebrew. You may also need to brew install pkg-config if you don't already have it: `brew install librdkafka pkg-config`.\n- For Alpine: `apk add librdkafka-dev pkgconf`\n- For Windows: there are no official/supported packages, but static builds are included for Windows/x64.\n  Installing from source is needed only for GSSAPI/Kerberos support.\n- For source builds, see instructions below.\n\nBuild from source:\n\n    git clone https://github.com/confluentinc/librdkafka.git\n    cd librdkafka\n    ./configure\n    make\n    sudo make install\n\nAfter installing librdkafka you will need to build your Go application\nwith `-tags dynamic`.\n\n**Note:** If you use the `master` branch of the Go client, then you need to use\n          the `master` branch of librdkafka.\n\n**confluent-kafka-go requires librdkafka v1.9.0 or later.**\n\nStatic builds on Linux\n----------------------\n\nSince we are using `cgo`, Go builds a dynamically linked library even when using\nthe prebuilt, statically-compiled librdkafka as described in the **librdkafka**\nchapter.\n\nFor `glibc` based systems, if the system where the client is being compiled is\ndifferent from the target system, especially when the target system is older,\nthere is a `glibc` version error when trying to run the compiled client.\n\nUnfortunately, if we try building a statically linked binary, it doesn't solve the problem,\nsince there is no way to have truly static builds using `glibc`. This is\nbecause there are some functions in `glibc`, like `getaddrinfo` which need the shared\nversion of the library even when the code is compiled statically.\n\nOne way around this is to either use a container/VM to build the binary, or install\nan older version of `glibc` on the system where the client is being compiled.\n\nThe other way is using `musl` to create truly static builds for Linux. To do this,\n[install it for your system](https://wiki.musl-libc.org/getting-started.html).\n\nStatic compilation command, meant to be used alongside the prebuilt librdkafka bundle:\n```bash\nCC=/path/to/musl-gcc go build --ldflags '-linkmode external -extldflags \"-static\"' -tags musl\n```\n\nFIPS 140-3 Compliance\n=====================\n\nThis client supports FIPS 140-3 compliance for Schema Registry operations when using Go 1.24.3 or newer.\n\n## Communication between client and Schema Registry\n\nThe Schema Registry Go client can operate in FIPS 140-3 compliant mode using Go's native FIPS support:\n\n**Build with FIPS support:**\n```bash\nGOFIPS140=inprocess go build -o myapp\n```\n\n**Run in FIPS mode:**\n```bash\nGODEBUG=fips140=only ./myapp\n```\n\nWhen running with `GODEBUG=fips140=only`, the application will use only FIPS 140-3 validated cryptographic implementations for all TLS connections to Schema Registry. The application will panic immediately if any non-FIPS-approved cryptographic operation is attempted.\n\nAPI Strands\n===========\n\nThe recommended API strand is the Function-Based one,\nthe Channel-Based one is documented in [examples/legacy](examples/legacy).\n\nFunction-Based Consumer\n-----------------------\n\nMessages, errors and events are polled through the `consumer.Poll()` function.\n\nIt has direct mapping to underlying librdkafka functionality.\n\nSee [examples/consumer_example](examples/consumer_example)\n\nFunction-Based Producer\n-----------------------\n\nApplication calls `producer.Produce()` to produce messages.\nDelivery reports are emitted on the `producer.Events()` or specified private channel.\n\n_Warnings_\n\n * `Produce()` is a non-blocking call, if the internal librdkafka queue is full\n   the call will fail and can be retried.\n\nSee [examples/producer_example](examples/producer_example)\n\nLicense\n=======\n\n[Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0)\n\nKAFKA is a registered trademark of The Apache Software Foundation and has been licensed for use\nby confluent-kafka-go. confluent-kafka-go has no affiliation with and is not endorsed by The Apache\nSoftware Foundation.\n\nDeveloper Notes\n===============\n\nSee [kafka/README](kafka/README.md)\n\nContributions to the code, examples, documentation, et.al, are very much appreciated.\n\nMake your changes, run `gofmt`, tests, etc, push your branch, create a PR, and [sign the CLA](http://clabot.confluent.io/cla).\n\nConfluent Cloud\n===============\n\nFor a step-by-step guide on using the Golang client with Confluent Cloud see [Getting Started with Apache Kafka and Golang](https://developer.confluent.io/get-started/go/) on [Confluent Developer](https://developer.confluent.io/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconfluentinc%2Fconfluent-kafka-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconfluentinc%2Fconfluent-kafka-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconfluentinc%2Fconfluent-kafka-go/lists"}