{"id":43752779,"url":"https://github.com/radekg/kafka-protocol-go","last_synced_at":"2026-02-05T13:35:49.585Z","repository":{"id":38329036,"uuid":"498872585","full_name":"radekg/kafka-protocol-go","owner":"radekg","description":"Apache Kafka protocol decoder and encoder library for go and tinygo","archived":false,"fork":false,"pushed_at":"2022-06-06T22:19:17.000Z","size":769,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-06-21T13:01:09.713Z","etag":null,"topics":["golang","kafka","protocol","tinygo"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/radekg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-01T19:30:23.000Z","updated_at":"2023-08-17T22:13:43.000Z","dependencies_parsed_at":"2022-08-25T03:53:17.397Z","dependency_job_id":null,"html_url":"https://github.com/radekg/kafka-protocol-go","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/radekg/kafka-protocol-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radekg%2Fkafka-protocol-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radekg%2Fkafka-protocol-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radekg%2Fkafka-protocol-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radekg%2Fkafka-protocol-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/radekg","download_url":"https://codeload.github.com/radekg/kafka-protocol-go/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/radekg%2Fkafka-protocol-go/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29122654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-05T10:47:47.471Z","status":"ssl_error","status_checked_at":"2026-02-05T10:45:08.119Z","response_time":65,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","kafka","protocol","tinygo"],"created_at":"2026-02-05T13:35:48.836Z","updated_at":"2026-02-05T13:35:49.578Z","avatar_url":"https://github.com/radekg.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Apache Kafka® Protocol for go and tinygo\n\nThis is a fork of the Apache Kafka® protocol implementation from [grepplabs/kafka-proxy](https://github.com/grepplabs/kafka-proxy). This fantastic work is a great donor because it:\n\n- Uses minimal reflection.\n- Does not use struct tags.\n- Does not use goroutines.\n- Does not use channels.\n\nWhich results in:\n\n- Being able to compile with [tinygo-org/tinygo](https://github.com/tinygo-org/tinygo).\n- Because it can be compiled with _tinygo_, it can be compiled with [tetratelabs/proxy-wasm-go-sdk](https://github.com/tetratelabs/proxy-wasm-go-sdk).\n  - Which means that it can be used as a foundation for Envoy WebAssembly Layer 7 proxies.\n\n## Why forking?\n\nThe upstream Kafka Proxy protocol implements sufficient encoding and decoding to support its own functionality, but does not implement everything else required to parse messages beyond request headers, response headers, `Metadata`, and `FindCoordinator`. The maintainer of Kafka Proxy isn't keen on supporting anything in the protocol encoder/decoder that does not find a direct use in the upstream project.\n\nThe goal of this fork is to add support for all Apache Kafka® messages.\n\n## Message support\n\n☑ Headers\n\n| Requests                             | Responses                            |\n| ------------------------------------ | ------------------------------------ |\n| ☑ `0`: Produce                       | ☑ `0`: Produce                       |\n| ☑ `1`: Fetch                         | ☑ `1`: Fetch                         |\n| ☑ `2`: ListOffsets                   | ☑ `2`: ListOffsets                   |\n| ☑ `3`: Metadata                      | ☑ `3`: Metadata                      |\n| ☑ `3`: Metadata                      | ☑ `3`: Metadata                      |\n| ☑ `5`: StopReplica                   | ☑ `5`: StopReplica                   |\n| ☑ `6`: UpdateMetadata                | ☑ `6`: UpdateMetadata                |\n| ☑ `7`: ControlledShutdown            | ☑ `7`: ControlledShutdown            |\n| ☑ `8`: OffsetCommit                  | ☑ `8`: OffsetCommit                  |\n| ☑ `9`: OffsetFetch                   | ☑ `9`: OffsetFetch                   |\n| ☑ `10`: FindCoordinator              | ☑ `10`: FindCoordinator              |\n| ☑ `11`: JoinGroup                    | ☑ `11`: JoinGroup                    |\n| ☑ `12`: Heartbeat                    | ☑ `12`: Heartbeat                    |\n| ☑ `13`: LeaveGroup                   | ☑ `13`: LeaveGroup                   |\n| ☑ `14`: SyncGroup                    | ☑ `14`: SyncGroup                    |\n| ☑ `15`: DescribeGroups               | ☑ `15`: DescribeGroups               |\n| ☑ `16`: ListGroups                   | ☑ `16`: ListGroups                   |\n| ☑ `17`: SaslHandshake                | ☑ `17`: SaslHandshake                |\n| ☑ `18`: ApiVersions                  | ☑ `18`: ApiVersions                  |\n| ☑ `19`: CreateTopics                 | ☑ `19`: CreateTopics                 |\n| ☑ `20`: DeleteTopics                 | ☑ `20`: DeleteTopics                 |\n| ☑ `21`: DeleteRecords                | ☑ `21`: DeleteRecords                |\n| ☑ `22`: InitProducerId               | ☑ `22`: InitProducerId               |\n| ☑ `23`: OffsetForLeaderEpoch         | ☑ `23`: OffsetForLeaderEpoch         |\n| ☑ `24`: AddPartitionsToTxn           | ☑ `24`: AddPartitionsToTxn           |\n| ☑ `25`: AddOffsetsToTxn              | ☑ `25`: AddOffsetsToTxn              |\n| ☑ `26`: EndTxn                       | ☑ `26`: EndTxn                       |\n| ☑ `27`: WriteTxnMarkers              | ☑ `27`: WriteTxnMarkers              |\n| ☑ `28`: TxnOffsetCommit              | ☑ `28`: TxnOffsetCommit              |\n| ☑ `29`: DescribeAcls                 | ☑ `29`: DescribeAcls                 |\n| ☑ `30`: CreateAcls                   | ☑ `30`: CreateAcls                   |\n| ☑ `31`: DeleteAcls                   | ☑ `31`: DeleteAcls                   |\n| ☑ `32`: DescribeConfigs              | ☑ `32`: DescribeConfigs              |\n| ☑ `33`: AlterConfigs                 | ☑ `33`: AlterConfigs                 |\n| ☑ `34`: AlterReplicaLogDirs          | ☑ `34`: AlterReplicaLogDirs          |\n| ☑ `35`: DescribeLogDirs              | ☑ `35`: DescribeLogDirs              |\n| ☑ `36`: SaslAuthenticate             | ☑ `36`: SaslAuthenticate             |\n| ☑ `37`: CreatePartitions             | ☑ `37`: CreatePartitions             |\n| ☑ `38`: CreateDelegationToken        | ☑ `38`: CreateDelegationToken        |\n| ☑ `39`: RenewDelegationToken         | ☑ `39`: RenewDelegationToken         |\n| ☑ `40`: ExpireDelegationToken        | ☑ `40`: ExpireDelegationToken        |\n| ☑ `41`: DescribeDelegationToken      | ☑ `41`: DescribeDelegationToken      |\n| ☑ `42`: DeleteGroups                 | ☑ `42`: DeleteGroups                 |\n| ☑ `43`: ElectLeaders                 | ☑ `43`: ElectLeaders                 |\n| ☑ `44`: IncrementalAlterConfigs      | ☑ `44`: IncrementalAlterConfigs      |\n| ☑ `45`: AlterPartitionReassignments  | ☑ `45`: AlterPartitionReassignments  |\n| ☑ `46`: ListPartitionReassignments   | ☑ `46`: ListPartitionReassignments   |\n| ☑ `47`: OffsetDelete                 | ☑ `47`: OffsetDelete                 |\n| ☑ `48`: DescribeClientQuotas         | ☑ `48`: DescribeClientQuotas         |\n| ☑ `49`: AlterClientQuotas            | ☑ `49`: AlterClientQuotas            |\n| ☑ `50`: DescribeUserScramCredentials | ☑ `50`: DescribeUserScramCredentials |\n| ☑ `51`: AlterUserScramCredentials    | ☑ `51`: AlterUserScramCredentials    |\n| ☑ `56`: AlterPartition               | ☑ `56`: AlterPartition               |\n| ☑ `57`: UpdateFeatures               | ☑ `57`: UpdateFeatures               |\n| ☑ `60`: DescribeCluster              | ☑ `60`: DescribeCluster              |\n| ☑ `61`: DescribeProducers            | ☑ `61`: DescribeProducers            |\n| ☑ `65`: DescribeTransactions         | ☑ `65`: DescribeTransactions         |\n| ☑ `66`: ListTransactions             | ☑ `66`: ListTransactions             |\n| ☑ `67`: AllocateProducerIds          | ☑ `67`: AllocateProducerIds          |\n\n### General notes\n\n- Compact arrays have been added in Apache Kafka® 2.4.\n- Flexible messages support tag buffers. Flexibility flag is declared per message, any structure schema type includes a tag buffer.\n- Strings and arrays for flexible messages by default use compact versions of their respective types; reference: https://github.com/apache/kafka/blob/3.2.0/generator/src/main/java/org/apache/kafka/message/SchemaGenerator.java#L224-L322.\n\n## License\n\nApache 2.0.\n\n## Run tests\n\n```sh\ngo test -count=1 -v ./...\n```\n\n## Generating messages\n\n1. Setup working environment:\n\n```sh\nexport KAFKA_SOURCE_ROOT=\"${HOME}/dev/my/kafka\"\n```\n\n2. Close Kafka sources somewhere on disk and checkout the release you want to use:\n\n```sh\nmkdir -p \"${KAFKA_SOURCE_ROOT}\"\ncd \"${KAFKA_SOURCE_ROOT}\"\ngit clone https://github.com/apache/kafka.git .\ngit checkout 3.2.0\ncd -\n```\n\n3. Run the generator command:\n\n```sh\nmake generate-request-types generate-response-types\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradekg%2Fkafka-protocol-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fradekg%2Fkafka-protocol-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fradekg%2Fkafka-protocol-go/lists"}