{"id":15710400,"url":"https://github.com/aws/go-kafka-event-source","last_synced_at":"2025-03-17T23:31:33.761Z","repository":{"id":61197953,"uuid":"529953629","full_name":"aws/go-kafka-event-source","owner":"aws","description":"Go/Kafka client library for developing event sourcing applications","archived":false,"fork":false,"pushed_at":"2023-12-18T22:58:12.000Z","size":305,"stargazers_count":12,"open_issues_count":2,"forks_count":4,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-15T23:44:17.124Z","etag":null,"topics":["event-sourcing","go","golang","kafka","kafka-consumer","kafka-producer","kafka-streams"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aws.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-28T19:02:04.000Z","updated_at":"2025-01-02T10:20:58.000Z","dependencies_parsed_at":"2024-06-19T23:04:40.087Z","dependency_job_id":null,"html_url":"https://github.com/aws/go-kafka-event-source","commit_stats":{"total_commits":82,"total_committers":2,"mean_commits":41.0,"dds":"0.012195121951219523","last_synced_commit":"d07f4a5fe53f48f5fa5ea3703c698b7ac3e3ff90"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws%2Fgo-kafka-event-source","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws%2Fgo-kafka-event-source/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws%2Fgo-kafka-event-source/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws%2Fgo-kafka-event-source/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aws","download_url":"https://codeload.github.com/aws/go-kafka-event-source/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243891962,"owners_count":20364611,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["event-sourcing","go","golang","kafka","kafka-consumer","kafka-producer","kafka-streams"],"created_at":"2024-10-03T21:07:14.987Z","updated_at":"2025-03-17T23:31:33.419Z","avatar_url":"https://github.com/aws.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"## Go Kafka Event Source\n\nGKES (Go Kafka Event Source) attempts to fill the gaps ub the Go/Kafka library ecosystem. It supplies EOS (Exactly Once Semantics),\nlocal state stores and incremental consumer rebalancing to Go Kafka consumer applications, making it a viable alternative to\na traditional Kafka Streams application written in Java.\n\n## Getting Started\n\nThis repository is organized into several modules. The primary module being \"streams\".\nTo use GKES in your go program, simply add it as you would any other Go module.\n\n```\ngo get github.com/aws/go-kafka-event-source/streams\n```\n\nIf you are using AWS MSK, you may find the provided \"msk\" module useful for cluster connectivity, though this is not required.\n\n```\ngo get github.com/aws/go-kafka-event-source/msk\n```\n\nFor API documentation on these modules, see https://pkg.go.dev/github.com/aws/go-kafka-event-source/streams and https://pkg.go.dev/github.com/aws/go-kafka-event-source/msk. The [docs](./docs) folder also contains detailed examples.\n\n## Project Status\n\nGKES is usable in it's current form but there are a few things slated for the very near future. This library is being used by AWS for internal workloads.\n\n- [X] More comprehensive documentation for and async processing - [Complete](./docs/asyncprocessing.md) \n- [ ] More comprehensive documentation for StateStores - In Progress\n- [X] Per topic partitioner support for EventContext.Forward() and Producer/BatchProducer APIs, Currently, only the Java default murmur2 partitioner is supported - Complete\n- [ ] Instructions for testing locally - In progress\n- [ ] More robust functional tests for async processing. Though this is tested extenisively using a local test harness, this process needs to be more repeatable and available to contributors - In progress\n\n## Compatibility Notes\n\nIt is recommnded tp use Go v1.19.2 or greater for GKES. There was a known compiler issue in previous versions of Go 1.19 which prevented modules using GKES from compiling. There was a back-port fix made to previous versions of Go, but it probably simpler and safer to update your Go environment to the latest available if you run into this issue.\n\nGKES has been extensively test with Kafka 3.2 but should be fine to use with any Kafka version \u003e 2.5.1. Kafka versions \u003c 2.5.1 are not likely to be compatible with GKES due to transaction semantics, and they have not been tested.\n\n## Security\n\nSee [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.\n\n## License\n\nThis project is licensed under the Apache-2.0 License.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faws%2Fgo-kafka-event-source","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faws%2Fgo-kafka-event-source","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faws%2Fgo-kafka-event-source/lists"}