{"id":15732976,"url":"https://github.com/pastuhov/python-kafka-groupbykey","last_synced_at":"2025-08-07T04:35:50.990Z","repository":{"id":93278937,"uuid":"350724394","full_name":"pastuhov/python-kafka-groupbykey","owner":"pastuhov","description":null,"archived":false,"fork":false,"pushed_at":"2021-03-24T06:20:32.000Z","size":3,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-23T11:36:42.981Z","etag":null,"topics":["aggregation","kafka","stateful","streams"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pastuhov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-23T13:32:40.000Z","updated_at":"2021-03-24T06:44:45.000Z","dependencies_parsed_at":"2023-03-15T16:00:39.511Z","dependency_job_id":null,"html_url":"https://github.com/pastuhov/python-kafka-groupbykey","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pastuhov%2Fpython-kafka-groupbykey","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pastuhov%2Fpython-kafka-groupbykey/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pastuhov%2Fpython-kafka-groupbykey/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pastuhov%2Fpython-kafka-groupbykey/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pastuhov","download_url":"https://codeload.github.com/pastuhov/python-kafka-groupbykey/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246413229,"owners_count":20773053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aggregation","kafka","stateful","streams"],"created_at":"2024-10-04T00:40:28.369Z","updated_at":"2025-03-31T03:43:51.754Z","avatar_url":"https://github.com/pastuhov.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# python-kafka-groupbykey\n\nRealizes Kafka streams code: \n```java\nbuilder.stream(\"commit_log_topic\").groupByKey().aggregate(...).toStream().to(\"snapshot_topic\");\n```\n\n## Example\n\n```python\nfrom confluent_kafka import SerializingProducer, DeserializingConsumer\nfrom http_server import start_server\nfrom reducer import Reducer\nfrom store import Store\nfrom reduced import Reduced\nfrom configuration import Configuration\nimport sys\n\nconfig = Configuration(\n    commit_log_topic='commit_log_topic_name',\n    snapshot_topic='snapshot_topic_topic_name',\n    bootstrap_servers='localhost',\n    group_id='app_name',\n)\n\ndef reduce(accumulator, current):\n    if accumulator is None:\n        accumulator = Reduced()\n        accumulator.value = {}\n    \n    # your reduce code\n\n    return accumulator\n\ns = Store(DeserializingConsumer(config.store_consumer), config.snapshot_topic)\ns.hydrate()\n\nstart_server(s, 8333)\n\nr = Reducer(\n    reduce,\n    DeserializingConsumer(config.consumer),\n    SerializingProducer(config.producer),\n    s,\n    config.commit_log_topic,\n    config.snapshot_topic,\n    config.batch_timeout_sec,\n    config.messages_per_transaction)\nr.process()\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpastuhov%2Fpython-kafka-groupbykey","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpastuhov%2Fpython-kafka-groupbykey","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpastuhov%2Fpython-kafka-groupbykey/lists"}