{"id":20613892,"url":"https://github.com/python-tools/kafkaproxy","last_synced_at":"2026-06-22T01:31:17.341Z","repository":{"id":65808392,"uuid":"598492408","full_name":"Python-Tools/kafkaproxy","owner":"Python-Tools","description":"kafka的代理对象","archived":false,"fork":false,"pushed_at":"2023-02-11T08:52:25.000Z","size":13,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-26T19:04:46.622Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Python-Tools.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-07T08:14:39.000Z","updated_at":"2023-02-11T08:48:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"f370ad0b-ed58-4767-97f0-25a1b001f225","html_url":"https://github.com/Python-Tools/kafkaproxy","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/Python-Tools/kafkaproxy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Python-Tools%2Fkafkaproxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Python-Tools%2Fkafkaproxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Python-Tools%2Fkafkaproxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Python-Tools%2Fkafkaproxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Python-Tools","download_url":"https://codeload.github.com/Python-Tools/kafkaproxy/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Python-Tools%2Fkafkaproxy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34630770,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-21T02:00:05.568Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T11:11:21.250Z","updated_at":"2026-06-22T01:31:17.329Z","avatar_url":"https://github.com/Python-Tools.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kafkaproxy\n\nkafka生产者和消费者的代理工具.\n\n代理对象用于推迟初始化.我们可以在需要的地方用代理对象的全局变量直接编写逻辑,避免被代理的对象来回在函数间传递.\n\n## 特性\n\n+ 支持代理`kafka-python`,`aiokafka`和`confluent-kafka`的生产者消费者对象.\n+ 提供统一通用的生产者消费者接口包装\n\n## 安装\n\n+ 只安装本项目不安装被代理对象的依赖: `pip install kafkaproxy`\n+ 安装本项目同时确定要代理的对象包为`kafka-python`: `pip install kafkaproxy[kafka]`\n+ 安装本项目同时确定要代理的对象包为`aiokafka`: `pip install kafkaproxy[aio]`\n+ 安装本项目同时确定要代理的对象包为`confluent-kafka`: `pip install kafkaproxy[confluent]`\n\n## 使用\n\n本项目支持代理3种kafka模块中的对应模块,使用枚举`KafkaType`中的取值在调用方法`initialize_from_addresses`初始化时指定.\n代理对象除了原样代理对象外还提供了生产者和消费者的统一通用接口包装.\n由于对应的方法是动态绑定的,因此如果需要他们的typehints可以用`typing.cast`将代理对象转化为对应的协议对象\n\n+ 同步接口生产者使用`ProducerProtocol`\n+ 异步接口生产者使用`AioProducerProtocol`\n+ 同步接消费产者使用`ConsumerProtocol`\n+ 异步接消费产者使用`AioConsumerProtocol`\n\n\u003e 代理`kafka-python`或`confluent-kafka`生产者\n\n```python\nfrom kafkaproxy import ProducerProxy, KafkaType, ProducerProtocol\nfrom typing import cast\nimport time\nkafkap = ProducerProxy()\n\n\ndef run() -\u003e None:\n    p = cast(ProducerProtocol, kafkap)\n    with p.mount() as cli:\n        for i in range(10):\n            cli.publish(\"topic1\", f\"send {i}\")\n            time.sleep(0.1)\n\n\n# kafkap.initialize_from_addresses(\"localhost:9094\", kafka_type=KafkaType.ConfluentKafka, acks=\"all\")\nkafkap.initialize_from_addresses(\"localhost:9094\", kafka_type=KafkaType.Kafka)\ntry:\n    print(\"start publishing\")\n    run()\nfinally:\n    print(\"stoped\")\n```\n\n\u003e 代理`kafka-python`或`confluent-kafka`消费者\n\n```python\nfrom kafkaproxy import ConsumerProxy, KafkaType, ConsumerProtocol\nfrom typing import cast\n\nkafkac = ConsumerProxy()\n\n\ndef run() -\u003e None:\n    c = cast(ConsumerProtocol, kafkac)\n    with c.watch() as g:\n        for record in g:\n            print(record.value)\n\n\n# kafkac.initialize_from_addresses(\"localhost:9094\", \"topic1\", group_id=\"test2\", kafka_type=KafkaType.Kafka)\nkafkac.initialize_from_addresses(\"localhost:9094\", \"topic1\", group_id=\"test2\", kafka_type=KafkaType.ConfluentKafka)\ntry:\n    print(\"start watching\")\n    run()\nfinally:\n    print(\"stoped\")\n\n```\n\n\u003e 代理`aiokafka`生产者\n\n```python\nimport asyncio\nfrom kafkaproxy import ProducerProxy, KafkaType, AioProducerProtocol\nfrom typing import cast\n\nkafkap = ProducerProxy()\n\n\nasync def run() -\u003e None:\n    p = cast(AioProducerProtocol, kafkap)\n    async with p.mount() as cli:\n        for i in range(10):\n            await cli.publish(\"topic1\", f\"send {i}\")\n            await asyncio.sleep(0.1)\n\n\nasync def main() -\u003e None:\n    kafkap.initialize_from_addresses(\"localhost:9094\", kafka_type=KafkaType.AioKafka, acks=\"all\")\n    await run()\n\n\ntry:\n    print(\"start watching\")\n    asyncio.run(main())\nfinally:\n    print(\"stoped\")\n\n```\n\n\u003e 代理`aiokafka`消费者\n\n```python\nimport asyncio\nfrom kafkaproxy import ConsumerProxy, KafkaAutoOffsetReset, KafkaType, AioConsumerProtocol\nfrom typing import cast\n\nkafkac = ConsumerProxy()\n\n\nasync def run() -\u003e None:\n    c = cast(AioConsumerProtocol, kafkac)\n    async with c.watch() as g:\n        async for record in g:\n            print(record.value)\n\n\nasync def main() -\u003e None:\n    kafkac.initialize_from_addresses(\"localhost:9094\", \"topic1\", group_id=\"test2\", kafka_type=KafkaType.AioKafka, auto_offset_reset=KafkaAutoOffsetReset.earliest)\n    await run()\n\n\ntry:\n    print(\"start watching\")\n    asyncio.run(main())\nfinally:\n    print(\"stoped\")\n\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpython-tools%2Fkafkaproxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpython-tools%2Fkafkaproxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpython-tools%2Fkafkaproxy/lists"}