{"id":26725665,"url":"https://github.com/tansu-io/example-kafka-python","last_synced_at":"2026-05-02T13:33:48.695Z","repository":{"id":279774380,"uuid":"939923657","full_name":"tansu-io/example-kafka-python","owner":"tansu-io","description":"Using the Kafka Python client to send and receive messages to Tansu a Kafka compatible broker with S3 or PostgreSQL storage","archived":false,"fork":false,"pushed_at":"2025-11-21T15:18:53.000Z","size":39,"stargazers_count":1,"open_issues_count":2,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-09T00:12:50.299Z","etag":null,"topics":["kafka","postgresql","python","s3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tansu-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-27T10:29:53.000Z","updated_at":"2026-01-11T12:01:31.000Z","dependencies_parsed_at":"2025-02-27T14:39:25.471Z","dependency_job_id":"229e4ddf-0581-466a-a293-087fc513d73c","html_url":"https://github.com/tansu-io/example-kafka-python","commit_stats":null,"previous_names":["tansu-io/example-kafka-python"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tansu-io/example-kafka-python","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tansu-io%2Fexample-kafka-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tansu-io%2Fexample-kafka-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tansu-io%2Fexample-kafka-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tansu-io%2Fexample-kafka-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tansu-io","download_url":"https://codeload.github.com/tansu-io/example-kafka-python/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tansu-io%2Fexample-kafka-python/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32536578,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-02T12:25:33.646Z","status":"ssl_error","status_checked_at":"2026-05-02T12:24:51.733Z","response_time":132,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka","postgresql","python","s3"],"created_at":"2025-03-27T21:19:57.399Z","updated_at":"2026-05-02T13:33:48.675Z","avatar_url":"https://github.com/tansu-io.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kafka Python client with Tansu\n\n[Tansu](https://tansu.io) is an Apache Kafka compatible broker that stores data\nin S3 or PostgreSQL. This example uses the\n[kafka-python](https://github.com/dpkp/kafka-python) client,\nto send and receive messages with [Tansu](https://tansu.io).\n\nIn this example, [MinIO](https://min.io), [PostgreSQL](https://www.postgresql.org)\nand [Tansu](https://tansu.io) run in [Docker](https://docs.docker.com/desktop/)\n[Compose](https://docs.docker.com/compose/)\nfor easy installation and setup.\n\nStart off by cloning this repository:\n\n```shell\ngit clone https://github.com/tansu-io/example-kafka-python.git\ncd example-kafka-python\n```\n\nCopy `example.env` into `.env` so that you have a local working copy:\n\n```shell\ncp example.env .env\n```\n\nTo use S3 with [MinIO](https://min.io), uncomment this line and comment out\nother `STORAGE_ENGINE` definitions in your `.env`:\n\n```\nSTORAGE_ENGINE=\"s3://tansu/\"\n```\n\nTo use [PostgreSQL](https://www.postgresql.org), uncomment this line and\ncomment out other `STORAGE_ENGINE` definitions in your `.env`:\n\n```\nSTORAGE_ENGINE=\"postgres://postgres:postgres@db\"\n```\n\nNow, start up Minio S3, PostgreSQL and Tansu:\n\n```shell\ndocker compose up -d\n```\n\n[Tansu's](https://tansu.io) PostgreSQL [schema](etc/initdb.d/010-schema.sql) is set up in [compose.yaml](compose.yaml),\nrequiring no extra configuration. If you're using MinIO, there are 3 extra steps.\n\nWait for MinIO to become ready:\n\n```shell\ndocker compose exec minio /usr/bin/mc ready local\n```\n\nThen, setup some default MinIO credentials:\n\n```shell\ndocker compose exec minio /usr/bin/mc alias set local http://localhost:9000 minioadmin minioadmin\n```\n\nFinally, create a bucket on MinIO for Tansu:\n\n```shell\ndocker compose exec minio /usr/bin/mc mb local/tansu\n```\n\nUsing [uv](https://docs.astral.sh/uv/), sync the project:\n\n```shell\nuv sync --all-groups\n```\n\nRun the example:\n\n```shell\nuv run example.py\n```\n\nYou should see the following output:\n\n```\nConsumerRecord(topic='my-topic', partition=0, offset=0, timestamp=1740584217091, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=1, timestamp=1740584217091, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=2, timestamp=1740584218091, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=3, timestamp=1740584218092, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=4, timestamp=1740584219092, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=5, timestamp=1740584219092, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=6, timestamp=1740584220092, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=7, timestamp=1740584220092, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=8, timestamp=1740584221092, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=9, timestamp=1740584221093, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=10, timestamp=1740584222093, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=11, timestamp=1740584222093, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=12, timestamp=1740584223093, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=13, timestamp=1740584223093, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=14, timestamp=1740584224094, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=15, timestamp=1740584224094, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=16, timestamp=1740584225094, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=17, timestamp=1740584225094, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=18, timestamp=1740584226094, timestamp_type=0, key=None, value=b'test', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=4, serialized_header_size=-1)\nConsumerRecord(topic='my-topic', partition=0, offset=19, timestamp=1740584226095, timestamp_type=0, key=None, value=b'\\xc2Hola, mundo!', headers=[], checksum=None, serialized_key_size=-1, serialized_value_size=13, serialized_header_size=-1)\n```\n\nIn this example we have used the [kafka-python](https://github.com/dpkp/kafka-python)\nclient to send and receive messages to Tansu a Kafka compatible broker using\nS3 or PostgreSQL for storage.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftansu-io%2Fexample-kafka-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftansu-io%2Fexample-kafka-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftansu-io%2Fexample-kafka-python/lists"}