{"id":14983875,"url":"https://github.com/limmer-laboratories/kafka-bundle","last_synced_at":"2025-07-13T20:31:48.825Z","repository":{"id":214381344,"uuid":"734704743","full_name":"limmer-laboratories/kafka-bundle","owner":"limmer-laboratories","description":"A symfony bundle for working with Kafka","archived":false,"fork":false,"pushed_at":"2025-04-02T05:44:36.000Z","size":41,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-06T20:16:40.816Z","etag":null,"topics":["kafka","kafka-consumer","kafka-producer","php8","symfony-bundle","symfony6"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/limmer-laboratories.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-12-22T11:39:11.000Z","updated_at":"2025-04-02T05:42:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"8b028649-8f95-4a30-99b9-a090a34fa05b","html_url":"https://github.com/limmer-laboratories/kafka-bundle","commit_stats":{"total_commits":25,"total_committers":2,"mean_commits":12.5,"dds":"0.040000000000000036","last_synced_commit":"e8eec374e426214de2353296f331700478b59583"},"previous_names":["limmer-laboratories/kafka-bundle"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/limmer-laboratories/kafka-bundle","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/limmer-laboratories%2Fkafka-bundle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/limmer-laboratories%2Fkafka-bundle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/limmer-laboratories%2Fkafka-bundle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/limmer-laboratories%2Fkafka-bundle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/limmer-laboratories","download_url":"https://codeload.github.com/limmer-laboratories/kafka-bundle/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/limmer-laboratories%2Fkafka-bundle/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265199808,"owners_count":23726718,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kafka","kafka-consumer","kafka-producer","php8","symfony-bundle","symfony6"],"created_at":"2024-09-24T14:08:06.403Z","updated_at":"2025-07-13T20:31:48.502Z","avatar_url":"https://github.com/limmer-laboratories.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# A Symfony Kafka Bundle\n- [Installation](#installation)\n- [Usage](#usage)\n  - [Configuration](#configuration)\n  - [Sending messages to kafka](#sending-messages-to-kafka)\n  - [Consuming kafka topics](#consuming-kafka-topics)\n    - [Implementing a consumer](#implementing-a-consumer)\n    - [Executing a consumer](#executing-a-consumer)\n\n# Installation\nTo install the symfony kafka bundle just execute the following composer command:\n```shell\ncomposer require limlabs/kafka-bundle\n```\n# Usage\n## Configuration\nTo use the kafka bundle you have to configure at least the `default` connection.  \nExample of such a configuration:\n```yaml\n# config/packages/kafka.yaml\n\nkafka:\n  clients:\n    default:\n      brokers: 'kafka:9092' #A comma separated list of kafka brokers\n      log_level: LOG_DEBUG #The rdkafka log level (PHP-Constants)\n      debug: all #Which debug information should be printed by rdkafka. You can remove this to disable it\n```\nIf you have multiple specific kafka connections, you can add multiple `clients` to this configuration.\n```yaml\n# config/packages/kafka.yaml\n\nkafka:\n  clients:\n    default:\n      brokers: 'kafka:9092'\n      log_level: LOG_DEBUG\n      debug: all\n    different_client:\n      brokers: 'kafka2:9092,kafka3:9092'\n      log_level: LOG_ERROR\n```\n\n## Sending messages to kafka\nTo send message to a topic over the default configuration you can simply use the factory to get the `KafkaClient`. \nFrom the `KafkaClient` you can get the `producer` for your kafka connection. With this producer you can create a `topic` in which you can send a message with the `produce` function.\nThe `createTopic` automatically gets an existing topic or creates a new one, if the specified topic does not exist. \nHere an example of this in a symfony controller:\n```php\nnamespace App\\Controller;\n\nuse LimLabs\\KafkaBundle\\Factory\\KafkaFactory;\nuse Symfony\\Bundle\\FrameworkBundle\\Controller\\AbstractController;\nuse Symfony\\Component\\HttpFoundation\\JsonResponse;\nuse Symfony\\Component\\Routing\\Annotation\\Route;\n\nclass ExampleController extends AbstractController\n{\n    public function __construct(\n        private readonly KafkaFactory  $kafkaFactory\n    ) {\n    }\n\n    #[Route('/example', name: 'app_example')]\n    public function index(): JsonResponse\n    {\n        $client = $this-\u003ekafkaFactory-\u003egetKafkaClient();\n        $producer = $client-\u003egetProducer();\n        $producer-\u003ecreateTopic('test')-\u003eproduce('TestMessage');\n        $producer-\u003eflush();\n\n        [...]\n    }\n```\n\u003e :warning: Don't forget the `$producer-\u003eflush()` method call. Without it, you will lose data!\n\nTo get a `KafkaClient` for a specific connection, just set the name of the desired connection as a parameter in the `getKafkaClient` function of the factory.  \n```php\n$client = $this-\u003ekafkaFactory-\u003egetKafkaClient('different_client');\n```\n\n## Consuming kafka topics\n### Implementing a consumer\nTo consume kafka topics, you need to setup a class which implements the `KafkaConsumer`-Interface.\nThis interface automatically tags the class correctly for symfony DI and forces you to implement the `consume` and the `getConsumerConfiguration` functions.  \nHere is a quick example of this:\n```php\n\u003c?php\n\nnamespace App\\Consumers;\n\nuse LimLabs\\KafkaBundle\\Enum\\ConsumerResponse;\nuse LimLabs\\KafkaBundle\\Kafka\\Consumer\\ConsumerConfiguration;\nuse LimLabs\\KafkaBundle\\Kafka\\Consumer\\KafkaConsumer;\n\nclass ExampleQueueConsumer implements KafkaConsumer\n{\n    public function consume(string $message): ConsumerResponse\n    {\n        var_dump($message);\n        return ConsumerResponse::SUCCESS;\n    }\n\n    public function getConsumerConfiguration(): ConsumerConfiguration\n    {\n        return ConsumerConfiguration::createConfiguration([\n            'consumer_group' =\u003e 'example_consumer_group',\n            'connection' =\u003e 'different_client', #if you do not set this 'default' will be used\n            'subscribed_topics' =\u003e [\n                'example_topic'\n            ]\n        ]);\n    }\n}\n```\nIn the `getConsumerConfiguration` function you have to return a `ConsumerConfiguration`-Object.\nThis describes the specific configuration for the consumer. The array keys `consumer_group` and `subscribed_topics` are required.\nWithout these, an exception will be thrown.  \nYou can create the configuration object like in the previous example or like this:\n```php\npublic function getConsumerConfiguration(): ConsumerConfiguration\n{\n    $configuration = new ConsumerConfiguration();\n    $configuration-\u003esetConnection('different_client');\n    $configuration-\u003esetConsumerGroup('example_consumer_group');\n    $configuration-\u003esetSubscribedTopics(['example_topic']);\n    return $configuration; \n}\n```\nThe `consume` function gets called on each message that comes in over the specified topics. \nThis function has to return a `ConsumerResponse`.  \nThe following responses are valid:\n```php\nConsumerResponse::SUCCESS;\nConsumerResponse::ERROR_DROP;\nConsumerResponse::ERROR_REQUEUE;\n```\nThe `SUCCESS` response signalises the executor that everything gone well. The `ERROR_DROP` response\ninforms the executor of an error. In this case, an error message will be logged in the terminal, but the message will be dropped if you don't take care of it yourself!  \nIn return of `ERROR_DROP`, there is `ERROR_REQUEUE`, which sends the payload of the message back to the Kafka topic from which it came to prevent data loss.\n\u003e :warning: The use of `ERROR_REQUEUE` can lead to infinit loops between the consumer and Kafka.\n\n### Executing a consumer\nTo list all via symfony DI loaded consumers, you can use the following command:\n```shell\nbin/console kafka:consumers:list\n```\nWhich would return something like this:\n```\n----------------------------------- \nName                             \n-----------------------------------\nApp\\Consumers\\ExampleQueueConsumer  \n-----------------------------------\n```\nNow to start a consumer use the following command:\n```shell\nbin/console kafka:topic:consume [CLASS_PATH]\n```\nIf you don't give the class path as an argument, the executor will automatically start the first loaded consumer.\nIf the consumer started correctly you will be greeted by the following message:\n```\n [INFO] 20:11 27-12-2023 Consumer [App\\Consumers\\ExampleQueueConsumer] running... Waiting for messages\n```\n:tada: Congrats now is your consumer consuming messages! ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flimmer-laboratories%2Fkafka-bundle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flimmer-laboratories%2Fkafka-bundle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flimmer-laboratories%2Fkafka-bundle/lists"}