https://github.com/simpod/phpkafka

PHP Kafka boilerplate wrapper around RdKafka
https://github.com/simpod/phpkafka

kafka kafka-consumer kafka-producer php php-library php7 rdkafka rdkafka-extension

Last synced: 19 days ago
JSON representation

PHP Kafka boilerplate wrapper around RdKafka

• Host: GitHub
• URL: https://github.com/simpod/phpkafka
• Owner: simPod
• License: mit
• Created: 2019-06-16T12:01:16.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2024-09-17T12:35:35.000Z (about 2 months ago)
• Last Synced: 2024-09-17T15:10:37.333Z (about 2 months ago)
• Topics: kafka, kafka-consumer, kafka-producer, php, php-library, php7, rdkafka, rdkafka-extension
• Language: PHP
• Homepage:
• Size: 145 KB
• Stars: 5
• Watchers: 3
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • Funding: .github/FUNDING.yml

Awesome Lists containing this project

README

        
# PHP Kafka boilerplate wrapper around RdKafka

[![GitHub Actions][GA Image]][GA Link]

[![Code Coverage][Coverage Image]][CodeCov Link]

[![Downloads][Downloads Image]][Packagist Link]

[![Packagist][Packagist Image]][Packagist Link]

[![Infection MSI][Infection Image]][Infection Link]

## Installation

Add as [Composer](https://getcomposer.org/) dependency:

```sh

composer require simpod/kafka

```

## Config Constants

Some config constants are provided like `ConsumerConfig`, `ProducerConfig` or `CommonClientConfigs`.

However, they are copied from Java API and not all are applicable to librdkafka. Consult with librdkafka documentation before use.

## Clients

### Consumer

`KafkaConsumer` boilerplate is available with `startBatch()` method ([to suplement this example in librdkafka](https://github.com/edenhill/librdkafka/blob/master/examples/rdkafka_consume_batch.cpp#L97)) and with `start()`. They also handle

termination signals for you.

#### Classic Consumer

```php

getConfig(), Logger::get());

        $kafkaConsumer->subscribe(['topic1']);

        $kafkaConsumer->start(

            120 * 1000,

            static function (Message $message) use ($kafkaConsumer) : void {

                // Process message here

                $kafkaConsumer->commit($message); // Autocommit is disabled

            }

        );

    }

    private function getConfig(): ConsumerConfig

    {

        $config = new ConsumerConfig();

        $config->set(ConsumerConfig::BOOTSTRAP_SERVERS_CONFIG, '127.0.0.1:9092');

        $config->set(ConsumerConfig::ENABLE_AUTO_COMMIT_CONFIG, false);

        $config->set(ConsumerConfig::CLIENT_ID_CONFIG, gethostname());

        $config->set(ConsumerConfig::AUTO_OFFSET_RESET_CONFIG, 'earliest');

        $config->set(ConsumerConfig::GROUP_ID_CONFIG, 'consumer_group_name');

        return $config;

    }

}

```

#### Batching Consumer

```php

getConfig());

        $kafkaConsumer->subscribe(['topic1']);

        $kafkaConsumer->startBatch(

            200000, 

            120 * 1000,

            static function (Message $message): void {

                // Process record

            },

            static function (ConsumerRecords $consumerRecords) use ($kafkaConsumer) : void {

                // Process records batch

    

                $kafkaConsumer->commit($consumerRecords->getLast());

            }

        );

    }

    private function getConfig(): ConsumerConfig

    {

        $config = new ConsumerConfig();

        $config->set(ConsumerConfig::BOOTSTRAP_SERVERS_CONFIG, '127.0.0.1:9092');

        $config->set(ConsumerConfig::ENABLE_AUTO_COMMIT_CONFIG, false);

        $config->set(ConsumerConfig::CLIENT_ID_CONFIG, gethostname());

        $config->set(ConsumerConfig::AUTO_OFFSET_RESET_CONFIG, 'earliest');

        $config->set(ConsumerConfig::GROUP_ID_CONFIG, 'consumer_group_name');

        return $config;

    }

}

```

[GA Image]: https://github.com/simPod/PhpKafka/workflows/CI/badge.svg

[GA Link]: https://github.com/simPod/PhpKafka/actions?query=workflow%3A%22CI%22+branch%3Amaster

[Coverage Image]: https://codecov.io/gh/simPod/PhpKafka/branch/master/graph/badge.svg

[CodeCov Link]: https://codecov.io/gh/simPod/PhpKafka/branch/master

[Downloads Image]: https://poser.pugx.org/simpod/kafka/d/total.svg

[Packagist Image]: https://poser.pugx.org/simpod/kafka/v/stable.svg

[Packagist Link]: https://packagist.org/packages/simpod/kafka

[Infection Image]: https://img.shields.io/endpoint?style=flat&url=https%3A%2F%2Fbadge-api.stryker-mutator.io%2Fgithub.com%2FsimPod%2FPhpKafka%2Fmaster

[Infection Link]: https://infection.github.io