{"id":16488702,"url":"https://github.com/grongor/php-kafka-rest-client","last_synced_at":"2025-03-21T07:31:20.143Z","repository":{"id":56983476,"uuid":"219070456","full_name":"grongor/php-kafka-rest-client","owner":"grongor","description":"PHP client for Confluent REST Proxy of Apache Kafka","archived":false,"fork":false,"pushed_at":"2025-02-11T15:26:48.000Z","size":45,"stargazers_count":8,"open_issues_count":6,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-17T22:39:29.394Z","etag":null,"topics":["client","confluent","confluent-kafka","confluent-platform","kafka","librdkafka","php","proxy","rdkafka","rdkafka-extension","rest"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/grongor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-11-01T22:15:25.000Z","updated_at":"2023-11-20T07:38:06.000Z","dependencies_parsed_at":"2023-02-18T05:46:13.114Z","dependency_job_id":null,"html_url":"https://github.com/grongor/php-kafka-rest-client","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grongor%2Fphp-kafka-rest-client","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grongor%2Fphp-kafka-rest-client/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grongor%2Fphp-kafka-rest-client/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grongor%2Fphp-kafka-rest-client/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/grongor","download_url":"https://codeload.github.com/grongor/php-kafka-rest-client/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244757646,"owners_count":20505440,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["client","confluent","confluent-kafka","confluent-platform","kafka","librdkafka","php","proxy","rdkafka","rdkafka-extension","rest"],"created_at":"2024-10-11T13:39:41.671Z","updated_at":"2025-03-21T07:31:19.824Z","avatar_url":"https://github.com/grongor.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"kafka-rest-client [![Build Status](https://travis-ci.com/grongor/php-kafka-rest-client.svg?branch=master)](https://travis-ci.com/grongor/php-kafka-rest-client)\n=================\n\nThis is a modern PHP client for [Confluent REST Proxy](https://docs.confluent.io/current/kafka-rest/) (version 2).\n\nHTTP client used for communication with API adheres to the modern [HTTP standards](http://php-http.org) like\n[PSR-7](https://www.php-fig.org/psr/psr-7/), [PSR-17](https://www.php-fig.org/psr/psr-17/)\nand [PSR-18](https://www.php-fig.org/psr/psr-18/).\n\nAside from implementing the REST Proxy API this library adds some convenient classes useful for the most interactions\nwith Apache Kafka - see [examples](#Examples) for the whole list and a simple tutorial on how to use them.\n\nMissing features\n----------------\n\nSome features were deliberately skipped to simplify the implementation/shorten the development time.\nIf you need anything and you're willing create a pull request, I'd be happy to check it out\nand (if it makes sense) merge it. You can start the discussion by opening an issue.\n\n- version 1 of the REST Proxy API\n  - It's easy to upgrade the REST Proxy, so I don't see any point in implementing the first version.\n- AVRO embedded format\n- JSON embedded format\n  - I think that the binary format is sufficient. You can always serialize/deserialize your objects before/after\n  they interact with this library, therefore direct integration here is just a complication.\n- async operations\n  - To simplify the library all the methods are synchronous. That might definitely change if the need arises.\n- all not-yet-implemented features mentioned [here](https://docs.confluent.io/current/kafka-rest/#features)\n  - When these features are implemented, they will be added here ASAP.\n\nExamples\n--------\n\n### Producer\n\nProducer allows you to produce messages, either one by one or in batches. Both use the same underlying API method\nso it's more efficient to use the batch one.\n\nBoth `produce` and `produceBatch` return nothing on success and throw an exception on failure.\nThere might be partial success/failure so the thrown exception `FailedToProduceMessages` contains two public properties,\n`$retryable` and `$nonRetryable`, where each contains an array of\n`['error' =\u003e 'error provided by Kafka', message =\u003e (Message object given to produce())]`.\nWhether the error is [non-]retryable is based on the `error_code` as is\n[documented](https://docs.confluent.io/current/kafka-rest/api.html#post--topics-(string-topic_name)).\nIt's up to you what you do with those.\n\n```php\n$producer = new Producer($restClient);\n$producer-\u003eproduce('your-topic', new Message('some-message'));\n\n$messages = [new Message('some-message'), new Message('and-some-other-message')];\n$producer-\u003eproduceBatch('your-topic', $messages);\n```\n\n### Consumer\n\nConsumer allows you to consume messages one by one until you return from the loop, application throws exception\nor is otherwise forced to exit.\nThe `consume` method returns a [Generator](https://www.php.net/manual/en/class.generator.php) and loops indefinitely,\nyielding messages as they are available. `consume` method accepts two parameters: `timeout` and `maxBytes`.\n`timeout` is the maximum time the consumer will wait for the messages. `maxBytes` is the maximum size of the messages\nto fetch in a single request. Both of these settings are complementary to the settings in `ConsumerOptions` and to the\nserver settings (check the Kafka documentation for more information).\n\nIf you set the consumer option `autoCommitEnable` to `false` then you may use consumer's `commit` method to commit\nmessages. Simply pass it a message you wish to commit. For most cases it's recommended to turn off the auto-commit\nand manually commit each message, so that you don't ever \"lose\" a message if your application dies\nin the middle of the processing.\n\nYou can also (optionally) set an idle callback using `setIdleCallback` method. This callback will be called whenever\nthere are no messages to yield. The idle interval is then equal to the `timeout` parameter if provided, or to the\n`consumerRequestTimeoutMs` option if set, otherwise to the proxy configuration option `consumer.request.timeout.ms`.\n\n```php\n$consumerFactory = new ConsumerFactory($restClient);\n$consumer = $consumerFactory-\u003ecreate('your-consumer-group', Subscription::topic('your-topic'));\nforeach ($consumer-\u003econsume() as $message) {\n    // Do your magic\n    $logger-\u003einfo('Got new message', $message-\u003econtent);\n\n    // ... and when you are done, commit the message (if you turned off auto-committing).\n    $consumer-\u003ecommit($message);\n}\n```\n\n### BatchConsumer\n\nBatchConsumer works the same way as Consumer does; the difference is that BatchConsumer doesn't yield each message\nseparately but first puts them in batches (`MessagesBatch`). These batches can be configured to be \"limited\" by either\ncount of messages in the batch (`maxCount`), by time (`maxDuration`) or by both (setting both `maxCount` and\n`maxDuration`). If you set `maxDuration` then the batch won't ever take longer than that (+ few ms for processing) as\nit changes the `timeout` parameter of the consumer (consumer won't get stuck on network waiting for more messages).\n\nBatchConsumer can be quite useful for processing of \"large\" data sets, where you would have to otherwise batch\nthe messages yourself (eg. for inserting into database, where batch operations are always better).\n\nAs mentioned in the Consumer example, you may need to commit the messages. For that there is a `commit` method,\nwhich accepts the yielded `MessagesBatch` and commits all the messages inside it in one request.\n\n```php\n$batchConsumerFactory = new BatchConsumerFactory($restClient);\n$batchConsumer = $batchConsumerFactory-\u003ecreate(\n    'your-consumer-group',\n    Subscription::topic('your-topic'),\n    $maxCount = 10000, // Yield the batch when there is 10 000 messages in it\n    $maxDuration = 60  // or when 60 seconds passes, whichever comes first.\n);\nforeach ($batchConsumer-\u003econsume() as $messagesBatch) {\n    // The batch might be empty if you specified the maxDuration.\n    if ($messagesBatch-\u003eisEmpty()) {\n        continue;\n    }\n\n    // Do your magic\n    $database-\u003einsert($messagesBatch-\u003egetMessages());\n\n    // ... and when you are done, commit the batch.\n    $batchConsumer-\u003ecommit($messagesBatch);\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrongor%2Fphp-kafka-rest-client","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgrongor%2Fphp-kafka-rest-client","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrongor%2Fphp-kafka-rest-client/lists"}