{"id":15337224,"url":"https://github.com/vy/reactor-pubsub","last_synced_at":"2026-01-14T03:42:46.419Z","repository":{"id":54443895,"uuid":"206168092","full_name":"vy/reactor-pubsub","owner":"vy","description":"Google Pub/Sub Java driver for mortals.","archived":true,"fork":false,"pushed_at":"2024-05-24T12:43:42.000Z","size":372,"stargazers_count":30,"open_issues_count":9,"forks_count":6,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-07-10T06:00:07.670Z","etag":null,"topics":["google-cloud","java","pubsub","queueing","reactive"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vy.png","metadata":{"files":{"readme":"README.adoc","changelog":"CHANGELOG.adoc","contributing":null,"funding":null,"license":"COPYING.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-03T20:38:45.000Z","updated_at":"2024-11-05T19:27:25.000Z","dependencies_parsed_at":"2024-11-01T00:02:16.492Z","dependency_job_id":"1f098d5f-efe9-4971-9dde-804a0c75396b","html_url":"https://github.com/vy/reactor-pubsub","commit_stats":{"total_commits":110,"total_committers":3,"mean_commits":"36.666666666666664","dds":"0.018181818181818188","last_synced_commit":"4f681b06001efc7d4d659bea84af1b8b11d26b1f"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/vy/reactor-pubsub","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vy%2Freactor-pubsub","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vy%2Freactor-pubsub/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vy%2Freactor-pubsub/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vy%2Freactor-pubsub/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vy","download_url":"https://codeload.github.com/vy/reactor-pubsub/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vy%2Freactor-pubsub/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28408858,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T01:52:23.358Z","status":"online","status_checked_at":"2026-01-14T02:00:06.678Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["google-cloud","java","pubsub","queueing","reactive"],"created_at":"2024-10-01T10:20:12.869Z","updated_at":"2026-01-14T03:42:46.402Z","avatar_url":"https://github.com/vy.png","language":"Java","readme":"https://github.com/vy/reactor-pubsub/actions[image:https://github.com/vy/reactor-pubsub/workflows/CI/badge.svg[Actions Status]]\nhttps://search.maven.org/search?q=g:com.vlkan%20a:reactor-pubsub[image:https://img.shields.io/maven-central/v/com.vlkan/reactor-pubsub.svg[Maven Central]]\nhttps://www.apache.org/licenses/LICENSE-2.0.txt[image:https://img.shields.io/github/license/vy/reactor-pubsub.svg[License]]\n\nFinally a https://cloud.google.com/pubsub[Google Cloud Pub/Sub] Java 8 driver\nthat you can wrap your head around its internals and put the fun (i.e.,\nbackpressure-aware, reactive, efficient, batteries-included, and *simple*!) back\ninto messaging.\n\n- It is *backpressure-aware*, because it only pulls batch of messages whenever\n  you need one and tell it to pull one. It doesn't operate a covert thread\n  (pool) to pull whenever it sees a fit and ram it through the application.\n\n- It is *reactive*, because every request is non-blocking, asynchronous, and\n  wired up with the rest of the application using\n  http://www.reactive-streams.org[Reactive Streams].\n\n- It is *efficient*, because it works in batches, the basic unit of message\n  exchange between your driver and Pub/Sub. You `pull()` some, you `ack()` some.\n  One-message-at-a-time is an illusion created by drivers and incurs a\n  significant performance penalty along with operational complexity under the\n  hood.\n\n- It is *batteries-included*, because it provides goodies (out of the box\n  metrics integration, an adaptive rate limiter to help you avoid burning money\n  by continuously pulling and nack'ing messages when something is wrong with\n  your consumer, a `ScheduledExecutorService` implementation with a bounded task\n  queue to mitigate backpressure violating consumption) that assist real-world\n  production deployments.\n\n- It is *simple*, because there are 2 dozens of classes where half is used to\n  represent JSON models transmitted over the wire and the rest is just reactive\n  streams gluing. There are no smart retry mechanisms (though thanks to reactive\n  streams, you can introduce yours), there are no message lease time extending\n  background tasks (hence, you better consume your batch before it times out),\n  etc. Spend a minute on the source code and you are ready to write your own\n  Pub/Sub driver!\n\nDue to the constraints on the development resources (read as, _\"it is just\nme\"_), I needed to take some shortcuts to get this project going:\n\n- https://github.com/reactor/reactor-core/[Reactor Core] for Reactive Streams\n\n- https://github.com/reactor/reactor-netty[Reactor Netty] for HTTP requests\n\n- https://github.com/googleapis/google-auth-library-java[Google Auth\n  Library]footnote:[This could have been replaced with a more lightweight\n  alternative, but given you have already been using Pub/Sub, it is highly\n  likely that you already sold your soul to some other Google Cloud services\n  too. Hence, no need to introduce an extra dependency.] for authentication\n\n- https://github.com/FasterXML/jackson-databind[Jackson]footnote:[https://github.com/googleapis/google-api-java-client[Google\n  APIs Client Library] already depends on Jackson.] for JSON serialization\n\n- http://micrometer.io/[Micrometer] for metrics _(optional)_\n\n== Getting started\n\nYou first need to add the `reactor-pubsub` artifact into your list of\ndependencies:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.vlkan\u003c/groupId\u003e\n    \u003cartifactId\u003ereactor-pubsub\u003c/artifactId\u003e\n    \u003cversion\u003e${reactor-pubsub.version}\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n(Note that the Java 9 module name is `com.vlkan.pubsub`.)\n\nYou can create a publisher and publish a message as follows:\n\n```java\n// Create the publisher.\nString projectName = \"awesome-project\";\nPubsubPublisherConfig publisherConfig = PubsubPublisherConfig\n        .builder()\n        .setProjectName(projectName)\n        .setTopicName(\"awesome-topic\")\n        .build();\nPubsubPublisher publisher = PubsubPublisher\n        .builder()\n        .setConfig(publisherConfig)\n        .build();\n\n// Publish a message.\npublisher\n        .publishMessage(\n                new PubsubDraftedMessage(\n                        \"Yolo like nobody's watching!\"\n                                .getBytes(StandardCharsets.UTF_8)))))\n        .doOnSuccess(publishResponse -\u003e\n                System.out.format(\n                        \"Published awesome message ids: %s%n\",\n                        publishResponse.getMessageIds()))\n        .subscribe();\n```\n\nNote that `PubsubDraftedMessage` constructor has two variants:\n\n- `PubsubDraftedMessage(byte[] payload)`\n- `PubsubDraftedMessage(byte[] payload, Map\u003cString, String\u003e attributes)`\n\n`PubsubPublisher` provides the following auxiliary methods:\n\n- `publishMessages(List\u003cPubsubDraftedMessage\u003e messages)`\n- `publishMessage(List\u003cPubsubDraftedMessage\u003e message)`\n- `publish(PubsubPublishRequest publishRequest)`\n\nYou can create a subscriber and start receiving messages from a subscription as\nfollows:\n\n```java\n// Create a puller.\nString subscriptionName = \"awesome-subscription\";\nPubsubPullerConfig pullerConfig = PubsubPullerConfig\n        .builder()\n        .setProjectName(projectName)\n        .setSubscriptionName(subscriptionName)\n        .build();\nPubsubPuller puller = PubsubPuller\n        .builder()\n        .setConfig(pullerConfig)\n        .build();\n\n// Create an acker.\nPubsubAckerConfig ackerConfig = PubsubAckerConfig\n        .builder()\n        .setProjectName(projectName)\n        .setSubscriptionName(subscriptionName)\n        .build();\nPubsubAcker acker = PubsubAcker\n        .builder()\n        .setConfig(ackerConfig)\n        .build();\n\n// Pull and consume continuously.\npuller\n        .pullAll()\n        .concatMap(pullResponse -\u003e {\n            int messageCount = pullResponse.getReceivedMessages().size();\n            System.out.format(\"Just got awesome %d messages!%n\", messageCount);\n            return acker.ackPullResponse(pullResponse);\n        })\n        .subscribe();\n```\n\nNote that `PubsubAcker` provides the following auxiliary methods:\n\n- `ackPullResponse(PubsubPullResponse pullResponse)`\n- `ackMessages(List\u003cPubsubReceivedMessage\u003e messages)`\n- `ackMessage(PubsubReceivedMessage message)`\n- `ackIds(List\u003cString\u003e ackIds)`\n- `ackId(String ackId)`\n- `ack(PubsubAckRequest ackRequest)`\n\n== Utilities\n\nThe project ships a couple of utilities where you might find them handy in\nassembling your messaging pipeline. Even though they are optional, we strongly\nrecommend their usage.\n\n=== Rate limiter\n\nWe strongly encourage everyone to employ the provided rate limiter while\nconsuming messages. The rationale is simple: In order to avoid burning GCP bills\nfor nothing, you better cut down the consumption rate if the rest of the system\nis indicating a failure.\n\n`reactor-pubsub` provides the following utilities for rate limiting purposes:\n\n- `RateLimiter` is a simple (_package local_) rate limiter.\n\n- `StagedRateLimiter` is a rate limiter with multiple stages. Each stage is\n  composed of a _success rate_ and _failure rate_ pair. In the absence of\n  failure acknowledgements, excessive permit claims replace the active stage\n  with the next faster one, if there is any. Likewise, excessive failure\n  acknowledgements replace the active stage with the next slower one, if there\n  is any.\n\nOne can employ the `StagedRateLimiter` for a `PubsubPuller` as follows:\n\n```java\n// Create the staged rate limiter and its reactor decorator.\nString stagedRateLimiterName = projectName + '/' + subscriptionName;\nStagedRateLimiter stagedRateLimiter = StagedRateLimiter\n        .builder()\n        .setName(stagedRateLimiterName)\n        .setSpec(\"1/1m:, 1/30s:1/1m, 1/1s:2/1m, :1/3m\")     // (default)\n        .build();\nStagedRateLimiterReactorDecoratorFactory stagedRateLimiterReactorDecoratorFactory =\n        StagedRateLimiterReactorDecoratorFactory\n                .builder()\n                .setStagedRateLimiter(stagedRateLimiter)\n                .build();\nFunction\u003cFlux\u003cPubsubPullResponse\u003e, Flux\u003cPubsubPullResponse\u003e\u003e stagedRateLimiterFluxDecorator =\n        stagedRateLimiterReactorDecoratorFactory.ofFlux();\n\n// Employ the staged rate limiter.\npuller\n        .pullAll()\n        .concatMap(pullResponse -\u003e {\n            // ...\n            return acker.ackPullResponse(pullResponse);\n        })\n        .transform(stagedRateLimiterFluxDecorator)\n        .subscribe();\n```\n\nThe stages are described in increasing success rate limit order using a\nspecification format as follows: `1/1m:, 1/30s:1/1m, 1/1s:2/1m, :1/3m`. The\nspecification is a comma-separated list of _[success rate limit]:[failure rate\nlimit]_ pairs where, e.g., `1/1h` is used to denote a rate limit of a single\npermit per 1 hour. Temporal unit must be one of h(ours), m(inutes), or\ns(econds). The initial failure rate limit and the last success rate limit can be\nomitted to indicate no rate limits.) This example will result in the following\nstages.\n\n.`StagedRateLimiter` stages for specification `1/1m:, 1/30s:1/1m, 1/1s:2/1m, :1/3m`.\n|===\n| stage | success rate limit | failure rate limit\n\n| 1\n| 1/1m (once per minute)\n| infinite\n\n| 2\n| 1/30s (once per 30 second)\n| 1/1m (once per minute)\n\n| 3\n| 1/1s (once per second)\n| 2/1m (twice per minute)\n\n| 4\n| infinite\n| 1/3m (once per 3 minute)\n|===\n\nBy contract, initially the active stage is set to the one with the slowest\nsuccess rate limit.\n\n=== Bounded `SchedulerExecutorService`\n\n`PubsubPuller`, `PubsubAccessTokenCache`, and\n`StagedRateLimiterReactorDecoratorFactory` optionally receive either a\n`ScheduledExecutorService` or a Reactor `Scheduler` in their builders for timed\ninvocations. One can explicitly change the implicit scheduler used by any\nReactor `Mono\u003cT\u003e` or `Flux\u003cT\u003e` as well. (See\nhttps://projectreactor.io/docs/core/release/reference/#schedulers[Threading and\nSchedulers] in Reactor reference manual.) We strongly suggest employing a common\ndedicated scheduler for all these cases with a _bounded task queue_. That said,\nunfortunately neither the default Reactor ``Scheduler``s nor the\n`ScheduledExecutorService` implementations provided by the Java Standard library\nallow one to put a bound on the task queue size. This shortcoming is severely\nprone to hiding backpressure problems. (See the\nhttp://cs.oswego.edu/pipermail/concurrency-interest/2019-April/016861.html[the\nrelevant concurrency-interest discussion].) To mitigate this, we provide\n`BoundedScheduledThreadPoolExecutor` wrapper and strongly recommend to employ it\nin your Reactor assembly line. Even though this will incur an extra thread\ncontext switching cost, this is almost negligible for a majority of the use\ncases and the benefit will overweight this minor expense. The usage is as simple\nas follows:\n\n```java\n// Create the executor.\nScheduledThreadPoolExecutor executor =\n        new ScheduledThreadPoolExecutor(\n                Runtime.getRuntime().availableProcessors());\nBoundedScheduledThreadPoolExecutor boundedExecutor =\n        new BoundedScheduledThreadPoolExecutor(100, executor);\nScheduler scheduler = Schedulers.fromExecutorService(boundedExecutor);\n\n// Set the access token cache executor.\nPubsubAccessTokenCache\n        .builder()\n        .setExecutorService(executor)\n        // ...\n        .build();\n\n// Set the puller scheduler.\nPubsubPuller puller = PubsubPuller\n        .builder()\n        .setScheduler(scheduler)\n        // ...\n        .build();\n\n// Employ the scheduler in the Reactor pipeline.\npuller\n        .pullAll()\n        .concatMap(pullResponse -\u003e {\n            // ...\n            return acker.ackPullResponse(pullResponse);\n        })\n        .flatMap(this::doSomeOtherAsyncIO)\n        .subscribeOn(scheduler)\n        .subscribe();\n```\n\n== F.A.Q\n\n=== How can I avoid stream termination when pull fails?\n\nIt is a common pitfall to build a message consumption pipeline as follows:\n\n```java\npuller\n        .pullAll()\n        .concatMap(pullResponse -\u003e businessLogic\n                .execute(pullResponse)\n                .then(acker.ackPullResponse(pullResponse)))\n        .subscribe();\n```\n\nHere the `Flux\u003cPubsubPullResponse\u003e` returned by `pullAll()` will be  terminated\nif any of the methods along the reactive chain (`pullAll()`,\n`businessLogic.execute()`, `ack()`, etc.) throws an exception. No matter how\nmany `doOnError()`, `onErrorResume()` you plaster there, the damage has been\ndone, the subscription has been cancelled, and `pullAll()` will not continue\npulling anymore. Note that this applies to any\nhttps://projectreactor.io/docs/core/release/reference/#flux[`Flux`] and nothing\nnew to the way we leverage it here. To prevent such premature stream\ntermination, you need to retry subscribing. While this can be done as simple as\ncalling `retry()`, you might also want to check out more fancy options like\n`retryBackoff()`. As one final remark, make sure you deal (log?) with the error\nprior to retrying.\n\n=== How can I retry ack's?\n\nSee\nhttps://projectreactor.io/docs/core/release/reference/#faq.exponentialBackoff[How\nto use `retryWhen` for exponential backoff?] in Reactor reference manual.\n\n=== How can I change the GCP credentials?\n\nUnless one provided, all `PubsubPublisher`, `PubsubPuller` and `PubsubAcker`\nclasses use the `PubsubAccessTokenCache.getDefaultInstance()` and\n`PubsubClient.getDefaultInstance()` defaults. By default,\n`PubsubAccessTokenCache` leverages `GoogleCredentials.getApplicationDefault()`\nprovided by the `google-auth-library-oauth2-http` artifact. This function\ndetermines the credentials by trying out the following steps in order:\n\n. Credentials file pointed to by the `GOOGLE_APPLICATION_CREDENTIALS`\n  environment variable\n\n. Credentials provided by the Google Cloud SDK `gcloud auth application-default\n  login` command\n\n. Google App Engine built-in credentials\n\n. Google Cloud Shell built-in credentials\n\n. Google Compute Engine built-in credentials\n\nRather than relying on this mechanism, one can explicitly set the credentials\nas follows:\n\n```java\n// Create the access token cache.\nPubsubAccessTokenCache accessTokenCache = PubsubAccessTokenCache\n        .builder()\n        .setCredentials(\"awesome-password\")     // null falls back to the defaults\n        .build();\n\n// Create the client.\nPubsubClient client = PubsubClient\n        .builder()\n        .setAccessTokenCache(accessTokenCache)\n        .build();\n\n// Create the puller.\nPubsubPuller puller = PubsubPuller\n        .builder()\n        .setClient(client)\n        // ...\n        .build();\n\n// Create the ack'er.\nPubsubAcker acker = PubsubAcker\n        .builder()\n        .setClient(client)\n        // ...\n        .build();\n\n// Create the publisher.\nPubsubPublisher publisher = PubsubPublisher\n        .builder()\n        .setClient(client)\n        // ...\n        .build();\n```\n\n=== How can I enable metrics?\n\nGiven http://micrometer.io/[Micrometer] is used for metrics, you first need to\nhave it in your list of dependencies:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003eio.micrometer\u003c/groupId\u003e\n    \u003cartifactId\u003emicrometer-core\u003c/artifactId\u003e\n    \u003cversion\u003e${micrometer.version}\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nBoth `PubsubClient` and `StagedRateLimiterReactorDecoratorFactory` provide\nmeans to configure metrics. Each can be simply configured as follows:\n\n```java\n// Create a meter registry.\nMeterRegistry meterRegistry = ...;\n\n// Pass the meter registry to the Pub/Sub client.\nPubsubClient\n        .builder()\n        .setMeterRegistry(meterRegistry)\n        .setMeterNamePrefix(\"pubsub.client\")            // default\n        .setMeterTags(Collections.emptyMap())           // default\n        // ...\n        .build();\n\n// Pass the meter registry to the rate limiter factory.\nStagedRateLimiterReactorDecoratorFactory\n        .builder()\n        .setMeterRegistry(meterRegistry)\n        .setMeterNamePrefix(\"pubsub.stagedRateLimiter\") // default\n        .setMeterTags(Collections.emptyMap())           // default\n        // ...\n        .build();\n```\n\nAbove will publish metrics with the following footprints:\n\n|===\n|Name |Tags |Description\n\n|`pubsub.client.publish.latency`\n|`projectName`, `topicName`, `result`\n|`publish` request latency\n\n|`pubsub.client.publish.count`\n|`projectName`, `topicName`\n|``publish``ed message count\n\n|`pubsub.client.{pull,ack}.latency`\n|`projectName`, `subscriptionName`, `result`\n|`pull` and `ack` request latency\n\n|`pubsub.client.{pull,ack}.count`\n|`projectName`, `subscriptionName`\n|``pulled``ed/``ack``ed message count\n\n|`pubsub.stagedRateLimiter.permitWaitPeriod`\n|`name`\n|permit wait period distribution summary\n|===\n\nThere are a couple of details that need further elaboration here:\n\n- When `PubsubPullerConfig#pullPeriod` is set to zero (default), `pull` requests\n  will only get completed when there are messages. Hence, one might experience\n  high latencies in queues that frequently become empty.\n\n- When `PubsubPullerConfig#pullPeriod` is set to a value greater than zero,\n  repeatedly executed `pull` requests by `PubsubPuller#pullAll()` will get\n  followed by a `pullPeriod` delay after an empty response. Hence the published\n  `pubsub.client.pull.latency` metrics are a combination of both the full and\n  the empty responses.\n\n- As of this writing, Pub/Sub blocks every `pull` requests at least ~1.5 seconds\n  before returning an empty response.\n\n=== How can I run it against the Pub/Sub emulator?\n\nPub/Sub provides an https://cloud.google.com/pubsub/docs/emulator[emulator]\nto test your applications locally. In order to use it in combination with\n`reactor-pubsub`, you need to configure the `baseUrl` of the `PubsubClient` as\nfollows:\n\n```java\n// Create a custom client.\nPubsubClientConfig clientConfig = PubsubClientConfig\n        .builder()\n        .setBaseUrl(\"http://localhost:8085\")\n        .build();\nPubsubClient client = PubsubClient\n        .builder()\n        .setConfig(clientConfig)\n        .build();\n\n// Create a publisher.\nPubsubPublisher publisher = PubsubPublisher\n        .builder()\n        .setClient(client)\n        // ...\n        .build();\n\n// Create a puller.\nPubsubPuller puller = PubsubPuller\n        .builder()\n        .setClient(client)\n        // ...\n        .build();\n\n// Create an acker.\nPubsubAcker acker = PubsubAcker\n        .builder()\n        .setClient(client)\n        // ...\n        .build();\n```\n\n=== How fast is ``reactor-pubsub``?\n\nOne of the most frequent questions `reactor-pubsub` is challenged with is how\ndoes it perform given the official Pub/Sub client uses Protobuf over HTTP/2\n(gRPC), whereas `reactor-pubsub` uses JSON over HTTP/1?\n\nBefore going into convincing figures to elaborate on ``reactor-pubsub``s\nperformance characteristics, there is one thing that deserves attention in\nparticular: _JSON over HTTP/1 is a deliberate design decision for simplicity_\nrather than a fallback due to technical limitations. Even though it is\nopinionated, ``reactor-pubsub`` strives to serve as a Pub/Sub client that\nleverages frequently used tools (e.g., JSON) and idioms Java developers are\naccustomed to. Further, in case of failures, it should be trivial to spot the\nsmoking gun using a decent IDE debugger.\n\n`reactor-pubsub` source code ships a reproducible link:benchmark[benchmark]\nalong with its link:benchmark/results.html[results]. As shared there, one can\nretrieve (i.e., `pull` \u0026 `ack`) a payload of 781 MiB in 2,083 ms using two\n2.70GHz CPU cores, pull batch size 50, pull concurrency 5, and message payload\nlength 16 KiB. That is, 11,998 messages per second on a single core!* Do you\nstill need more juice? 🙇 Go ahead and create a ticket with your use case,\nobserved performance, and implementation details.\n\nimage:benchmark/results.png[Benchmark Results]\n\n\n== Historical account\n\nI (_Volkan Yazıcı_) would like to take this opportunity to share the historical\naccount from my perspective to justify the effort and defend it against any\npotential https://en.wikipedia.org/wiki/Not_invented_here[NIH] syndrome\naccusations.\n\n*Why did I feel a need to implement a Pub/Sub Java driver from scratch?* At\nhttps://bol.com[bol.com], we heavily use Pub/Sub. There we started our pursuit\nlike the rest of the Pub/Sub users with\nhttps://cloud.google.com/pubsub/docs/quickstart-client-libraries[the official\nJava drivers] provided by Google. Later on we started bumping into backpressure\nproblems: tasks on the shared `ScheduledExecutorService` were somehow awkwardly\ndating back and constantly piling up. That was the point I introduced a\nlink:src/main/java/com/vlkan/pubsub/util/BoundedScheduledThreadPoolExecutor.java[BoundedScheduledThreadPoolExecutor]\nand shit hit the fan. I figured the official Pub/Sub driver was ramming the\nfetched batch of messages through the shared executor. My first reaction was to\ncut down the pull buffer size and the concurrent pull count. That solved a\nmajority of our backpressure-related problems, though created a new one:\nefficiency. Then I started examining the source code and wasted quite a lot of\ntime trying to make forsaken\nhttps://github.com/googleapis/gax-java/blob/master/gax/src/main/java/com/google/api/gax/batching/FlowControlSettings.java[FlowControlSettings]\nwork. This disappointing inquiry resulted in something remarkable: I understood\nhow Pub/Sub works and amazed by the extent of complexity for such a simple task.\nI have already been using Reactive Streams (RxJava and Reactor) every single\nwork day in the last five years and compiled a thick collection of lessons and\nrecipes out of it. The more I examined the official Pub/Sub Java driver source\ncode, the more I was convinced that I could very well engineer this into\nsomething way more simple. I know how to pump JSON payloads over HTTP via\nReactor Netty and enjoy a backpressure-aware, reactive comfort out of the box.\nBut that wasn't the tipping point I had decided to implement my own Pub/Sub Java\ndriver. I made my mind when I witnessed that\nhttps://github.com/spring-cloud/spring-cloud-gcp/pull/1461#discussion_r274098603[Google\nengineers are clueless about these problems].\n\n*Why all the fuss about the rate limiting?* One morning I came to the  office\nand read an e-mail from one of the platform teams asking how come we managed to\nburn hundreds of dollars worth of Pub/Sub messaging in the middle of the night.\nOne of the application (non-critical) databases happened to go down for a couple\nof hours and during that period nodes constantly sucked up messages and nack'ed\nthem due to the database failure. This is an opinionated Pub/Sub driver and in\nmy opinion you should not relentlessly burn Pub/Sub bills if the rest of the\napplication is shouting out there is something going on wrong. Hence, please\nconfigure and use the god damn rate limiter.\n\n== Security policy\n\nIf you have encountered an unlisted security vulnerability or other unexpected\nbehaviour that has security impact, please report them privately to the\nmailto:volkan@yazi.ci[volkan@yazi.ci] email address.\n\n== Contributors\n\n- https://github.com/akoshterek[Alex Koshterek]\n- https://github.com/berkaybuharali[Berkay Buharalı]\n- https://github.com/luiccn[Luiz Neto]\n- https://github.com/bsideup[Sergei Egorov]\n\n== License\n\nCopyright \u0026copy; 2019-2020 https://vlkan.com/[Volkan Yazıcı]\n\nLicensed under the Apache License, Version 2.0 (the \"License\");  you may not use\nthis file except in compliance with the License. You may obtain a copy of the\nLicense at\n\n```\nhttp://www.apache.org/licenses/LICENSE-2.0\n```\n\nUnless required by applicable law or agreed to in writing, software distributed\nunder the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR\nCONDITIONS OF ANY KIND, either express or implied. See the License for the\nspecific language governing permissions and limitations under the License.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvy%2Freactor-pubsub","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvy%2Freactor-pubsub","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvy%2Freactor-pubsub/lists"}