{"id":28380912,"url":"https://github.com/threedotslabs/watermill-benchmark","last_synced_at":"2025-07-10T09:34:35.999Z","repository":{"id":57571293,"uuid":"208112752","full_name":"ThreeDotsLabs/watermill-benchmark","owner":"ThreeDotsLabs","description":"Benchmarking tools for Watermill.","archived":false,"fork":false,"pushed_at":"2024-11-19T15:38:52.000Z","size":132,"stargazers_count":22,"open_issues_count":2,"forks_count":8,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-06T03:54:07.685Z","etag":null,"topics":["benchmark","go","google-cloud-pubsub","kafka","mysql","nats","rabbitmq","watermill"],"latest_commit_sha":null,"homepage":"https://watermill.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ThreeDotsLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-12T17:58:27.000Z","updated_at":"2025-05-20T14:54:52.000Z","dependencies_parsed_at":"2024-11-18T16:39:31.369Z","dependency_job_id":"14df9da6-8f2e-488b-b1e4-ac491465a4dc","html_url":"https://github.com/ThreeDotsLabs/watermill-benchmark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ThreeDotsLabs/watermill-benchmark","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreeDotsLabs%2Fwatermill-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreeDotsLabs%2Fwatermill-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreeDotsLabs%2Fwatermill-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreeDotsLabs%2Fwatermill-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ThreeDotsLabs","download_url":"https://codeload.github.com/ThreeDotsLabs/watermill-benchmark/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreeDotsLabs%2Fwatermill-benchmark/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260690833,"owners_count":23047100,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","go","google-cloud-pubsub","kafka","mysql","nats","rabbitmq","watermill"],"created_at":"2025-05-30T03:30:21.545Z","updated_at":"2025-07-10T09:34:35.994Z","avatar_url":"https://github.com/ThreeDotsLabs.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Watermill Benchmark\n\u003cimg align=\"right\" width=\"300\" src=\"https://watermill.io/img/gopher.svg\"\u003e\n\nThis is a set of tools for benchmarking [watermill](https://github.com/ThreeDotsLabs/watermill).\n\n**Warning:** This tool is meant to provide a rough estimate on how fast each Pub/Sub can process messages.\nIt uses very simplified infrastructure to set things up and default configurations.\n\nKeep in mind that final performance depends on multiple factors.\n\n**It's not meant to be a definitive answer on which Pub/Sub is the fastest.**\nIt should give you an idea of the ballpark performance you can expect.\n\n## How it works\n\n* All tests are run on a single 16 CPU GCloud compute instance (`n1-highcpu-16`).\n* Docker Compose is used to run Pub/Sub infrastructure and benchmark code (except for Google Cloud Pub/Sub).\n* The tool will first produce a big number of messages on a generated topic.\n* Then it will subscribe to the topic and consume all of the messages.\n* Multiple message sizes can be chosen (by default: 16, 64 and 256 bytes).\n\n## Results (as of 18 November 2024)\n\n### Kafka (one node)\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages / s) |\n|----------------------|------------------------|--------------------------|\n| 16                   | 41,492                 | 101,669                  |\n| 64                   | 40,189                 | 106,264                  |\n| 256                  | 40,044                 | 107,278                  |\n\n### NATS Jetstream (16 Subscribers)\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages / s) | Subscribe (messages / s - async ack) |\n|----------------------|------------------------|--------------------------|--------------------------------------|\n| 16                   | 50,668                 | 34,713                   | 59,728                               |\n| 64                   | 49,204                 | 34,561                   | 59,743                               |\n| 256                  | 48,242                 | 34,097                   | 59,385                               |\n\n### NATS Jetstream (48 Subscribers)\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages /s ) | Subscribe (messages / s - async ack) |\n|----------------------|------------------------|--------------------------|--------------------------------------|\n| 16                   | 50,680                 | 46,377                   | 86,348                               |\n| 64                   | 49,341                 | 46,307                   | 86,078                               |\n| 256                  | 48,744                 | 46,035                   | 86,499                               |\n\n### Redis\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages / s) |\n|----------------------|------------------------|--------------------------|\n| 16                   | 59,158                 | 12,134                   |\n| 64                   | 58,988                 | 12,392                   |\n| 256                  | 58,038                 | 12,133                   |\n\n### SQL (MySQL)\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages / s - batch size = 1) | Subscribe (messages / s - batch size = 100) |\n|----------------------|------------------------|-------------------------------------------|---------------------------------------------|\n| 16                   | 6,371                  | 283                                       | 2,794                                       |\n| 64                   | 9,887                  | 281                                       | 2,637                                       |\n| 256                  | 9,596                  | 271                                       | 2,766                                       |\n\n### SQL (PostgreSQL)\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages / s - batch size = 1) | Subscribe (messages / s - batch size = 100) | \n|----------------------|------------------------|-------------------------------------------|---------------------------------------------|\n| 16                   | 2,552                  | 122                                       | 9,460                                       |\n| 64                   | 2,831                  | 118                                       | 9,045                                       |\n| 256                  | 2,744                  | 104                                       | 7,843                                       |\n\n### SQL (PostgreSQL Queue)\n\n| Subscribe Batch Size | Message size (bytes) | Publish (messages / s) |  Subscribe (messages / s - batch size = 1) | Subscribe (messages / s - batch size = 100) | \n|----------------------|----------------------|------------------------|--------------------------------------------|---------------------------------------------|\n| 100                  | 16                   | 2,825                  | 146                                        | 10,466                                      |\n| 100                  | 64                   | 2,842                  | 147                                        | 9,626                                       |\n| 100                  | 256                  | 2,845                  | 138                                        | 8,276                                       |\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages / s) |\n|----------------------|------------------------|--------------------------|\n| 16                   | 2,825                  | 0,146                    |\n| 64                   | 2,842                  | 0,147                    |\n| 256                  | 2,845                  | 0,138                    |\n\n### Google Cloud Pub/Sub (16 subscribers)\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages / s) |\n|----------------------|------------------------|--------------------------|\n| 16                   | 3,027                  | 28,589                   |\n| 64                   | 3,020                  | 31,057                   |\n| 256                  | 2,918                  | 32,722                   |\n\n### AMQP (RabbitMQ, 16 subscribers)\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages / s) |\n|----------------------|------------------------|--------------------------|\n| 16                   | 2,770                  | 14,604                   |\n| 64                   | 2,752                  | 12,128                   |\n| 256                  | 2,750                  | 8,550                    |\n\n### GoChannel\n\n| Message size (bytes) | Publish (messages / s) | Subscribe (messages / s) |\n|----------------------|------------------------|--------------------------|\n| 16                   | 315,776                | 138,743                  |\n| 64                   | 325,341                | 163,034                  |\n| 256                  | 341,223                | 145,718                  |\n\n## VM Setup\n\nThe project includes [Terraform](https://www.terraform.io/) definition for setting up an instance on Google Cloud Platform.\n\nIt will spin up a fresh Ubuntu 19.04 instance, install docker with dependencies and clone this repository.\n\nSet environment variables:\n\n```bash\n# project name on GCP\nTF_VAR_project=\n# public part of the key that you will use to access SSH\nTF_VAR_pub_key_path=\n```\n\nCreate the VM:\n\n```bash\ncd setup\nterraform apply\n```\n\nThe command will output the public IP address of the server. Use ssh with user `benchmark` to access it.\n\nAfter running all benchmarks, destroy the VM:\n\n```bash\nterraform destroy\n```\n\n## Configuration\n\n### Google Pub/Sub\n\nSet environment variables in `compose/.env`:\n\n```bash\n# path to json file within the project with GCP credentials\nGOOGLE_APPLICATION_CREDENTIALS=compose/key.json\n# project name on GCP\nGOOGLE_CLOUD_PROJECT=\n```\n\n## Running\n\nRun benchmarks with:\n\n```bash\n./run.sh \u003cpubsub\u003e\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreedotslabs%2Fwatermill-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthreedotslabs%2Fwatermill-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreedotslabs%2Fwatermill-benchmark/lists"}