{"id":13413427,"url":"https://github.com/ThreeDotsLabs/watermill","last_synced_at":"2025-03-14T19:32:21.600Z","repository":{"id":37395911,"uuid":"156768326","full_name":"ThreeDotsLabs/watermill","owner":"ThreeDotsLabs","description":"Building event-driven applications the easy way in Go.","archived":false,"fork":false,"pushed_at":"2024-10-29T13:52:03.000Z","size":26701,"stargazers_count":7478,"open_issues_count":104,"forks_count":401,"subscribers_count":81,"default_branch":"master","last_synced_at":"2024-10-29T14:34:05.827Z","etag":null,"topics":["cqrs","event-driven","event-sourcing","events","go","golang","kafka","nats","rabbitmq","reactive","sagas","stream-processing","watermill"],"latest_commit_sha":null,"homepage":"https://watermill.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ThreeDotsLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-08T21:01:09.000Z","updated_at":"2024-10-29T14:08:25.000Z","dependencies_parsed_at":"2024-03-30T09:26:43.182Z","dependency_job_id":"0388430b-7761-4108-a400-3de6b0787b1f","html_url":"https://github.com/ThreeDotsLabs/watermill","commit_stats":{"total_commits":402,"total_committers":63,"mean_commits":6.380952380952381,"dds":0.5895522388059702,"last_synced_commit":"db2c50ae3aa4837fe64e659308bd89b29cca1d98"},"previous_names":[],"tags_count":45,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreeDotsLabs%2Fwatermill","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreeDotsLabs%2Fwatermill/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreeDotsLabs%2Fwatermill/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThreeDotsLabs%2Fwatermill/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ThreeDotsLabs","download_url":"https://codeload.github.com/ThreeDotsLabs/watermill/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243635463,"owners_count":20322944,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cqrs","event-driven","event-sourcing","events","go","golang","kafka","nats","rabbitmq","reactive","sagas","stream-processing","watermill"],"created_at":"2024-07-30T20:01:40.133Z","updated_at":"2025-03-14T19:32:21.593Z","avatar_url":"https://github.com/ThreeDotsLabs.png","language":"Go","readme":"# Watermill\n\u003cimg align=\"right\" width=\"300\" src=\"https://watermill.io/img/gopher.svg\"\u003e\n\n[![CI Status](https://github.com/ThreeDotsLabs/watermill/actions/workflows/master.yml/badge.svg)](https://github.com/ThreeDotsLabs/watermill/actions/workflows/master.yml)\n[![Go Reference](https://pkg.go.dev/badge/github.com/ThreeDotsLabs/watermill.svg)](https://pkg.go.dev/github.com/ThreeDotsLabs/watermill)\n[![Go Report Card](https://goreportcard.com/badge/github.com/ThreeDotsLabs/watermill)](https://goreportcard.com/report/github.com/ThreeDotsLabs/watermill)\n[![codecov](https://codecov.io/gh/ThreeDotsLabs/watermill/branch/master/graph/badge.svg)](https://codecov.io/gh/ThreeDotsLabs/watermill)\n\nWatermill is a Go library for working efficiently with message streams. It is intended\nfor building event driven applications, enabling event sourcing, RPC over messages,\nsagas and basically whatever else comes to your mind. You can use conventional pub/sub\nimplementations like Kafka or RabbitMQ, but also HTTP or MySQL binlog if that fits your use case.\n\n## Goals\n\n* **Easy** to understand.\n* **Universal** - event-driven architecture, messaging, stream processing, CQRS - use it for whatever you need.\n* **Fast** (see [Benchmarks](#benchmarks)).\n* **Flexible** with middlewares, plugins and Pub/Sub configurations.\n* **Resilient** - using proven technologies and passing stress tests (see [Stability](#stability)).\n\n## Getting Started\n\nPick what you like the best or see in order:\n\n1. Follow the [Getting Started guide](https://watermill.io/docs/getting-started/).\n2. See examples below.\n3. Read the full documentation: https://watermill.io/\n\n## Our online hands-on training\n\n\u003ca href=\"https://threedots.tech/event-driven/?utm_source=watermill-readme\"\u003e\u003cimg align=\"center\" width=\"400\" src=\"https://threedots.tech/event-driven-banner.png\"\u003e\u003c/a\u003e\n\n## Examples\n\n* Basic\n    * [Your first app](_examples/basic/1-your-first-app) - **start here!**\n    * [Realtime feed](_examples/basic/2-realtime-feed)\n    * [Router](_examples/basic/3-router)\n    * [Metrics](_examples/basic/4-metrics)\n    * [CQRS with protobuf](_examples/basic/5-cqrs-protobuf)\n* [Pub/Subs usage](_examples/pubsubs)\n    * These examples are part of the [Getting started guide](https://watermill.io/docs/getting-started/) and show usage of a single Pub/Sub at a time.\n* Real-world examples\n    * [Exactly-once delivery counter](_examples/real-world-examples/exactly-once-delivery-counter)\n    * [Receiving webhooks](_examples/real-world-examples/receiving-webhooks)\n    * [Sending webhooks](_examples/real-world-examples/sending-webhooks)\n    * [Synchronizing Databases](_examples/real-world-examples/synchronizing-databases)\n    * [Persistent Event Log](_examples/real-world-examples/persistent-event-log)\n    * [Transactional Events](_examples/real-world-examples/transactional-events)\n    * [Real-time HTTP updates with Server-Sent Events](_examples/real-world-examples/server-sent-events)\n    * [Real-time HTTP updates with Server-Sent Events and htmx](_examples/real-world-examples/server-sent-events-htmx)\n* Complete projects\n    * [NATS example with live code reloading](https://github.com/ThreeDotsLabs/nats-example)\n    * [RabbitMQ, webhooks and Kafka integration](https://github.com/ThreeDotsLabs/event-driven-example)\n\n## Background\n\nBuilding distributed and scalable services is rarely as easy as some may suggest. There is a\nlot of hidden knowledge that comes with writing such systems. Just like you don't need to know the\nwhole TCP stack to create a HTTP REST server, you shouldn't need to study all of this knowledge to\nstart with building message-driven applications.\n\nWatermill's goal is to make communication with messages as easy to use as HTTP routers. It provides\nthe tools needed to begin working with event-driven architecture and allows you to learn the details\non the go.\n\nAt the heart of Watermill there is one simple interface:\n```go\nfunc(*Message) ([]*Message, error)\n```\n\nYour handler receives a message and decides whether to publish new message(s) or return\nan error. What happens next is up to the middlewares you've chosen.\n\nYou can find more about our motivations in our [*Introducing Watermill* blog post](https://threedots.tech/post/introducing-watermill/).\n\n## Pub/Subs\n\nAll publishers and subscribers have to implement an interface:\n\n```go\ntype Publisher interface {\n\tPublish(topic string, messages ...*Message) error\n\tClose() error\n}\n\ntype Subscriber interface {\n\tSubscribe(ctx context.Context, topic string) (\u003c-chan *Message, error)\n\tClose() error\n}\n```\n\nSupported Pub/Subs:\n\n- AMQP Pub/Sub [(`github.com/ThreeDotsLabs/watermill-amqp/v2`)](https://github.com/ThreeDotsLabs/watermill-amqp/)\n- Bolt Pub/Sub [(`github.com/ThreeDotsLabs/watermill-bolt`)](https://github.com/ThreeDotsLabs/watermill-bolt/)\n- Firestore Pub/Sub [(`github.com/ThreeDotsLabs/watermill-firestore`)](https://github.com/ThreeDotsLabs/watermill-firestore/)\n- Google Cloud Pub/Sub [(`github.com/ThreeDotsLabs/watermill-googlecloud`)](https://github.com/ThreeDotsLabs/watermill-googlecloud/)\n- HTTP Pub/Sub [(`github.com/ThreeDotsLabs/watermill-http`)](https://github.com/ThreeDotsLabs/watermill-http/)\n- io.Reader/io.Writer Pub/Sub [(`github.com/ThreeDotsLabs/watermill-io`)](https://github.com/ThreeDotsLabs/watermill-io/)\n- Kafka Pub/Sub [(`github.com/ThreeDotsLabs/watermill-kafka/v2`)](https://github.com/ThreeDotsLabs/watermill-kafka/)\n- NATS Pub/Sub [(`github.com/ThreeDotsLabs/watermill-nats`)](https://github.com/ThreeDotsLabs/watermill-nats/)\n- Redis Stream Pub/Sub [(`github.com/ThreeDotsLabs/watermill-redisstream`)](https://github.com/ThreeDotsLabs/watermill-redisstream/)\n- SQL Pub/Sub [(`github.com/ThreeDotsLabs/watermill-sql/v2`)](https://github.com/ThreeDotsLabs/watermill-sql/)\n\nAll Pub/Subs implementation documentation can be found in the [documentation](https://watermill.io/pubsubs/).\n\n## Unofficial libraries\n\nCan't find your favorite Pub/Sub or library integration? Check [Awesome Watermill](https://watermill.io/docs/awesome/).\n\nIf you know another library or are an author of one, please [add it to the list](https://github.com/ThreeDotsLabs/watermill/edit/master/docs/content/docs/awesome.md).\n\n## Contributing\n\nPlease check our [contributing guide](CONTRIBUTING.md).\n\n## Stability\n\nWatermill v1.0.0 has been released and is production-ready. The public API is stable and will not change without changing the major version.\n\nTo ensure that all Pub/Subs are stable and safe to use in production, we created a [set of tests](https://github.com/ThreeDotsLabs/watermill/blob/master/pubsub/tests/test_pubsub.go#L34) that need to pass for each of the implementations before merging to master.\nAll tests are also executed in [*stress*](https://github.com/ThreeDotsLabs/watermill/blob/master/pubsub/tests/test_pubsub.go#L171) mode - that means that we are running all the tests **20x** in parallel.\n\nAll tests are run with the race condition detector enabled (`-race` flag in tests).\n\nFor more information about debugging tests, you should check [tests troubleshooting guide](http://watermill.io/docs/troubleshooting/#debugging-pubsub-tests).\n\n## Benchmarks\n\nInitial tools for benchmarking Pub/Subs can be found in [watermill-benchmark](https://github.com/ThreeDotsLabs/watermill-benchmark).\n\nAll benchmarks are being done on a single 16 CPU VM instance, running one binary and dependencies in Docker Compose.\n\nThese numbers are meant to serve as a rough estimate of how fast messages can be processed by different Pub/Subs.\nKeep in mind that the results can be vastly different, depending on the setup and configuration (both much lower and higher).\n\nHere's the short version for message size of 16 bytes.\n\n| Pub/Sub                         | Publish (messages / s) | Subscribe (messages / s) |\n|---------------------------------|------------------------|--------------------------|\n| GoChannel                       | 315,776                | 138,743                  |\n| Redis Streams                   | 59,158                 | 12,134                   |\n| NATS Jetstream (16 Subscribers) | 50,668                 | 34,713                   |\n| Kafka (one node)                | 41,492                 | 101,669                  |\n| SQL (MySQL, batch size=100)     | 6,371                  | 2,794                    |\n| SQL (PostgreSQL, batch size=1)  | 2,831                  | 9,460                    |\n| Google Cloud Pub/Sub            | 3,027                  | 28,589                   |\n| AMQP (RabbitMQ)                 | 2,770                  | 14,604                   |\n\n## Support\n\nIf you didn't find the answer to your question in [the documentation](https://watermill.io/), feel free to ask us directly!\n\nPlease join us on the `#watermill` channel on the [Three Dots Labs Discord](https://discord.gg/QV6VFg4YQE).\n\n## Why the name?\n\nIt processes streams!\n\n## License\n\n[MIT License](./LICENSE)\n","funding_links":[],"categories":["Popular","Go","开源类库","Open source library","消息","kafka","Messaging","Repositories"],"sub_categories":["流处理","Stream Processing","检索及分析资料库","Search and Analytic Databases"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FThreeDotsLabs%2Fwatermill","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FThreeDotsLabs%2Fwatermill","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FThreeDotsLabs%2Fwatermill/lists"}