https://github.com/goadesign/pulse
⚡Streamlined event streaming and distributed worker pools in Go⚡
https://github.com/goadesign/pulse
distributed-systems events go microservices streaming worker-pool workers
Last synced: 18 days ago
JSON representation
⚡Streamlined event streaming and distributed worker pools in Go⚡
- Host: GitHub
- URL: https://github.com/goadesign/pulse
- Owner: goadesign
- License: mit
- Created: 2023-03-06T05:07:50.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2025-04-12T18:38:37.000Z (9 months ago)
- Last Synced: 2025-04-12T19:35:19.777Z (9 months ago)
- Topics: distributed-systems, events, go, microservices, streaming, worker-pool, workers
- Language: Go
- Homepage:
- Size: 10.7 MB
- Stars: 13
- Watchers: 6
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Pulse
Pulse consists of a set of packages that enable event driven distributed
architectures at scale. Each package is designed to be used independently but
they can also be combined to implement more complex architectures.
## Replicated Maps
Replicated maps provide a mechanism for sharing data across distributed nodes
and receiving events when the data changes.
```mermaid
%%{init: {'themeVariables': { 'edgeLabelBackground': '#7A7A7A'}}}%%
flowchart LR
A[Node A]
B[Node B]
Map[Replicated Map]
A-->|Set|Map
Map-.->|Update|B
classDef userCode fill:#9A6D1F, stroke:#D9B871, stroke-width:2px, color:#FFF2CC;
classDef pulse fill:#25503C, stroke:#5E8E71, stroke-width:2px, color:#D6E9C6;
class A,B userCode;
class Map pulse;
linkStyle 0 stroke:#DDDDDD,color:#DDDDDD,stroke-width:3px;
linkStyle 1 stroke:#DDDDDD,color:#DDDDDD,stroke-width:3px;
```
See the [rmap package README](rmap/README.md) for more details.
## Streaming
Pulse streams provide a flexible mechanism for routing events across a fleet of
microservices. Streams can be used to implement pub/sub, fan-out and fan-in
topologies.
```mermaid
%%{init: {'themeVariables': { 'edgeLabelBackground': '#7A7A7A'}}}%%
flowchart LR
A[Event Producer]
subgraph SA[Stream A]
TA[Topic]
end
subgraph SB[Stream B]
TB[Topic]
end
B[Event Consumer]
A-->|Add|TA
A-->|Add|TB
TA-.->|Event|B
TB-.->|Event|B
classDef userCode fill:#9A6D1F, stroke:#D9B871, stroke-width:2px, color:#FFF2CC;
classDef pulse fill:#25503C, stroke:#5E8E71, stroke-width:2px, color:#D6E9C6;
class A,B userCode;
class SA,SB,TA,TB pulse;
linkStyle 0 stroke:#DDDDDD,color:#DDDDDD,stroke-width:3px;
linkStyle 1 stroke:#DDDDDD,color:#DDDDDD,stroke-width:3px;
linkStyle 2 stroke:#DDDDDD,color:#DDDDDD,stroke-width:3px;
linkStyle 3 stroke:#DDDDDD,color:#DDDDDD,stroke-width:3px;
```
See the [streaming package README](streaming/README.md) for more details.
## Dedicated Worker Pool
Pulse builds on top of [replicated maps](rmap/README.md) and
[streaming](streaming/README.md) to implement a dedicated worker pool where jobs
are dipatched to workers based on their key and a consistent hashing algorithm.
```mermaid
%%{init: {'themeVariables': { 'edgeLabelBackground': '#7A7A7A'}}}%%
flowchart LR
A[Job Producer]
subgraph Pool[Pool Node]
Sink
end
subgraph Worker[Pool Node]
Reader
B[Worker]
end
A-->|Job+Key|Sink
Sink-.->|Job|Reader
Reader-.->|Job|B
classDef userCode fill:#9A6D1F, stroke:#D9B871, stroke-width:2px, color:#FFF2CC;
classDef pulse fill:#25503C, stroke:#5E8E71, stroke-width:2px, color:#D6E9C6;
class A,B userCode;
class Pool,Sink,Reader,Worker pulse;
linkStyle 0 stroke:#DDDDDD,color:#DDDDDD,stroke-width:3px;
linkStyle 1 stroke:#DDDDDD,color:#DDDDDD,stroke-width:3px;
linkStyle 2 stroke:#DDDDDD,color:#DDDDDD,stroke-width:3px;
```
See the [pool package README](pool/README.md) for more details.
## Examples
See the [examples](examples) directory for examples of how to use the packages
in this repository.
If you are looking for a more complete example of how to use Pulse to build a
distributed system, check out the [weather](examples/weather) example.
## License
Pulse is licensed under the MIT license. See [LICENSE](LICENSE) for the full
license text.
## Contributing
See [CONTRIBUTING](CONTRIBUTING.md) for details on submitting patches and the
contribution workflow.
## Code of Conduct
This project adheres to the Contributor Covenant [code of conduct](CODE_OF_CONDUCT.md).
By participating, you are expected to uphold this code. Please report unacceptable
behavior to [pulse@goa.design](mailto:pulse@goa.design).
## Credits
Pulse was originally created by [Raphael Simon](https://github.com/raphael).