Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/italolelis/outboxer
A library that implements the outboxer pattern in go
https://github.com/italolelis/outboxer
distributed-systems go resiliency
Last synced: about 2 months ago
JSON representation
A library that implements the outboxer pattern in go
- Host: GitHub
- URL: https://github.com/italolelis/outboxer
- Owner: italolelis
- License: mit
- Created: 2019-02-01T09:50:13.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2024-07-29T11:20:07.000Z (5 months ago)
- Last Synced: 2024-07-31T20:51:11.726Z (5 months ago)
- Topics: distributed-systems, go, resiliency
- Language: Go
- Homepage:
- Size: 267 KB
- Stars: 153
- Watchers: 2
- Forks: 27
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
- awesome-go - outboxer - Outboxer is a go library that implements the outbox pattern. (Distributed Systems / Search and Analytic Databases)
- awesome-go-extra - outboxer - 02-01T09:50:13Z|2022-08-22T11:16:08Z| (Distributed Systems / Advanced Console UIs)
README
# Outboxer
[](https://github.com/italolelis/outboxer/actions)
[](https://codecov.io/gh/italolelis/outboxer)
[](https://goreportcard.com/report/github.com/italolelis/outboxer)
[](https://godoc.org/github.com/italolelis/outboxer)
Outboxer is a go library that implements the [outbox pattern](http://www.kamilgrzybek.com/design/the-outbox-pattern/).
## Getting Started
Outboxer simplifies the challenging work of orchestrating message reliability. Essentially we are trying to solve this question:
> How can producers reliably send messages when the broker/consumer is unavailable?
If you have a distributed system architecture and especially mainly deal
with [Event Driven Architecture](https://martinfowler.com/articles/201701-event-driven.html), you might
want to use *outboxer*.
The first thing to do is include the package in your project.
```sh
go get github.com/italolelis/outboxer
```
### Usage
Let's setup a simple example where you are using Google's `PubSub` and `Postgres` as your outbox pattern components:
```go
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
db, err := sql.Open("postgres", os.Getenv("DS_DSN"))
if err != nil {
fmt.Printf("could not connect to amqp: %s", err)
return
}
// we need to create a data store instance first
ds, err := postgres.WithInstance(ctx, db)
if err != nil {
fmt.Printf("could not setup the data store: %s", err)
return
}
defer ds.Close()
// we create an event stream passing the pusub connection
client, err := pubsub.NewClient(ctx, os.Getenv("GCP_PROJECT_ID"))
if err != nil {
fmt.Printf("failed to connect to gcp: %s", err)
return
}
es := pubsubOut.New(client)
// now we create an outboxer instance passing the data store and event stream
o, err := outboxer.New(
outboxer.WithDataStore(ds),
outboxer.WithEventStream(es),
outboxer.WithCheckInterval(1*time.Second),
outboxer.WithCleanupInterval(5*time.Second),
outboxer.WithCleanUpBefore(time.Now().AddDate(0, 0, -5)),
)
if err != nil {
fmt.Printf("could not create an outboxer instance: %s", err)
return
}
// here we initialize the outboxer checks and cleanup go rotines
o.Start(ctx)
defer o.Stop()
// finally we are ready to send messages
if err = o.Send(ctx, &outboxer.OutboxMessage{
Payload: []byte("test payload"),
Options: map[string]interface{}{
amqpOut.ExchangeNameOption: "test",
amqpOut.ExchangeTypeOption: "topic",
amqpOut.RoutingKeyOption: "test.send",
},
}); err != nil {
fmt.Printf("could not send message: %s", err)
return
}
// we can also listen for errors and ok messages that were send
for {
select {
case err := <-o.ErrChan():
fmt.Printf("could not send message: %s", err)
case <-o.OkChan():
fmt.Printf("message received")
return
}
}
```
## Features
Outboxer comes with a few implementations of Data Stores and Event Streams.
### Data Stores
- [Postgres DataStore](storage/postgres/)
- [MySQL DataStore](storage/mysql/)
- [SQLServer DataStore](storage/sqlserver/)
### Event Streams
- [AMQP EventStream](es/amqp/)
- [Kinesis EventStream](es/kinesis/)
- [SQS EventStream](es/sqs/)
- [GCP PubSub](es/pubsub/)
## Contributing
Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct and the process for submitting pull requests to us.
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details