https://github.com/amirhnajafiz/jetstream-mirroring
NATS jetstream mirroring with Benthos.
https://github.com/amirhnajafiz/jetstream-mirroring
benthos golang mirroring nats nats-jetstream
Last synced: 6 months ago
JSON representation
NATS jetstream mirroring with Benthos.
- Host: GitHub
- URL: https://github.com/amirhnajafiz/jetstream-mirroring
- Owner: amirhnajafiz
- Created: 2022-05-10T04:02:33.000Z (about 3 years ago)
- Default Branch: master
- Last Pushed: 2024-01-06T06:19:22.000Z (over 1 year ago)
- Last Synced: 2024-06-21T14:05:21.146Z (12 months ago)
- Topics: benthos, golang, mirroring, nats, nats-jetstream
- Language: Go
- Homepage: https://golangexample.com/nats-jetstream-mirroring-with-benthos/
- Size: 73.2 KB
- Stars: 12
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# JetStream Mirroring
Mirroring our NATS (JetStream) cluster from different
regions to each other, using __Benthos__.
In this project, I provided an example in which I mirrored one stream inside a JetStream cluster to another stream
inside a different JetStream cluster.
In order to do this, I used __Benthos__ as a third party service between the two clusters.
## Benthos configs
Benthos is a high performance and resilient stream processor, able to connect
various sources and sinks in a range of brokering patterns and perform hydration, enrichments,
transformations and filters on payloads.
Read more about [Benthos](https://github.com/benthosdev/benthos).
### Input
Input is our first NATS (JetStream) service that is the main cluster.
```yaml
input:
label: "nats1"
nats_jetstream:
urls: [ "nats://0.0.0.0:4222" ]
queue: ""
subject: "snapp*" # subject
durable: ""
stream: "snapp" # stream
bind: false
deliver: all
```
### Output
Output is the other NATS (JetStream) service, which will be mirrored
by the main service.
```yaml
output:
label: "nats2"
nats_jetstream:
urls: [ "nats://0.0.0.0:4223" ]
subject: "snapp*"
```
## Example Setup
Use the following command to start Benthos, two NATS clusters, bootstrap service, two consumers, and one publisher
to see the mirroring between clusters:
```shell
docker-compose up
```