https://github.com/choria-io/stream-replicator

Data replication for NATS JetStream
https://github.com/choria-io/stream-replicator

jetstream nats replication streaming

Last synced: about 1 year ago
JSON representation

Data replication for NATS JetStream

• Host: GitHub
• URL: https://github.com/choria-io/stream-replicator
• Owner: choria-io
• License: apache-2.0
• Created: 2022-03-09T10:01:28.000Z (over 4 years ago)
• Default Branch: main
• Last Pushed: 2024-11-11T07:48:28.000Z (over 1 year ago)
• Last Synced: 2025-04-11T19:21:47.918Z (about 1 year ago)
• Topics: jetstream, nats, replication, streaming
• Language: Go
• Homepage: https://choria.io
• Size: 11.6 MB
• Stars: 16
• Watchers: 3
• Forks: 5
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
![Choria Stream Replicator](https://choria-io.github.io/stream-replicator/logo.png)

## Overview

This is a Stream data replicator for [Choria Streams](https://choria.io/docs/streams/) aka [NATS JetStream](https://docs.nats.io/jetstream/).

Conceptually it's like a [JetStream Source](https://docs.nats.io/running-a-nats-service/nats_admin/jetstream_admin/replication)

that runs outside the JetStream process and can therefore be used to copy data between 2 completely independent clusters.

In addition to the core Source-like behavior this replicator has some features of particular use to the Choria project in 

that it can sample a Source Stream and, for any unique publisher, only replicate data on a set interval like once an hour

or when there is a significant change in the data being sent. 

When sampling it publishes advisories about senders that have not been seen recently and other significant events 

allowing a central aggregator to have a more granular time window awareness than it could get by looking at its data alone.

![Deployment](https://choria-io.github.io/stream-replicator/deployment.png)

Multiple Streams can be replicated and order is preserved.

 * [Documentation](https://choria-io.github.io/stream-replicator/)

 * [Discussions](https://github.com/choria-io/stream-replicator/discussions)

 * [Slack](https://slack.puppet.com/) in channel `#choria`

[![Go Report Card](https://goreportcard.com/badge/github.com/choria-io/stream-replicator)](https://goreportcard.com/report/github.com/choria-io/stream-replicator)

[![CodeQL](https://github.com/choria-io/stream-replicator/workflows/CodeQL/badge.svg)](https://github.com/choria-io/stream-replicator/actions/workflows/codeql.yaml)

[![Unit Tests](https://github.com/choria-io/stream-replicator/actions/workflows/test.yaml/badge.svg)](https://github.com/choria-io/stream-replicator/actions/workflows/test.yaml)