https://github.com/qovery/replibyte
Seed your development database with real data ⚡️
https://github.com/qovery/replibyte
aws backup cloud cloudnative database mongodb mysql postgres postgresql rust rust-lang s3
Last synced: 6 months ago
JSON representation
Seed your development database with real data ⚡️
- Host: GitHub
- URL: https://github.com/qovery/replibyte
- Owner: Qovery
- License: gpl-3.0
- Created: 2022-02-15T20:31:25.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-08-14T20:20:41.000Z (about 1 year ago)
- Last Synced: 2025-05-08T08:43:22.525Z (6 months ago)
- Topics: aws, backup, cloud, cloudnative, database, mongodb, mysql, postgres, postgresql, rust, rust-lang, s3
- Language: Rust
- Homepage: https://www.replibyte.com
- Size: 7.49 MB
- Stars: 4,244
- Watchers: 23
- Forks: 130
- Open Issues: 118
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Seed Your Development Database With Real Data ⚡️
Replibyte is a blazingly fast tool to seed your databases with your production data while keeping sensitive data safe 🔥
## Prerequisites
- MacOSX / Linux / Windows
- Nothing more! Replibyte is stateless and does not require anything special.
## Usage
Create a dump
```shell
replibyte -c conf.yaml dump create
```
List all dumps
```shell
replibyte -c conf.yaml dump list
type name size when compressed encrypted
PostgreSQL dump-1647706359405 154MB Yesterday at 03:00 am true true
PostgreSQL dump-1647731334517 152MB 2 days ago at 03:00 am true true
PostgreSQL dump-1647734369306 149MB 3 days ago at 03:00 am true true
```
Restore the latest dump in a local container
```shell
replibyte -c conf.yaml dump restore local -v latest -i postgres -p 5432
```
Restore the latest dump in a remote database
```shell
replibyte -c conf.yaml dump restore remote -v latest
```
## Features
- [x] Support data dump and restore for PostgreSQL, MySQL and MongoDB
- [x] Analyze your data schema 🔎
- [x] Replace sensitive data with fake data
- [x] Works on large database (> 10GB)
- [x] Database Subsetting: Scale down a production database to a more reasonable size 🔥
- [x] Start a local database with the production data in a single command 🔥
- [x] On-the-fly data (de)compression (Zlib)
- [x] On-the-fly data de/encryption (AES-256)
- [x] Fully stateless (no server, no daemon) and lightweight binary 🍃
- [x] Use [custom transformers](examples/wasm)
Here are the features we plan to support
- [ ] Auto-detect and version database schema change
- [ ] Auto-detect sensitive fields
- [ ] Auto-clean backed up data
## Getting Started
1. [How Replibyte works](https://www.replibyte.com/docs/how-replibyte-works)
2. Initial setup:
1. [Install](https://www.replibyte.com/docs/getting-started/installation)
2. [Configure](https://www.replibyte.com/docs/getting-started/configuration)
3. Step-by-step guides:
1. [Create a dump](https://www.replibyte.com/docs/guides/create-a-dump)
2. [Restore a dump](https://www.replibyte.com/docs/guides/restore-a-dump)
3. [Subset a dump](https://www.replibyte.com/docs/guides/subset-a-dump)
4. [Delete a dump](https://www.replibyte.com/docs/guides/delete-a-dump)
5. Deploy Replibyte
1. [Container](https://www.replibyte.com/docs/guides/deploy-replibyte/container)
2. [Qovery](https://www.replibyte.com/docs/guides/deploy-replibyte/qovery)
## Demo
[](https://www.youtube.com/watch?v=IKeLnZvECQw)
## Contributing
Check [here](https://www.replibyte.com/docs/contributing).
## Thanks
Thanks to all people sharing their ideas to make Replibyte better. We do appreciate it. I would also thank [AirByte](https://airbyte.com/),
a great product and a trustworthy source of inspiration for this project.
---
Replibyte is initiated and maintained by [Qovery](https://www.qovery.com?ref=replibyte-readme).