Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/zhashkevych/spacer
🚀 Golang library & CLI tool that helps you export encrypted database dumps to S3-compatible storages
https://github.com/zhashkevych/spacer
cli cli-app digitalocean digitalocean-spaces dump golang golang-library pgdump pgrestore postgres postgres-dump psql s3 s3-bucket s3-storage
Last synced: 26 days ago
JSON representation
🚀 Golang library & CLI tool that helps you export encrypted database dumps to S3-compatible storages
- Host: GitHub
- URL: https://github.com/zhashkevych/spacer
- Owner: zhashkevych
- Created: 2020-12-14T17:38:16.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2020-12-18T14:26:37.000Z (almost 4 years ago)
- Last Synced: 2024-09-28T18:05:21.844Z (about 1 month ago)
- Topics: cli, cli-app, digitalocean, digitalocean-spaces, dump, golang, golang-library, pgdump, pgrestore, postgres, postgres-dump, psql, s3, s3-bucket, s3-storage
- Language: Go
- Homepage:
- Size: 86.9 KB
- Stars: 10
- Watchers: 3
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Spacer is the Golang library & CLI tool that helps you manage Database dumping/restoring with ease
Spacer provides functionality to dump Postgres database, encrypt and export it to S3-compatible object storage.
Also, it can restore Database using latest saved dump file.
Spacer uses symmetric-key encryption (AES/GCM). You can generate secrey key using keygen, simply run `make keygen`.
Example (dump, encrypt & save):
```go
package main
import (
"context"
"github.com/zhashkevych/spacer/pkg"
"log"
"time"
"io/ioutil"
)
func main() {
// Create DB client
postgres, err := spacer.NewPostgres("localhost", "5432", "postgres", "qwerty", "postgres")
if err != nil {
log.Fatalf("failed to create Postgres: %s", err.Error())
}
// Create dump
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
if err := postgres.Dump(ctx, "test_dump.sql"); err != nil {
log.Fatalf("failed to create dump: %s", err.Error())
}
// OR use File to create files using following scheme .dump_.sql
dumpFile, err := spacer.NewDumpFile("local")
if err != nil {
log.Fatalf("failed to create dump file: %s", err.Error())
}
if err := postgres.Dump(ctx, dumpFile.Name()); err != nil {
log.Fatalf("failed to create dump: %s", err.Error())
}
// Now you can do anything you want
// For example, encrypt and save it to Object Storage
encryptor, err := spacer.NewEncryptor([]byte("your-key-should-have-32-bytes!!!"))
if err != nil {
log.Fatalf("failed to create Encryptor: %s", err.Error())
}
// encrypt file
fileData, err := ioutil.ReadAll(dumpFile.Reader())
if err != nil {
log.Fatalf("failed to read dump file: %s", err.Error())
}
encrypted, err := encryptor.Encrypt(fileData)
if err != nil {
log.Fatalf("failed to encrypt: %s", err.Error())
}
if err := dumpFile.Write(encrypted); err != nil {
log.Fatalf("failed to rewrite encrypted data to file: %s", err.Error())
}
// and save
storage, err := spacer.NewSpacesStorage("ams3.digitaloceanspaces.com", "test-bucket", "your-access-key", "your-secret-key")
if err != nil {
log.Fatalf("failed to create SpacesStorage: %s", err.Error())
}
url, err := storage.Save(ctx, dumpFile, "dumps")
if err != nil {
log.Fatalf("failed to save dump file: %s", err.Error())
}
log.Println("Dump exported to", url)
}
```
Restore:
```go
package main
import (
"context"
"github.com/zhashkevych/spacer/pkg"
"log"
)
func main() {
postgres, err := spacer.NewPostgres("localhost", "5432", "postgres", "qwerty", "postgres")
if err != nil {
log.Fatalf("failed to create Postgres: %s", err.Error())
}
storage, err := spacer.NewSpacesStorage("ams3.digitaloceanspaces.com", "test-bucket", "your-access-key", "your-secret-key")
if err != nil {
log.Fatalf("failed to create SpacesStorage: %s", err.Error())
}
ctx := context.Background()
dumpFile, err := storage.GetLatest(ctx, "dumps", "local")
if err != nil {
log.Fatalf("failed to get latest dump file: %s", err.Error())
}
if err := postgres.Restore(ctx, dumpFile.Name()); err != nil {
log.Fatalf("failed to restore latest dump: %s", err.Error())
}
}
```
### Use it in your command line:
Prerequisites:
- go 1.15
- psql & pg_dump installed
Steps:
1) Run `make keygen` to generate encryption key
2) Set connection info variables in .env file (look at .env.example)
3) Run `make build` to create binaries
4) Run `./.bin/spacer export -p -f ` to create dump and export it to storage OR `./bin/spacer restore` to restore DB from latest dump in your storage bucket
Example:
```shell script
❯ ./.bin/spacer spacer/git/main
NAME:
CLI tool that helps you export encrypted Postgres dumps to DigitalOcean Spaces - A new cli application
USAGE:
spacer [global options] command [command options] [arguments...]
COMMANDS:
export, e create and export dump
restore, r restore from latest dump
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
--help, -h show help (default: false)
❯ ./.bin/spacer export -p local -f dumps spacer/git/main
2020/12/18 16:22:45 Starting export
2020/12/18 16:22:47 dump successfully exported to https://.ams3.digitaloceanspaces.com/dumps/local.dump_1608301365.tar.gz
❯ ./.bin/spacer restore -p local -f dumps spacer/git/main
2020/12/18 16:24:29 Starting restore
2020/12/18 16:24:30 DB successfully restored from latest dump
```
## TODO
- Implement dump files compression