https://github.com/devgeniem/flynn-s3-backup
Docker image for Flynn cluster backup into AWS S3 which you can run as Flynn app
https://github.com/devgeniem/flynn-s3-backup
backup bash docker flynn s3
Last synced: about 2 months ago
JSON representation
Docker image for Flynn cluster backup into AWS S3 which you can run as Flynn app
- Host: GitHub
- URL: https://github.com/devgeniem/flynn-s3-backup
- Owner: devgeniem
- Created: 2017-02-17T16:36:12.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2019-09-30T11:25:21.000Z (over 6 years ago)
- Last Synced: 2025-03-30T15:02:55.881Z (about 1 year ago)
- Topics: backup, bash, docker, flynn, s3
- Language: Shell
- Homepage:
- Size: 15.6 KB
- Stars: 2
- Watchers: 6
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Hacky Flynn s3 backup script
This is a docker image which uses minio client to connect into s3 bucket in amazon.
It downloads [Flynn](https://flynn.io) cluster backup tar file from http api and transfers it into append-only versioned s3.
Backups will first get uploaded into highly available `STANDARD` zone in s3.
They are transitioned to `STANDARD_IA` after 60 days.
And then transitioned to `GLACIER` after 120 days.
You can decide when AWS expires old backups from `GLACIER`.
**Note:**This script will exit when the backup is ready and it counts in the Flynn's ability to spin up new process for the next backup.
## Usage
### Create and deploy the backup process as app in Flynn
```bash
$ docker build -t devgeniem/flynn-backup .
$ flynn create s3-backup --remote=""
$ flynn -a s3-backup docker push devgeniem/flynn-backup
```
### Create append only s3 bucket and aws iam user
**Note:** Check that [terraform cli](https://www.terraform.io/) has been installed before this.
```bash
$ cd terraform
# This creates new bucket and aws iam user credentials for the account that you provide
$ terraform apply
```
### Configure the backup process
```bash
# Get the flynn controller AUTH_KEY which you can use as FLYNN_AUTH_KEY later
$ flynn -a controller env get AUTH_KEY
# Setup the backup process
$ flynn -a s3-backup env set \
FLYNN_AUTH_KEY=auth-key-from-flynn \
AWS_S3_BUCKET=your-bucket-name \
AWS_ACCESS_KEY_ID=YYYYYYYYYYYYYYYY \
AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXX \
BACKUP_INTERVAL_SECONDS=10800 # Put any interval >1800 here
```
### Start the backup process
```
$ flynn -a s3-backup scale app=1
```
## How to retrieve old backups from versioned bucket
**Note:** Check that [aws commandline tools](http://docs.aws.amazon.com/cli/latest/userguide/installing.html) has been installed before this.
```bash
# This outputs json of all versions
$ aws s3api list-object-versions --bucket your-bucket-name
# This is how you restore older version
# In this example the version we want to restore is rehtEuCbtlaaJWnP0jfdHMQLkyrBPHG_
$ aws s3api get-object --bucket your-bucket-name \
--key backups/flynn-backup.tar \
--version-id rehtEuCbtlaaJWnP0jfdHMQLkyrBPHG_ \
flynn-backup.tar
```
## Maintainers
[@onnimonni](https://github.com/onnimonni)
## License
MIT