https://github.com/loafoe/hsdp-task-streaming-backup
hsdp-task for backing up a PostgreSQL database to S3
https://github.com/loafoe/hsdp-task-streaming-backup
backup function-as-a-service hsdp-task psql terraform
Last synced: 3 months ago
JSON representation
hsdp-task for backing up a PostgreSQL database to S3
- Host: GitHub
- URL: https://github.com/loafoe/hsdp-task-streaming-backup
- Owner: loafoe
- Created: 2021-03-18T12:46:35.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2021-11-29T13:10:40.000Z (over 4 years ago)
- Last Synced: 2026-02-28T17:59:37.904Z (3 months ago)
- Topics: backup, function-as-a-service, hsdp-task, psql, terraform
- Language: Shell
- Homepage:
- Size: 38.1 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# hsdp-task-streaming-backup
A Docker image that you can schedule on HSDP IronIO to perform PostgreSQL backups to an S3 bucket
# Features
- Streaming backups so not dependent on runner disk storage
- Compresses backups
# Usage
```hcl
module "siderite_backend" {
source = "philips-labs/siderite-backend/cloudfoundry"
cf_region = "eu-west"
cf_org_name = "hsdp-demo-org"
cf_user = var.cf_user
iron_plan = "medium-encrypted-4GB"
}
resource "hsdp_function" "psql_backup" {
name = "psql_backup"
docker_image = "philipslabs/hsdp-task-streaming-backup:v0.2.0"
command = ["/app/backup.sh"]
environment = {
DB_HOST = "postgres-xxx.eu-west-1.rds.amazonaws.com"
DB_PORT = "5432"
DB_USER = "[REDACTED]"
DB_PASSWORD = "[REDACTED]"
DB_NAME = "mydb"
BACKUP_NAME = "backups/mydb"
S3_ENDPOINT = "s3-eu-west-1.amazonaws.com"
S3_BUCKET = "cf-s3-xxx"
AWS_ACCESS_KEY_ID = "[REDACTED]"
AWS_SECRET_ACCESS_KEY = "[REDACTED]"
}
schedule {
run_every = "7d"
}
backend {
credentials = module.siderite_backend.credentials
}
}
```
# Bucket lifecycle policy
It is advised to set a S3 Bucket lifecycle policy. A good practice is to move your database backups to the `GLACIER` storage class after a couple of days and to set a expiration date to automatically delete older backups. The below policy moves dumps to `CLACIER` after 7 days and deletes them after 6 months (180 days)
```json
[
{
"Expiration": {
"Days": 180
},
"ID": "Move to Glacier and expire after 6 months",
"Prefix": "",
"Status": "Enabled",
"Transitions": [
{
"Days": 7,
"StorageClass": "GLACIER"
}
]
}
]
```
# License
License is MIT