{"id":14973921,"url":"https://github.com/loafoe/hsdp-task-streaming-backup","last_synced_at":"2026-03-03T01:39:17.572Z","repository":{"id":45890910,"uuid":"349070876","full_name":"loafoe/hsdp-task-streaming-backup","owner":"loafoe","description":"hsdp-task for backing up a PostgreSQL database to S3","archived":false,"fork":false,"pushed_at":"2021-11-29T13:10:40.000Z","size":39,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-02-28T17:59:37.904Z","etag":null,"topics":["backup","function-as-a-service","hsdp-task","psql","terraform"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/loafoe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-18T12:46:35.000Z","updated_at":"2022-03-18T12:28:21.000Z","dependencies_parsed_at":"2022-09-23T09:43:17.461Z","dependency_job_id":null,"html_url":"https://github.com/loafoe/hsdp-task-streaming-backup","commit_stats":null,"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/loafoe/hsdp-task-streaming-backup","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loafoe%2Fhsdp-task-streaming-backup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loafoe%2Fhsdp-task-streaming-backup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loafoe%2Fhsdp-task-streaming-backup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loafoe%2Fhsdp-task-streaming-backup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/loafoe","download_url":"https://codeload.github.com/loafoe/hsdp-task-streaming-backup/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loafoe%2Fhsdp-task-streaming-backup/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29988054,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-01T22:42:38.399Z","status":"ssl_error","status_checked_at":"2026-03-01T22:41:51.863Z","response_time":124,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","function-as-a-service","hsdp-task","psql","terraform"],"created_at":"2024-09-24T13:49:41.106Z","updated_at":"2026-03-03T01:39:17.552Z","avatar_url":"https://github.com/loafoe.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# hsdp-task-streaming-backup\n\nA Docker image that you can schedule on HSDP IronIO to perform PostgreSQL backups to an S3 bucket\n\n# Features\n- Streaming backups so not dependent on runner disk storage\n- Compresses backups\n\n# Usage\n\n```hcl\nmodule \"siderite_backend\" {\n  source = \"philips-labs/siderite-backend/cloudfoundry\"\n\n  cf_region   = \"eu-west\"\n  cf_org_name = \"hsdp-demo-org\"\n  cf_user     = var.cf_user\n  iron_plan   = \"medium-encrypted-4GB\"\n}\n\nresource \"hsdp_function\" \"psql_backup\" {\n  name         = \"psql_backup\"\n  docker_image = \"philipslabs/hsdp-task-streaming-backup:v0.2.0\"\n  command      = [\"/app/backup.sh\"]\n\n  environment = {\n    DB_HOST               = \"postgres-xxx.eu-west-1.rds.amazonaws.com\"\n    DB_PORT               = \"5432\"\n    DB_USER               = \"[REDACTED]\"\n    DB_PASSWORD           = \"[REDACTED]\"\n    DB_NAME               = \"mydb\"\n\n    BACKUP_NAME           = \"backups/mydb\"\n    S3_ENDPOINT           = \"s3-eu-west-1.amazonaws.com\"\n    S3_BUCKET             = \"cf-s3-xxx\"\n    AWS_ACCESS_KEY_ID     = \"[REDACTED]\"\n    AWS_SECRET_ACCESS_KEY = \"[REDACTED]\"\n  }\n\n  schedule {\n    run_every = \"7d\"\n  }\n\n  backend {\n    credentials = module.siderite_backend.credentials\n  }\n}\n```\n\n# Bucket lifecycle policy\nIt is advised to set a S3 Bucket lifecycle policy. A good practice is to move your database backups to the `GLACIER` storage class after a couple of days and to set a expiration date to automatically delete older backups. The below policy moves dumps to `CLACIER` after 7 days and deletes them after 6 months (180 days)\n\n```json\n[\n  {\n    \"Expiration\": {\n      \"Days\": 180\n    },\n    \"ID\": \"Move to Glacier and expire after 6 months\",\n    \"Prefix\": \"\",\n    \"Status\": \"Enabled\",\n    \"Transitions\": [\n      {\n        \"Days\": 7,\n        \"StorageClass\": \"GLACIER\"\n      }\n    ]\n  }\n]\n```\n\n# License\n\nLicense is MIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floafoe%2Fhsdp-task-streaming-backup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Floafoe%2Fhsdp-task-streaming-backup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floafoe%2Fhsdp-task-streaming-backup/lists"}