{"id":15489759,"url":"https://github.com/theandrew168/pg2s3","last_synced_at":"2025-04-22T18:29:26.642Z","repository":{"id":64306485,"uuid":"407661105","full_name":"theandrew168/pg2s3","owner":"theandrew168","description":"Simple PostgreSQL backups to S3-compatible storage","archived":false,"fork":false,"pushed_at":"2025-02-14T04:22:09.000Z","size":777,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-19T13:09:41.223Z","etag":null,"topics":["backup","postgresql","s3"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/theandrew168.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-17T19:43:27.000Z","updated_at":"2025-02-14T04:19:57.000Z","dependencies_parsed_at":"2025-03-04T04:30:58.906Z","dependency_job_id":"ecd25d14-0d35-43ae-a6df-f5e65a6b40ce","html_url":"https://github.com/theandrew168/pg2s3","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theandrew168%2Fpg2s3","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theandrew168%2Fpg2s3/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theandrew168%2Fpg2s3/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theandrew168%2Fpg2s3/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/theandrew168","download_url":"https://codeload.github.com/theandrew168/pg2s3/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250296846,"owners_count":21407101,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","postgresql","s3"],"created_at":"2024-10-02T07:07:41.407Z","updated_at":"2025-04-22T18:29:26.582Z","avatar_url":"https://github.com/theandrew168.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pg2s3\nSimple PostgreSQL backups to S3-compatible storage\n\n## Overview\nThis project strives to be a simple and reliable backup solution for [PostgreSQL](https://www.postgresql.org/) databases.\nIn general, pg2s3 dumps a given database and uploads it to an S3-compatible storage bucket.\nHowever, there is a bit more nuance involved in bookkeeping, restoration, and pruning.\n\n## Install\nThe pg2s3 tool is distributed as a single, static binary for all major platforms.\nIt is also released as a `.deb` for Debian-based Linux environments.\nCheck the [releases page](https://github.com/theandrew168/pg2s3/releases) to find and download the latest version.\n\nAdditionally, the environment where pg2s3 is executed must have `pg_dump` and `pg_restore` installed.\nThese tools are part of the collection of [PostgreSQL Client Applications](https://www.postgresql.org/docs/12/reference-client.html).\nOn an Ubuntu server, these tools are contained within the package `postgresql-client-\u003cversion\u003e` based on the major version of PostgreSQL being used.\n\n## Configuration\nConfiguration for pg2s3 is handled exclusively through a config file written in [TOML](https://github.com/toml-lang/toml).\nBy default, pg2s3 will look for a config file named `pg2s3.conf` in the current directory.\nThis file can be overridden by using the `-conf` flag.\n\nNote that the S3 bucket defined by `s3_url` must be created outside of this tool.\nBucket creation has more configuration and security options than pg2s3 is positioned to deal with.\n\nAdditionally, the value defined by `backup.retention` simply refers to the _number_ of backups kept during a prune.\nIt has nothing to do with a backup's age or total bucket size.\nIf `backup.schedule` is set, you'll want to consider the scheduling frequency when determining an appropriate retention count.\n\nThe following settings are required to run pg2s3:\n\n| Setting            | Required? | Description |\n| ------------------ | --------- | ----------- |\n| `pg_url`           | Yes       | PostgreSQL connection string |\n| `s3_url`           | Yes       | S3-compatible storage connection string |\n| `backup.prefix`    | No        | Prefix attached to the name of each backup (default `\"pg2s3\"`) |\n| `backup.retention` | No        | Number of backups to retain after pruning |\n| `backup.schedule`  | No        | Backup schedule as a standard cron expression (UTC) |\n| `restore.schemas`  | No        | List of schemas to restore (default `[\"public\"]`) |\n\n## Encryption\nBackups managed by pg2s3 can be optionally encrypted using [age](https://github.com/FiloSottile/age).\nTo enable this feature, an age public key must be defined within the config file.\nNote that the private key associated with this public key must be kept safe and secure!\nWhen restoring a backup, pg2s3 will prompt for the private key.\nThis key is intentionally absent from pg2s3's configuration in order to require user intervention for any data decryption.\n\n| Setting                  | Required? | Description |\n| ------------------------ | --------- | ----------- |\n| `encryption.public_keys` | No        | Public keys for backup encryption |\n\n## Usage\nThe pg2s3 command-line tool offers three mutually-exclusive actions:\n* `pg2s3 backup` - Create a new backup and upload to S3\n* `pg2s3 restore` - Download the latest backup from S3 and restore\n* `pg2s3 prune` - Prune old backups from S3\n\nIf none of these are provided, pg2s3 will attempt to run in scheduled mode: sleeping until `backup.schedule` arrives and then performing a backup + prune.\n\n## Local Development\nTo develop and test locally, containers for [PostgreSQL](https://www.postgresql.org/) and [MinIO](https://min.io/) must be running:\n```\ndocker compose up -d\n```\n\nThese containers can be stopped via:\n```\ndocker compose down\n```\n\n## Testing\nWith the above containers running:\n```\ngo test ./...\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftheandrew168%2Fpg2s3","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftheandrew168%2Fpg2s3","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftheandrew168%2Fpg2s3/lists"}