{"id":50430452,"url":"https://github.com/theoneoh1/pg-pitr","last_synced_at":"2026-05-31T14:01:06.718Z","repository":{"id":359827689,"uuid":"1247645435","full_name":"TheOneOh1/pg-pitr","owner":"TheOneOh1","description":"PostgreSQL Point-in-Time Recovery (PITR) with pgBackRest","archived":false,"fork":false,"pushed_at":"2026-05-23T16:02:51.000Z","size":33,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-23T18:05:39.530Z","etag":null,"topics":["backup","database","pgbackrest","pitr","postgresql","restore"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TheOneOh1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-23T15:38:48.000Z","updated_at":"2026-05-23T16:02:55.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/TheOneOh1/pg-pitr","commit_stats":null,"previous_names":["theoneoh1/pg-pitr"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/TheOneOh1/pg-pitr","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheOneOh1%2Fpg-pitr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheOneOh1%2Fpg-pitr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheOneOh1%2Fpg-pitr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheOneOh1%2Fpg-pitr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TheOneOh1","download_url":"https://codeload.github.com/TheOneOh1/pg-pitr/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheOneOh1%2Fpg-pitr/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33733754,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-31T02:00:06.040Z","response_time":95,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","database","pgbackrest","pitr","postgresql","restore"],"created_at":"2026-05-31T14:01:05.741Z","updated_at":"2026-05-31T14:01:06.707Z","avatar_url":"https://github.com/TheOneOh1.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PostgreSQL Point-in-Time Recovery (PITR) with pgBackRest\n\nContinuous transaction archiving, scheduled backups, and automated one-command recovery for PostgreSQL 14+ clusters on Ubuntu 24.04 VM or bare-metal instances.\n\n[![PostgreSQL Version](https://img.shields.io/badge/PostgreSQL-14%20%7C%2015%20%7C%2016-blue.svg)](https://www.postgresql.org/)\n[![pgBackRest Version](https://img.shields.io/badge/pgBackRest-2.50+-blue.svg)](https://pgbackrest.org/)\n[![License](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)\n[![Linting](https://img.shields.io/badge/ShellCheck-Passing-brightgreen.svg)](https://www.shellcheck.net/)\n\n---\n\n## Table of Contents\n\n1. [Key Concepts for Beginners](#key-concepts-for-beginners)\n2. [Architecture Diagrams](#architecture-diagrams)\n3. [Project Directory Tour](#project-directory-tour)\n4. [Local Sandbox Tutorial (Step-by-Step Learning Guide)](#local-sandbox-tutorial-step-by-step-learning-guide)\n5. [Daily Backup Operations Guide](#daily-backup-operations-guide)\n6. [Production Deployment Walkthrough](#production-deployment-walkthrough)\n7. [Troubleshooting \u0026 Operational FAQ](#troubleshooting--operational-faq)\n8. [Scripting Standards \u0026 Contributing](#scripting-standards--contributing)\n9. [License](#license)\n\n---\n\n## Key Concepts for Beginners\n\nIf you are new to database administration or backup systems, review these concepts to understand how this repository operates:\n\n### 1. Point-in-Time Recovery (PITR)\nTraditional backups only allow you to restore the database to the exact moment the backup was taken (e.g., last night at 2:00 AM). If a disaster occurs at 4:30 PM, you lose 14.5 hours of transactions.\nPoint-in-Time Recovery (PITR) solves this. By combining a physical base backup with a continuous stream of Write-Ahead Logs (WAL), you can reconstruct the database state at any specific second in time, like rewinding a tape.\n\n### 2. Write-Ahead Logs (WAL)\nPostgreSQL writes every database change (inserts, updates, deletes) to an internal transaction log called the Write-Ahead Log (WAL) before modifying the actual data files. This log is split into 16 MB files called WAL segments.\n\n### 3. Continuous WAL Archiving\nWAL archiving is the process of copying completed 16 MB WAL segments from the live database server to a secure, separate backup repository as soon as they fill up. Even if the primary database disk fails completely, the transaction history remains safe in the repository.\n\n### 4. Stanza\nIn pgBackRest terminology, a **Stanza** is a named configuration block representing a specific PostgreSQL database cluster and its associated backup repository details. This repository uses the default stanza name `main`.\n\n### 5. Backup Types\npgBackRest uses three backup types to optimize speed and disk storage:\n* **Full Backup:** A complete copy of all database files. This forms the anchor of the restore chain.\n* **Differential Backup:** A copy of only the files that have changed since the last **Full** backup. It is faster and smaller than a full backup.\n* **Incremental Backup:** A copy of only the files that have changed since the last backup of **any** type (Full, Differential, or Incremental). This uses the least disk space.\n\n---\n\n## Architecture Diagrams\n\n### Backup and Continuous Archiving Architecture\n\nThis diagram shows how transactions are continuously written to WAL segments, which pgBackRest immediately pushes to the backup repository while cron schedules regular full, differential, and incremental backups.\n\n```mermaid\ngraph TD\n    PG[PostgreSQL Database Instance] --\u003e|1. Writes transactions| WAL[WAL Segments 16MB]\n    WAL --\u003e|2. archive-push command| REPO[(pgBackRest Repository)]\n    CRON[System Cron Daemon] --\u003e|3. Triggers schedule| PG\n    PG --\u003e|4. Full / Diff / Incr Backups| REPO\n    \n    subgraph Local or Remote Storage\n        REPO\n    end\n```\n\n### Point-in-Time Recovery (PITR) Workflow\n\nThis diagram shows the process when the automated recovery script is executed. The database directory is cleared, the closest base backup is extracted, transaction logs are replayed up to the target timestamp, and the database is promoted.\n\n```mermaid\ngraph TD\n    START[1. Stop PostgreSQL Service] --\u003e MOVE[2. Move corrupted data directory to safe backup location]\n    MOVE --\u003e RESTORE[3. Run pgBackRest restore command with target time]\n    RESTORE --\u003e BASE[4. Restore closest base backup before target time]\n    BASE --\u003e REPLAY[5. Replay WAL segments in sequential order]\n    REPLAY --\u003e STOP[6. Halt replay at exact target timestamp]\n    STOP --\u003e PROMOTE[7. Promote database cluster to read-write mode]\n    PROMOTE --\u003e STARTDB[8. Start PostgreSQL and verify active connections]\n```\n\n---\n\n## Project Directory Tour\n\nNavigate the repository using this structural guide:\n\n* **`.github/workflows/ci.yml`**: Automates syntax checking and ShellCheck linting on all repository bash scripts during push and pull requests.\n* **`configs/pgbackrest.example.conf`**: Standalone configuration template containing optimal production parameters (asynchronous archiving, spool paths, retention sizes, and compression levels).\n* **`configs/postgresql.example.conf`**: Standalone configuration parameters to enable WAL archiving within `postgresql.conf`.\n* **`docker/`**: The local containerized sandbox environment.\n  - `Dockerfile`: Pre-configures Ubuntu 24.04 with PostgreSQL, pgBackRest, cron, and sudo.\n  - `docker-compose.yml`: Spins up the sandbox container and mounts the local `scripts/` directory.\n  - `entrypoint.sh`: Automates database initialization, stanza creation, mock data loading, and baseline backup creation on container start.\n* **`docs/setup-guide.md`**: Production step-by-step setup, configuration, crontab, and cron syntax guide.\n* **`docs/understanding-pitr.md`**: Standard Operating Procedure (SOP) covering concepts, verification checks, and disaster scenarios.\n* **`scripts/auto-recovery.sh`**: The core, automated, one-command PITR execution script.\n* **`scripts/backup-now.sh`**: A helper script to run manual backups and view repository statistics.\n* **`scripts/setup-pgbackrest.sh`**: An idempotent script to automate installation and verification on a clean server.\n\n---\n\n## Local Sandbox Tutorial (Step-by-Step Learning Guide)\n\nThis sandbox environment allows you to simulate database operations, delete data, and perform a Point-in-Time Recovery safely on your local computer.\n\n### Step 1: Start the Sandbox\n\nFrom your host terminal, navigate to the `docker` directory and run the compose command:\n```bash\ncd docker\ndocker-compose up -d --build\n```\nThis starts a container named `pg_pitr_sandbox`. It automatically initializes the database, creates the pgBackRest configuration, starts WAL archiving, loads a sample database table, and takes a baseline full backup.\n\n### Step 2: Access the Running Container Shell\n\nOpen a bash terminal inside the running sandbox:\n```bash\ndocker exec -it pg_pitr_sandbox bash\n```\n\n### Step 3: Verify the Stanza and Baseline Data\n\nConfirm that pgBackRest has successfully recorded the stanza and baseline backup:\n```bash\nsudo -u postgres pgbackrest info\n```\nNow, query the sample employee table inside the `testpitr` database:\n```bash\nsudo -u postgres psql -d testpitr -c \"SELECT * FROM employees;\"\n```\nExpected output:\n```\n id | name  | salary |         created_at         \n----+-------+--------+----------------------------\n  1 | Levi |  50000 | 2026-05-20 22:15:00.123456\n  2 | Mikasa  |  60000 | 2026-05-20 22:15:00.123456\n  3 | Kenny |  70000 | 2026-05-20 22:15:00.123456\n```\n\n### Step 4: Record Your Recovery Target Timestamp\n\nPrint the current system time in the correct format. This is the moment in time you will restore the database to:\n```bash\ndate \"+%Y-%m-%d %H:%M:%S\"\n```\n*Note this timestamp (e.g., `2026-05-20 22:16:05`).*\n\n### Step 5: Simulate a Database Disaster\n\nWait 5 seconds, then execute a query to delete all employee records:\n```bash\nsudo -u postgres psql -d testpitr -c \"DELETE FROM employees;\"\n```\nQuery the database again to verify that all data is gone:\n```bash\nsudo -u postgres psql -d testpitr -c \"SELECT * FROM employees;\"\n```\nExpected output:\n```\n(0 rows)\n```\n\n### Step 6: Execute Automated Point-in-Time Recovery\n\nExecute the recovery script, passing the target time recorded in Step 4 as the single argument:\n```bash\nbash /scripts/auto-recovery.sh \"YOUR-TIMESTAMP-HERE\"\n```\nDuring execution, the script will:\n1. Stop the PostgreSQL service.\n2. Back up the active database directory to `/var/lib/postgresql/14/main-old-\u003ctimestamp\u003e`.\n3. Restore the baseline physical backup using pgBackRest.\n4. Instruct PostgreSQL to replay transactions and stop at the exact target second.\n5. Automatically promote the database to read-write mode.\n6. Verify database readiness.\n\n### Step 7: Confirm Successful Recovery\n\nQuery the table to verify that the deleted employee rows have been restored:\n```bash\nsudo -u postgres psql -d testpitr -c \"SELECT * FROM employees;\"\n```\nExpected output:\n```\n id | name  | salary |         created_at         \n----+-------+--------+----------------------------\n  1 | Levi |  50000 | 2026-05-20 22:15:00.123456\n  2 | Mikasa  |  60000 | 2026-05-20 22:15:00.123456\n  3 | Kenny |  70000 | 2026-05-20 22:15:00.123456\n```\nYou have successfully re-wound your database back in time!\n\n---\n\n## Daily Backup Operations Guide\n\nOnce your environment is configured, use these operations to manage backups:\n\n### 1. Triggering Manual Backups\n\nThe script `scripts/backup-now.sh` is a CLI helper that executes backups and prints status logs. It supports three types: `full`, `diff`, and `incr`.\n\n* **Incremental Backup (Default):**\n  Only copies blocks modified since the last backup of any type.\n  ```bash\n  sudo ./scripts/backup-now.sh --type=incr\n  ```\n* **Differential Backup:**\n  Only copies blocks modified since the last full backup.\n  ```bash\n  sudo ./scripts/backup-now.sh --type=diff\n  ```\n* **Full Backup:**\n  Copies the entire database cluster.\n  ```bash\n  sudo ./scripts/backup-now.sh --type=full\n  ```\n\n### 2. Checking Repository Information\n\nTo view the history, type, sizes, and timestamps of all backups stored in your repository:\n```bash\nsudo -u postgres pgbackrest info\n```\n\n### 3. Running Repository Integrity Verification\n\nRun a verification sweep to check for file corruption in the backup repository:\n```bash\nsudo -u postgres pgbackrest --stanza=main verify\n```\n\n---\n\n## Production Deployment Walkthrough\n\nTo deploy this setup on an active, clean Ubuntu 24.04 server:\n\n### Step 1: Run the Idempotent Installation Script\n\nCopy this repository to your target server and run the automated setup script with root privileges:\n```bash\nsudo chmod +x scripts/*.sh\nsudo ./scripts/setup-pgbackrest.sh\n```\nThis script automates package installations, folder setup, permissions, deploys `/etc/pgbackrest.conf`, configures WAL archiving, restarts PostgreSQL, registers the stanza, and runs a validation check.\n\n### Step 2: Configure the Backup Schedule\n\nConfigure system cron jobs for the `postgres` user. Avoid using root's crontab.\n```bash\nsudo -u postgres crontab -e\n```\nAdd these lines to configure weekly full, daily differential, and hourly incremental backups:\n```cron\n# Full backup - weekly on Sunday at 01:00 AM\n0 1 * * 0 pgbackrest --stanza=main backup --type=full\n\n# Differential backup - daily Monday through Saturday at 02:00 AM\n0 2 * * 1-6 pgbackrest --stanza=main backup --type=diff\n\n# Incremental backup - hourly (skips hours covered by full and diff)\n0 3-23 * * * pgbackrest --stanza=main backup --type=incr\n0 0 * * 1-6 pgbackrest --stanza=main backup --type=incr\n\n# Weekly repository verification - Sunday at 03:30 AM\n30 3 * * 0 pgbackrest --stanza=main verify\n```\n\n### Step 3: Run the First Physical Backup\n\nExecute a manual full backup to anchor the cron backup chain:\n```bash\nsudo -u postgres pgbackrest --stanza=main backup --type=full\n```\n\n---\n\n## Troubleshooting \u0026 Operational FAQ\n\n### Q1: The restore fails with an error: \"unable to restore because path contains files\"\n* **Why:** As a safety mechanism, pgBackRest refuses to restore files into a non-empty data directory.\n* **Resolution:** You must stop PostgreSQL and move or delete the contents of your data directory. The `auto-recovery.sh` script does this automatically. If running manually:\n  ```bash\n  sudo systemctl stop postgresql\n  sudo mv /var/lib/postgresql/14/main /var/lib/postgresql/14/main-old\n  sudo mkdir -p /var/lib/postgresql/14/main\n  sudo chown postgres:postgres /var/lib/postgresql/14/main\n  sudo chmod 700 /var/lib/postgresql/14/main\n  ```\n\n### Q2: After a recovery, I get errors: \"cannot execute INSERT in a read-only transaction\"\n* **Why:** By default, PostgreSQL remains in a read-only standby/recovery state after completing a restore.\n* **Resolution:** Ensure you pass `--target-action=promote` to your restore command (which is default in `auto-recovery.sh`). If the database is already running and read-only, promote it manually:\n  ```bash\n  sudo -u postgres psql -c \"SELECT pg_promote();\"\n  ```\n\n### Q3: How do I monitor repository storage growth?\n* **Resolution:** Run a disk usage check on your repository path:\n  ```bash\n  df -h /var/lib/pgbackrest\n  ```\n  If repository usage exceeds 80%, consider lowering the full backup retention counts in `/etc/pgbackrest.conf` (e.g., set `repo1-retention-full=4`) or adding disk space.\n\n---\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftheoneoh1%2Fpg-pitr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftheoneoh1%2Fpg-pitr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftheoneoh1%2Fpg-pitr/lists"}