{"id":23906832,"url":"https://github.com/tks-devops/postgrass-buckup-automation","last_synced_at":"2026-02-06T09:33:43.717Z","repository":{"id":269488098,"uuid":"907542473","full_name":"Tks-Devops/postgrass-buckup-automation","owner":"Tks-Devops","description":"automated backup system for PostgreSQL databases, compress them, and upload them to an S3 bucket in AWS, I’ll guide you step by step. Prerequisites","archived":false,"fork":false,"pushed_at":"2024-12-25T12:06:57.000Z","size":6,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-15T17:57:38.138Z","etag":null,"topics":["aws-ec2","aws-s3","bash-scripting","git","github","linux","postgresql","s3-bucket"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Tks-Devops.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-23T20:36:50.000Z","updated_at":"2024-12-25T12:07:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"8ce6e2b9-ce56-44b5-a158-24cbeeffdbf8","html_url":"https://github.com/Tks-Devops/postgrass-buckup-automation","commit_stats":null,"previous_names":["tks-devops/postgrass-buckup-automation"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Tks-Devops/postgrass-buckup-automation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tks-Devops%2Fpostgrass-buckup-automation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tks-Devops%2Fpostgrass-buckup-automation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tks-Devops%2Fpostgrass-buckup-automation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tks-Devops%2Fpostgrass-buckup-automation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Tks-Devops","download_url":"https://codeload.github.com/Tks-Devops/postgrass-buckup-automation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tks-Devops%2Fpostgrass-buckup-automation/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29157421,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-06T07:18:23.844Z","status":"ssl_error","status_checked_at":"2026-02-06T07:13:32.659Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-ec2","aws-s3","bash-scripting","git","github","linux","postgresql","s3-bucket"],"created_at":"2025-01-05T02:14:56.243Z","updated_at":"2026-02-06T09:33:43.698Z","avatar_url":"https://github.com/Tks-Devops.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"Markdown\n\n# PostgreSQL Backup to AWS S3\n\n## Step 1: Install AWS CLI\n\nInstall AWS CLI on your server:\n\n### For Linux (Ubuntu/Debian):\n```bash\nsudo apt update\nsudo apt install awscli\n\nFor macOS (using Homebrew):\n\nbrew install awscli\n\nFor Windows, you can download and install from AWS CLI Download.\n\nConfigure AWS CLI with your IAM credentials:\n\nRun the following command and enter your AWS Access Key and Secret Key when prompted.\n\naws configure\n\nYou'll need to provide:\n\n    AWS Access Key ID\n    AWS Secret Access Key\n    Default region name (e.g., us-east-1)\n    Default output format (e.g., json)\n\nStep 2: Create an S3 Bucket\n\n    Log in to your AWS Management Console.\n    Go to the S3 service.\n    Click Create bucket.\n    Provide a unique bucket name (e.g., my-postgresql-backups).\n    Choose a region and click Create.\n\nStep 3: Set Permissions for the S3 Bucket\n\nEnsure your IAM user has the following permissions:\n\n    s3:PutObject\n    s3:ListBucket\n\nIf you’re using the AWS Free Tier, the permissions will be included by default, but make sure your IAM policy looks something like this:\nJSON\n\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"s3:PutObject\",\n        \"s3:ListBucket\"\n      ],\n      \"Resource\": [\n        \"arn:aws:s3:::my-postgresql-backups\",\n        \"arn:aws:s3:::my-postgresql-backups/*\"\n      ]\n    }\n  ]\n}\n\nStep 4: Write the Bash Script\n\nHere’s a bash script that will:\n\n    Loop through all PostgreSQL databases.\n    Backup and compress each database.\n    Upload each backup to the S3 bucket.\n\nBash Script: backup_postgresql.sh\nbash\n\n#!/bin/bash\n\n# Define the PostgreSQL user and password\nPGUSER=\"your_pg_user\"\nPGPASSWORD=\"your_pg_password\"\n\n# Define the S3 bucket name\nS3_BUCKET=\"s3://your-s3-bucket-name\"\n\n# Define the backup directory\nBACKUP_DIR=\"/path/to/backup/directory\"\n\n# Set PostgreSQL environment variables\nexport PGPASSWORD=$PGPASSWORD\n\n# Create backup directory if it doesn't exist\nmkdir -p $BACKUP_DIR\n\n# Get the list of all databases\ndatabases=$(psql -U $PGUSER -d postgres -t -c \"SELECT datname FROM pg_database WHERE datistemplate = false;\")\n\n# Loop through all databases\nfor db in $databases; do\n  echo \"Backing up database: $db\"\n\n  # Define backup file name\n  BACKUP_FILE=\"$BACKUP_DIR/$db-$(date +%Y-%m-%d).sql.gz\"\n\n  # Backup the database and compress it\n  pg_dump -U $PGUSER -d $db | gzip \u003e $BACKUP_FILE\n\n  # Upload the backup to S3\n  aws s3 cp $BACKUP_FILE $S3_BUCKET/\n\n  # Optionally, remove the local backup file after upload\n  rm $BACKUP_FILE\n\n  echo \"Backup for $db completed and uploaded to S3.\"\ndone\n\n# Unset the PostgreSQL password\nunset PGPASSWORD\n\n3 vulnerabilities detected\n\nExplanation of the Script:\n\n    PGUSER and PGPASSWORD: Replace these with your PostgreSQL user credentials.\n    S3_BUCKET: Replace this with your actual S3 bucket name.\n    BACKUP_DIR: This is the local directory where the backups will be stored before uploading to S3.\n    pg_dump: This command backs up each PostgreSQL database and compresses it using gzip.\n    aws s3 cp: This command uploads the compressed backup to your S3 bucket.\n    Cleanup: After uploading, the backup file is deleted from the local directory.\n\nStep 5: Make the Script Executable\n\nSave the script as backup_postgresql.sh.\nMake it executable:\n\nchmod +x backup_postgresql.sh\n\nStep 6: Schedule the Script with Cron (Optional)\n\nIf you want to automate this backup process, you can schedule it to run at a specific time using cron.\n\nOpen the cron configuration file:\n\ncrontab -e\n\nAdd a cron job to run the backup script every day at 2 AM:\n\n0 2 * * * /path/to/backup_postgresql.sh\n\nThis will run the script daily at 2 AM. Adjust the timing as needed.\nStep 7: Test the Backup Script\n\nRun the script manually to ensure everything works correctly:\n\n./backup_postgresql.sh\n\nCheck your S3 bucket to verify that the backups are uploaded.\nStep 8: Monitor and Maintain\n\n    Logs: You can modify the script to log the output to a file for troubleshooting.\n    Backup Retention: You may want to add logic to delete old backups from S3 after a certain period or only keep the latest backups.\n\nConclusion\n\nThis guide walks you through setting up a PostgreSQL backup system using a bash script, compressing the backups, and uploading them to an S3 bucket. You can automate the process with cron for regular backups. Let me know if you need further clarification or assistance!\nInstalling PostgreSQL on Ubuntu\n\nHere’s a step-by-step guide to installing PostgreSQL on an Ubuntu instance:\nStep 1: Update the Package Index\n\nBefore installing PostgreSQL, update the package index to ensure you have the latest package information.\n\nsudo apt update\n\nStep 2: Install PostgreSQL\n\nUse the apt package manager to install PostgreSQL and its associated packages.\n\nsudo apt install postgresql postgresql-contrib -y\n\n    postgresql: The core PostgreSQL database server.\n    postgresql-contrib: Additional features and extensions.\n\nBy following this structure, you can create a well-formatted README.md file that includes all the necessary steps and explanations.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftks-devops%2Fpostgrass-buckup-automation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftks-devops%2Fpostgrass-buckup-automation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftks-devops%2Fpostgrass-buckup-automation/lists"}