{"id":13587426,"url":"https://github.com/luispabon/borg-s3-home-backup","last_synced_at":"2026-02-05T16:06:22.732Z","repository":{"id":100775657,"uuid":"102870976","full_name":"luispabon/borg-s3-home-backup","owner":"luispabon","description":"Simple script to back up your home folder into s3 using borg backup","archived":false,"fork":false,"pushed_at":"2023-02-02T17:09:45.000Z","size":31,"stargazers_count":131,"open_issues_count":0,"forks_count":26,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-02-13T21:49:28.097Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/luispabon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2017-09-08T14:36:49.000Z","updated_at":"2024-02-13T21:49:30.861Z","dependencies_parsed_at":"2023-06-09T20:45:21.100Z","dependency_job_id":null,"html_url":"https://github.com/luispabon/borg-s3-home-backup","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luispabon%2Fborg-s3-home-backup","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luispabon%2Fborg-s3-home-backup/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luispabon%2Fborg-s3-home-backup/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luispabon%2Fborg-s3-home-backup/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/luispabon","download_url":"https://codeload.github.com/luispabon/borg-s3-home-backup/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247732932,"owners_count":20986956,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:06:12.404Z","updated_at":"2026-02-05T16:06:22.688Z","avatar_url":"https://github.com/luispabon.png","language":"Shell","funding_links":[],"categories":["Shell"],"sub_categories":[],"readme":"# Borg s3/gcloud storage home backup\n\n## What\n\nThis is a script to backup your home folder using borg into a local borg repository which is synced\ninto an AWS S3 or Google Cloud Storage bucket.\n\n## Introduction\n\nHaving enough of overcomplicated or crappy commercial backup tools for Linux, I wrote this script to\nback up my home folder into s3 on a cron schedule using borg backup:\n\n  * Borg is quick, saves a great deal of space compared to traditional incremental backup solutions\n  * s3 and gcloud storage are both cheap compared to commercial backup solutions, and also reliable\n  * And cron is always easy to get going on linux\n\nThis script will backup your home folder using borg. The backup will be both compressed and encrypted.\nSince borg does not natively support these cloud storage solutions, and the fuse/mount solutions are\nslow as heck. We'll be using the rsync-like capabilities of the cli tools for each cloud provider,\nwhich actually work pretty well.\n\nThis also prunes your backups to keep 7 dailies and 4 end-of-weeks.\n\nThe side effect however is you'll have two backups, one wherever borg has its backup repo, and another\non the bucket. This is good though, always a good idea to keep several backups even if one is on the same\ncomputer.\n\n## Compatibility\n\n  * Linux for sure.\n  * MacOS likely.\n  * Linux on Windows 10, long shot.\n\n## Requirements\n\n### Tools\n\n  * [Borg backup.](https://www.borgbackup.org/) - your distro will probably have this on its repos\n  * The cloud provider's cli tool: either `awscli` or `gsutil` (part of `google-cloud-sdk`). Same as above.\n  * A bucket\n\n### Environment variables\n\nThese are borg-standard, as per [borg's documentation](https://borgbackup.readthedocs.io/en/stable/usage.html#environment-variables):\n\n  * **BORG_REPO** (mandatory): location where all your backups will go into as they're made (NOT s3).\n  Whereas borg supports ssh paths here as well as any mounted folder (eg s3 via fuse), I would recommend\n  this to be a local folder on a real drive, if you can afford the space, for speed.\n  * **BORG_PASSPHRASE** (optional, recommended): borg will encrypt your backups using this passphrase. You should.\n  Make sure you keep this somewhere safe, other than your backup as you'll also need it to restore your files.\n\nThese are required by the script to function:\n\n  * **BORG_S3_BACKUP_GOOGLE**: set to `true` if you want to use gcloud storage instead of AWS\n  * **BORG_S3_BACKUP_BUCKET** (for both cloud providers): put in here the bucket name only. The naming with the `s3`\n        word is a backwards compatibility measure from when this script supported AWS s3 only.\n  * **BORG_S3_BACKUP_AWS_PROFILE** (s3 only): put in here the aws cli profile that has access to that bucket (eg `default`).\n\n## How to use\n\n  * Git clone this repo somewhere in your computer.\n  * Copy [excludes.txt.dist](excludes.txt.dist) into `excludes.txt` and check its contents. The file is\n  mandatory, but it can be empty. Have a look at\n  [borg's documentation on the subject of excludes](https://borgbackup.readthedocs.io/en/stable/usage/help.html#borg-help-patterns) for more info.\n  [excludes.txt.dist](excludes.txt.dist) has a few examples, indeed these are for my specific use case. If not a regular expression, paths must be\n  absolute.\n  * Install borg backup according to your platform. Possibly already on your distro's software repositories.\n  * For S3:\n      * Install `awscli`. Possibly on your distro's repos already.\n      * Generate programmatic credentials\n      * Configure aws cli with these credentials, eg `aws configure`.\n  * For Gcloud Storage:\n      * [Install gsutil](https://cloud.google.com/storage/docs/gsutil_install). Comes with the `google-cloud-sdk`.\n      * [Authenticate with the `gcloud` tool](https://cloud.google.com/storage/docs/authentication)\n  * Make yourself a bucket in your cloud provider.\n  * Give full access to the bucket the account you're authenticating the cli tool with.\n  * Set up the environment variables discussed above (eg on your `~/.profile`). I do recommend you also set `BORG_PASSPHRASE`.\n  * Create a borg repo. Eg: `borg init -e repokey-blake2` - see [borg quickstart guide](https://borgbackup.readthedocs.io/en/stable/quickstart.html) and [borg init docs](https://borgbackup.readthedocs.io/en/stable/usage/init.html) for more info\n  * Run [borg-backup.sh](borg-backup.sh)!\n\n### Cron\n\nBy default, cron won't load any environment as it runs - make sure you source the file holding your environment\nvariables before running the script!.\n\nExample crontab (assuming you added your environment variables into `~/.profile` that runs the backup every\nworking weekday at 17:30, on low priority, piping the output into a log file in your home folder overwritting any\nprevious logs:\n\n```cron\n30 17 * * MON-FRI . $HOME/.profile \u0026\u0026 nice -n19 $HOME/Projects/borg-s3-home-backup/borg-backup.sh \u003e $HOME/backup.log 2\u003e\u00261\n```\n\n#### Failed backups\n\nSometimes borg will report a failed backup. In my experience, this is typically due to files owned by another user.\nI for instance work a lot with docker and sometimes files owned by root are created in my home folder. You can\nset up a crontab for your root user to fix this:\n\n```shell script\n~ sudo crontab -e\n```\n\nthen add\n\n```cron\n28 17 * * MON-FRI nice -n19 /usr/bin/find /home/YOURUSERNAME \\! -user YOURUSERNAME -print | /usr/bin/xargs /usr/bin/chown YOURUSERNAME:YOURUSERNAME -Rf\n```\n\nQuadruple-check the folder `find` is searching on. You don't want to be running this in `/` by accident.\n\nMake sure this cronjob is run (and has time to finish) before the cronjob that backs up your files.\n\n### Note: borg backup locking\n\nWe lock the borg backup repository during bucket sync to ensure it doesn't change during uploads. Borg achieves locks using lock files within the repo,\ntherefore these files will also be backed up to the bucket during sync. If you ever need to download your backup from the bucket it will thus be locked and will need\nunlocking with `borg break-lock`, after you've restored it to the correct repo location.\n\n## Restoring backups\n\nWe provide with a script to download your backup from the cloud provider. It has the same requirements for env vars as the\nbackup script:\n\n```shell script\n~ ./download-backup.sh /path/to/folder\n```\n\nThen move that folder to the expected location of your borg repo.\n\nIf your borg repo is not present on your computer, you can of course simply:\n\n```shell script\n~ ./download-backup.sh $BORG_REPO\n```\n\nAfter the backup is present at `$BORG_REPO`:\n  * CD into the folder and run `borg break-lock` to unlock your backup repo. See [note above](#note-borg-backup-locking)\n        for explanation on why your backup is locked.\n  * `borg list` will show you available backups\n  * CD into some folder then `borg extract ::backup-name`. This will extract your backup to the current folder. See\n        [borg's documentation](https://borgbackup.readthedocs.io/en/stable/usage.html#borg-extract) for more info.\n  * Move extracted files where they're meant to be.\n\nExample for a typical desktop computer - total restore of your home folder:\n  * Make an administrative user whichever way you'd like, make sure they can `sudo` (for instance, on ubuntu\n  they must be on the `adm` group). You'll be using this user to restore your data. Do this even if your user\n  is already sudo-able to avoid issues when running the commands below.\n  * Log out of your desktop back to the log in screen.\n  * `CTRL+ALT+F1` to switch to tty1.\n  * Log in as said user.\n\nThen:\n\n```bash\n# Move your current home folder out of the way\n~ cd /\n/ ~ mv $HOME $HOME-old\n\n# Make yourself a new one belonging to you\n/ ~ sudo mkdir $HOME\nsudo chown $(whoami):$(getent group $(whoami) | awk 'BEGIN { FS = \":\" } ; { print $1 }') $HOME\n\n# Work out available backups and extract!\n/ ~ borg list\nhome-2022-10-05T17.30          Weds, 2022-10-05 17:30:36 [1234567890abcdef1234567890abcdef]\nhome-2022-10-06T17.30          Thu, 2022-10-06 17:30:36  [1234567890abcdef1234567890abcdef]\n\n/ ~ borg extract ::home-2022-10-05T17.30\n\n[... check your restored home folder ...]\n/ ~ reboot\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluispabon%2Fborg-s3-home-backup","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fluispabon%2Fborg-s3-home-backup","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluispabon%2Fborg-s3-home-backup/lists"}