{"id":19853863,"url":"https://github.com/magnetikonline/rsync-backup-rotation","last_synced_at":"2025-05-02T01:30:23.411Z","repository":{"id":6134205,"uuid":"7362645","full_name":"magnetikonline/rsync-backup-rotation","owner":"magnetikonline","description":"Bash script for automated incremental backups via Rsync.","archived":false,"fork":false,"pushed_at":"2024-02-06T22:22:17.000Z","size":20,"stargazers_count":14,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-06T20:25:06.352Z","etag":null,"topics":["backup","bash","rsync","utility"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/magnetikonline.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-12-29T03:49:17.000Z","updated_at":"2024-04-15T12:16:34.000Z","dependencies_parsed_at":"2022-08-31T15:00:46.563Z","dependency_job_id":null,"html_url":"https://github.com/magnetikonline/rsync-backup-rotation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magnetikonline%2Frsync-backup-rotation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magnetikonline%2Frsync-backup-rotation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magnetikonline%2Frsync-backup-rotation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magnetikonline%2Frsync-backup-rotation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/magnetikonline","download_url":"https://codeload.github.com/magnetikonline/rsync-backup-rotation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251969229,"owners_count":21673180,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","bash","rsync","utility"],"created_at":"2024-11-12T14:07:52.880Z","updated_at":"2025-05-02T01:30:23.140Z","avatar_url":"https://github.com/magnetikonline.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Rsync backup rotation\n\nEnable automated incremental [Rsync](https://rsync.samba.org/) backups with a module `post-xfer exec` script and the `--link-dest` option.\n\nIncremental backups are stored using hard-links between identical files, providing very efficient storage using nothing more than the Linux file system.\n\nBackup runs are numbered in directories from `001 -\u003e REVISION_COUNT`, with script automatically truncating directories beyond `REVISION_COUNT`.\n\n- [Installation](#installation)\n\t- [Target server](#target-server)\n\t\t- [Optional arguments](#optional-arguments)\n\t- [Source server](#source-server)\n\t- [All done](#all-done)\n- [Tests](#tests)\n\n## Installation\n\n### Target server\n\n- Place [`rsyncd-rotation.sh`](rsyncd-rotation.sh) somewhere usable and set executable for the run-as user of _receiving_ `rsyncd` process.\n- Configure target Rsync module(s) to execute `rsyncd-rotation.sh` *after* a successful run via:\n\t- `/etc/rsyncd.conf` if running daemon mode, or...\n\t- `~/rsyncd.conf` if via SSH.\n\nAn example target module config:\n\n```ini\nlist = false\nlog file = /var/log/rsyncd/00default.log\nlog format = %i %f [%l]\nread only = false\ntransfer logging = true\nuse chroot = false\n\n[MODULE_NAME]\nlog file = /var/log/rsyncd/MODULE_NAME.log\npath = /target/path/to/backup\npost-xfer exec = /path/to/rsyncd-rotation.sh\n```\n\nBreaking this down:\n\n- Before validating user/module, everything logged to `/var/log/rsyncd/00default.log`.\n- Increasing the level of transfer logging to include filename and bytes moved via `log format`.\n- Setting `use chroot = false` (usually) required if not running `rsyncd` under root.\n- Validated module logging to `/var/log/rsyncd/MODULE_NAME.log`.\n- Backup location for the target defined in `path = /target/path/to/backup`.\n\t- Incremental backups stored at `/target/path/to/backup/001` to `/target/path/to/backup/REVISION_COUNT`.\n- Finally, stanza `post-xfer exec = /path/to/rsyncd-rotation.sh` instructs `rsyncd` to execute rotation script *after* file transfer is complete.\n\n#### Optional arguments\n\nIn addition `rsyncd-rotation.sh` accepts arguments:\n\n- Logging of key script events to file via the `-l LOG_FILE` option, handy for debugging correct operation.\n- Adjustment of backup retention count from [default](rsyncd-rotation.sh#L3) of `25` via the `-r RETENTION_COUNT` option. Given count must be `2` or greater.\n\nExample use:\n\n```ini\n[MODULE_NAME]\nlog file = /var/log/rsyncd/MODULE_NAME.log\npath = /target/path/to/backup\npost-xfer exec = /path/to/rsyncd-rotation.sh -l /path/to/file.log -r 6\n```\n\n### Source server\n\nTo perform a backup execute `rsync` in a method like below:\n\n```sh\n# to an rsync listening daemon (unencrypted)\n$ rsync \\\n\t--archive \\\n\t--delete \\\n\t--link-dest \"../001\" \\\n\t/backup/from/path TARGET_HOST::MODULE_NAME/000\n\n# or over an encrypted SSH connection\n$ rsync \\\n\t--archive \\\n\t--delete \\\n\t--link-dest \"../001\" \\\n\t--rsh \"ssh -l TARGET_USER\" \\\n\t/backup/from.path TARGET_HOST::MODULE_NAME/000\n```\n\nThe *critical* command line components are:\n\n- Current backup is hard-linked against previous incremental with the `--link-dest \"../001\"` argument where files are identical between source and incremental.\n- Backup is placed into a `/target/path/to/backup/000` directory via the `TARGET_HOST::MODULE_NAME/000` destination set.\n\nOnce Rsync completes, `rsyncd-rotation.sh` is executed to increase all incremental directories by one, dropping any that exceed set `REVISION_COUNT`.\n\n### All done\n\nYou should now have automated, space saving and easy to manage incremental backups running under Rsync. Enjoy!\n\n## Tests\n\nSmall test suite for `rsyncd-rotation.sh` provided by [`test/test.sh`](test/test.sh).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmagnetikonline%2Frsync-backup-rotation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmagnetikonline%2Frsync-backup-rotation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmagnetikonline%2Frsync-backup-rotation/lists"}