{"id":25126712,"url":"https://github.com/sstark/snaprd","last_synced_at":"2025-07-01T16:05:37.047Z","repository":{"id":57575287,"uuid":"151544929","full_name":"sstark/snaprd","owner":"sstark","description":"snapshot backups using rsync","archived":false,"fork":false,"pushed_at":"2021-03-27T11:32:14.000Z","size":213,"stargazers_count":13,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-23T13:08:55.330Z","etag":null,"topics":["backup","cloudless","rsync","snapshot"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"isc","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sstark.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-10-04T09:02:43.000Z","updated_at":"2024-11-15T13:48:49.000Z","dependencies_parsed_at":"2022-09-26T19:01:53.760Z","dependency_job_id":null,"html_url":"https://github.com/sstark/snaprd","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sstark%2Fsnaprd","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sstark%2Fsnaprd/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sstark%2Fsnaprd/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sstark%2Fsnaprd/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sstark","download_url":"https://codeload.github.com/sstark/snaprd/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250439288,"owners_count":21430823,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","cloudless","rsync","snapshot"],"created_at":"2025-02-08T10:34:02.888Z","updated_at":"2025-04-23T13:09:07.915Z","avatar_url":"https://github.com/sstark.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"![](snaprd_logo_sq.png?raw=true)\n\nsnaprd - backup utility\n=======================\n\n[![Build Status](https://travis-ci.com/sstark/snaprd.svg?branch=master)](https://travis-ci.com/sstark/snaprd)\n\nOverview\n--------\n\nSnaprd is a program that helps you to make backups of directories to another\nfile system or server. You can run it on a server that has enough disk space\nand let it continuously fetch incremental changes from another server. Snaprd\nwill make sure that only as many snapshots are created as match your given\nschedule or free space restrictions.\n\n- Continuous creation of snapshots at certain intervals\n- Pruning (sieving) snapshots based on fixed schedule, make snapshots more\n  scarce the older they get\n- Pruning snapshots based on free disk space requirements\n- Uses rsync to create snapshots\n- Every snapshot is a complete copy, using hard links to save disk space\n- Designed to run silently in the background\n- Repository is designed to be exported via e. g. nfs or smb to enable users to\n  do restores of single files or directories\n- Tested with small and huge (100TB) backup sources.\n\nThe project homepage is https://github.com/sstark/snaprd\n\n\nBuilding\n--------\n\nInstead of building the program yourself you can download the latest\nlinux binary from https://github.com/sstark/snaprd/releases. This is usually\nbuilt using the latest Ubuntu LTS release.\n\nBuild yourself:\n\nInstall go either from https://golang.org/ or from your distribution\nrepository, e. g. \"apt-get install golang\".\n\nDownload the archive, unpack and run `make`. Then copy the binary to a\nconvenient place or run `make install` to place it in /usr/local/bin\n\nOR\n\nRun `go get github.com/sstark/snaprd`. The binary will be in\n`$GOPATH/bin` afterwards.\n\n\nInstalling\n----------\n\nSnaprd does not daemonize, logs are printed to the standard output, also the\nstdout and stderr of the rsync command that is being run. Choose whatever you\nlike for starting it at boot: rc.local, SysVinit, upstart, systemd,\nsupervisord, BSD-init, launchd, daemontools, ...\n\nIn case your repository resides in a separate file system you may want to put\nsome mechanism before startup that makes sure this file system is mounted.\n\nSee below for an example how to run snaprd using systemd.\n\n\nRunning\n-------\n\nsnaprd is supposed to be run on the system where the repository is located\n(\"the backup system\"). You will want to make sure ssh works non-interactively\nfor the connection to the system to backup from.\n\nBasic operation:\n\n```\n\u003e snaprd run -schedule=shortterm -repository=/target/dir -origin=someserver:some/dir -noWait\n2016/09/14 20:32:29 snaprd 1.0 started with pid 50606\n2016/09/14 20:32:29 ### Repository: /target/dir, Origin: someserver:some/dir, Schedule: shortterm\n2016/09/14 20:32:29 run: [/usr/bin/rsync --delete -a --link-dest=/target/dir/.data/1473875491-1473875492-complete someserver:some/dir /target/dir/.data/1473877949-0-incomplete]\n2016/09/14 20:32:29 finished: 1473877949-1473877950-complete\n2016/09/14 20:32:29 wait 9m59.817467794s before next snapshot\n[...]\n```\n\nThe above command will create a hard-linked copy (see the `--link-dest` option\nin *rsync(1)*) via ssh from the directory \"some/dir\" on \"someserver\" every 10\nminutes. The copy will be written into a directory within the `.data`\nsub-directory of the target directory \"/target/dir\" on the local system. The\ndirectory name for the snapshots consists of the start time and end time of the\nrsync run (in unix time) and the state of the snapshot. While rsync is running\nthe name will be `\u003cstart\u003e-0-incomplete`. Only after rsync is done, the\ndirectory will be renamed to `\u003cstart\u003e-\u003cend\u003e-complete`.\n\nAfter each snapshot snaprd will also create user-friendly names as symlinks\ninto the .data dir, so if you export the snapshot directory read-only, users\nshould find a resonably convenient way to find their backups.\n\nNext, snaprd will *prune* the existing snapshots. That means it will check if a\nsnapshot is suitable for being advanced into the next level of the schedule (in\nthis example that means the \"two-hourly\" interval) or if it should stay in the\ncurrent interval. If the current interval is \"full\" already, but no snapshot is\nsuitable for being advanced, snaprd will *obsolete* as many snapshots as needed\nto match the schedule.\n\nMarking a snapshot \"obsolete\" simply means renaming it to\n`\u003cstart\u003e-\u003cend\u003e-obsolete`. From then on it will not show up anymore in normal\nlistings and also not be considered as a target for --link-dest. The default\nfor snaprd is to eventually mark those obsolete snapshots as\n`\u003cstart\u003e-\u003cend\u003e-purging` and delete them from disk. You can tweak this behaviour\nwith the \"-maxKeep\", \"-noPurge\", \"-minGbSpace\" and \"-minPercSpace\" parameters\nfor snaprd.\n\nTo get a full list of options available to the run command, use `snaprd run -h`:\n\n    $ snaprd run -h\n    Usage of run:\n    -maxKeep int\n            how many snapshots to keep in highest (oldest) interval. Use 0 to keep all\n    -minGbSpace int\n            if set, keep at least x GiB of the snapshots filesystem free\n    -minPercSpace float\n            if set, keep at least x% of the snapshots filesystem free\n    -noLogDate\n            if set, does not print date and time in the log output. Useful if output is redirected to syslog\n    -noPurge\n            if set, obsolete snapshots will not be deleted (minimum space requirements will still be honoured)\n    -noWait\n            if set, skip the initial waiting time before the first snapshot\n    -notify string\n            specify an email address to send reports\n    -origin string\n            data source (default \"/tmp/snaprd_test/\")\n    -r string\n            (shorthand for -repository) (default \"/tmp/snaprd_dest\")\n    -repository string\n            where to store snapshots (default \"/tmp/snaprd_dest\")\n    -rsyncOpts value\n            additional options for rsync\n    -rsyncPath string\n            path to rsync binary (default \"/usr/bin/rsync\")\n    -schedFile string\n            path to external schedules (default \"/etc/snaprd.schedules\")\n    -schedule string\n            one of longterm,shortterm (default \"longterm\")\n\n\n```\n\u003e snaprd list -repository /tmp/snaprd_dest\n### Repository: /tmp/snaprd_dest, Origin: /tmp/snaprd_test2, Schedule: shortterm\n### From past, 0/∞\n### From 866h0m0s ago, 0/4\n### From 194h0m0s ago, 0/7\n### From 26h0m0s ago, 2/12\n2016-09-14 Wednesday 12:14:31 (1s, 2h0m0s)\n2016-09-14 Wednesday 12:19:46 (2s, 2h0m0s)\n### From 2h0m0s ago, 5/12\n2016-09-14 Wednesday 19:51:07 (1s, 10m0s)\n2016-09-14 Wednesday 19:51:21 (1s, 10m0s)\n2016-09-14 Wednesday 19:51:26 (1s, 10m0s)\n2016-09-14 Wednesday 19:51:31 (1s, 10m0s)\n2016-09-14 Wednesday 20:32:29 (1s, 10m0s)\n```\n\nThe above list command will output some information about the intervals for the\ngiven schedule and how many snapshots are in them.\n\nObviously the list command needs to know which schedule was used for creating\nthe snapshots, but in the above example you can see that no schedule was given\nat the command line. This works because snaprd writes all settings that were\nused for the last *run* command to the repository as `.snaprd.settings`.\n\n\nE-Mail Notification\n-------------------\n\nIf you add the `-notify` option to the run sub-command you will get an email\nin case of a problem. Use it like this:\n\n```\n\u003e snaprd run -notify root \u003cother options...\u003e\n```\n\nIf snaprd has a severe problem it will stop execution and send an email to the\nspecified address, along with the last few lines of log output.\n\nSending happens through use of the standard mail(1) command, make sure your\nsystem is configured accordingly.\n\n\nSystem Prerequisites\n--------------------\n\nObviously you need a file system where you can store enough data to fit the\ndataset you are backing up. It is not possible to predict how much space will\nbe needed for a given schedule and update pattern of data. You should at least\nmake the snapshot file system such that it can be easily extended if needed.\nStarting with a factor of 1.5 to 2 should be sufficient.\n\nIf you are using mlocate or a similar mechanism to index your files, make sure\nyou exclude your snapshot file system from it, e. g. like this:\n\n\u003cpre\u003e\n$ cat /etc/updatedb.conf\nPRUNE_BIND_MOUNTS=\"yes\"\n# PRUNENAMES=\".git .bzr .hg .svn\"\nPRUNEPATHS=\"/tmp /var/spool /media /var/lib/os-prober /var/lib/ceph /home/.ecryptfs /var/lib/schroot \u003cb\u003e/snapshots\u003c/b\u003e\"\nPRUNEFS=\"NFS nfs nfs4 rpc_pipefs afs binfmt_misc proc smbfs autofs iso9660 ncpfs coda devpts ftpfs devfs devtmpfs fuse.mfs shfs sysfs cifs lustre tmpfs usbfs udf fuse.glusterfs fuse.sshfs curlftpfs ceph fuse.ceph fuse.rozofs ecryptfs fusesmb\"\n\u003c/pre\u003e\n\nIf you do not exclude your snapshots you will get enormously big mlocate.db\nfiles with lots of redundant information.\n\n\n\nStopping\n--------\n\nsnaprd will immediately exit when sent the TERM or INT (ctrl-c) signal. If a\nbackup is running at this time it will be left in incomplete state. (On the next\nrun it will be reused potentially.)\n\nYou can also send the USR1 signal, in which case snaprd will wait until the\ncurrent backup has finished, and exit afterwards.\n\nYou can find the pid of the running process in the repository directory in the\nfile `.pid`.\n\nSignals\n-------\n\nsnaprd responds to various signals.\n\n  - **INT**, **TERM**: Makes snaprd exit immediately, killing a potentially\n    running rsync process using the same signal.\n  - **USR1**: While rsync is running, wait until it finished, then exit.\n    Otherwise just exit.\n  - **USR2**: If snaprd is idle waiting for the next scheduled snapshot,\n    sending SIGUSR2 will cancel this waiting time and force an immediate\n    snapshot.\n\nSchedules\n---------\n\nThere are currently two builtin schedules for snapshots which you can choose\nwith the -schedule switch to the run command:\n\n  - shortterm: 10m 2h 24h 168h 672h\n  - longtterm: 6h 24h 168h 672h\n\nThe duration listed define how long a snapshot stays in that interval until it\nis either promoted to the next higher interval or deleted.\n\nWhich schedule you choose is entirely up to you, just make sure the smallest\n(first) interval is large enough so the expected runtime of a single rsync\nsnapshot fits in it with a good margin.\n\nYou can define your own schedules by editing a json-formatted file\n`/etc/snaprd.schedules` with entries like:\n\n```\n{\n    \"production\" : [ {\"day\":1}, {\"week\":2}, {\"month\":1}, {\"long\":1} ],\n    \"moreoften\" : [ {\"hour\":6}, {\"day\":1}, {\"week\":2}, {\"month\":1}, {\"long\":1} ]\n}\n```\n\nThe above 'production' schedule will tell snaprd to make a snapshot every day,\nkeep one of those every 2 weeks, keep one of those every month. The last entry\n(\"long\") should not be omitted and basically means eternity.\n\nAs many snapshots are kept as \"fit\" into the next interval.\n\nThe 'moreoften' schedule will do almost the same as 'production', but make\nsnapshots every 6 hours, thus keeping 4 snapshots per day.\n\nYou can verify your schedule by running `snaprd scheds`, and later, when\nsnapshots have already been created, by `snaprd list`.\n\n\nExample Unit File for Systemd\n-----------------------------\n\nPlace in `/etc/systemd/system/snaprd-srv-home.service`\n\n    [Unit]\n    Description=snapshots for srv:/homes\n    Documentation=https://github.com/sstark/snaprd\n    Requires=network.target\n\n    [Service]\n    User=root\n    StandardOutput=syslog\n    ExecStart=/usr/local/bin/snaprd run -noLogDate -notify root -repository=/export/srv-home-snap -origin=srv:/export/homes\n    Restart=on-failure\n    # for a shared machine you probably want to set the scheduling class:\n    IOSchedulingClass=idle\n\n    [Install]\n    WantedBy=multi-user.target\n\nEnable with\n\n    sudo systemctl enable snaprd-srv-home \u0026\u0026 sudo systemctl start snaprd-srv-home\n\nCheck logs with \n\n    journalctl -u snaprd-srv-home\n\n\nTesting\n-------\n\nTo run regression testing, run `make test`\n\nDebug output can be enabled by setting the environment variable SNAPRD_DEBUG=1\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsstark%2Fsnaprd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsstark%2Fsnaprd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsstark%2Fsnaprd/lists"}