Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/Jehops/zap

Maintain and replicate ZFS snapshots
https://github.com/Jehops/zap

backup replication zfs zfs-snapshots

Last synced: 7 days ago
JSON representation

Maintain and replicate ZFS snapshots

Awesome Lists containing this project

README 

        
** Overview

   zap automates the management of zfs snapshots.  With a few crontab entries, it can be used to create a comprehensive zfs backup system.  There are no configuration files.  Parameters are supplied on the command line or in zfs properties and all snapshot information is stored in snapshot names.



   zap will not interfere with manually created snapshots or snapshots from other tools.  It will only operate on snapshots it creates.



** Synopsis

   =# zap snap|snapshot [-DLSv] TTL [[-r] dataset]...=



   =# zap rep|replicate [-DFLSv] [-h host] [[[user@]host:]parent_dataset [-r] dataset [[-r] dataset]...]=



   =# zap destroy [-Dlsv] [host[,host]...]=



   =# zap -v | -version | --version=



** Examples

   Create snapshots that will expire after 3 weeks.  The =#= prompt indicates commands that are run as root.  A solution that delegates permissions for most of these commands to an unprivileged user is described [[http://ftfl.ca/blog/2016-12-27-zfs-replication.html][here]].

#+BEGIN_SRC sh

  # zfs set zap:snap=on zroot/usr/home/nox zroot/var/

  # zfs set zap:snap=off zroot/var/crash zroot/var/tmp zroot/var/mail

  # zap snap 3w

#+END_SRC

   Create snapshots specifying the datasets on the command line.

#+BEGIN_SRC sh

   # zap snap 3w zroot/usr/home/nox -r zroot/var

#+END_SRC

   Create snapshots that will expire after one day.  Be verbose.

#+BEGIN_SRC sh

   # zfs set zap:snap=on zroot/usr/home

   # zap snap -v 1d

#+END_SRC

   Replicate datasets to the remote host bravo, under the zback/phe dataset.  If you use a non-default ssh port, specify it in =~/.ssh/config=.

#+BEGIN_SRC sh

   # zfs set zap:rep='zap@bravo:zback/phe' zroot/ROOT zroot/usr/home/jrm

   # zap rep -v

#+END_SRC

   Replicate datasets (recursively for zroot/ROOT) to the remote host bravo, under the =rback/phe= dataset, but this time specify the datasets on the command line.  If you use a non-default ssh port, specify it in =~/.ssh/config=.

#+BEGIN_SRC sh

   # zap rep zap@bravo:rback/phe -r zroot/ROOT zroot/usr/home/jrm

#+END_SRC

   Replicate datasets that originated from the host awarnach to the remote host bravo, under the zback/phe dataset.  If you use a non-default ssh port, specify it in =~/.ssh/config=.  Filter the transfer through =~mbuffer~= by setting the =~ZAP_FILTER~= environment variable.  Note that =~mbuffer~= must be installed on both the sending and receiving hosts.

#+BEGIN_SRC sh

   # zfs set zap:rep='zap@bravo:zback/phe' zroot/ROOT zroot/usr/home/jrm

   # ZAP_FILTER="mbuffer -s 128k -m 10M" zap rep -v -h awarnach

#+END_SRC

   Destroy expired snapshots.  Be verbose.

#+BEGIN_SRC sh

   # zap destroy -v

#+END_SRC

   Destroy expired snapshots that originated from either the host awarnach or the host gly.  Be verbose.

#+BEGIN_SRC sh

   # zap destroy -v awarnach,gly

#+END_SRC

     Example crontab entries for rolling snapshots and remote replication.



     - Refer to http://ftfl.ca/blog/2016-12-27-zfs-replication.html for a detailed description of a backup and replication strategy.

     - Snapshots are created for datasets with the =zap:snap= property set to =on=.

     - Datasets with the =zap:rep= property set are replicated.

     - Taking snapshots is normally cheap, so do it often. Destroying snapshots can thrash disks, so only do it every 24 hours.



#+BEGIN_SRC sh

# crontab -e

#+END_SRC



#+BEGIN_SRC sh

#minute	hour	mday	month	wday	command



# take snapshots

*/5	*	*	*	*	zap snap 1d

14	*/4	*	*	*	zap snap 1w

14	00	*	*	1	zap snap 1m



# replicate datasets

54	*/1	*	*	*	zap rep -v



# destroy snapshots

44	04	*	*	*	zap destroy

#+END_SRC

** Subcommands

   =snap= | =snapshot=



   Use the =snap= subcommand to create snapshots that will expire after =TTL= (time to live) has elapsed.  An expired snapshot will be destroyed the next time =zap destroy= is run.  =TTL= takes the form =[0-9]{1,4}[dwmy]=.  That is, one to four digits followed by a character to represent the time unit

(day, week, month, or year). If datasets are not not supplied on the command line, snapshots will be created for datasets with the property =zap:snap= set to =on=.



   =rep= | =replicate=



   Use the =rep= subcommand to replicate datasets.  If a destination and datasets are not supplied on the command line, datasets with a destination set in the =zap:rep= user property are replicated.  If the destination does not contain a =host=, or if the supplied host is one of =localhost=, =127.x.x.x=, or =::1=, then any =user@= is ignored and =ssh= is not be used.  If the =canmount= property of the local dataset is set to =on=, after replication an attempt is made to set =canmount= to =noauto= on the remote side.  This is done to prevent mountpoint collisions.  Set the =~ZAP_FILTER~= environment variable to a command to filter the transfer.  For example, =~ZAP_FILTER="mbuffer -s 128k -m 10M"~= will filter the transfer through =~mbuffer~=.  Note that the filter command must be installed on both the sending and receiving hosts.



   =destroy=



   Use the =destroy= subcommand to destroy expired snapshots.  By default, only snapshots originating from the local host are destroyed.  If a comma separated list of hosts are specified, then only snapshots originating from those hosts are destroyed. Hosts must be specified without any domain information, that is, as returned by =hostname -s=.



** Options



   =-v= | =-version= | =--version= Show the version.



*** Subcommand options



    =-D=  Do not operate on snapshots when the pool is in a DEGRADED state.



    =-F=  Supply =-F= to =zfs receive=, which destroys remote changes that do not exist on the sending side.



    =-L=  Do not operate on snapshots if the pool has a resilver in progress.  This is the default for the =destroy= subcommand.



    =-l=  Operate on snapshots, even if the pool has a resilver in progress.  This is the default for the

=snap= and =rep= subcommands.



    =-S=  Do not operate on snapshots if the pool is being scrubbed.  This is the default for the =destroy= subcommand.



    =-s=  Operate on snapshots, even if the pool is being scrubbed.  This is the default for the =snap=

and =rep= subcommands.



    =-r=  Recursively create or replicate snapshots of all descendants.



    =-v=  Be verbose.



** Author and Contributors

   - Joseph Mingrone 

   - Tobias Kortkamp 

   - David Samms 

   - Victor Naumov 

   - Dries Michiels 

   - HIROSE yuuji 

   - Maxime Soulé

   - sevmonster

** License

   zap is released under a BSD 2-Clause License.  Refer to the header of each

   source file for details.



-----



zap was influenced by zfSnap, which is under a BEER-WARE license.  We owe the author a beer.