{"id":13648159,"url":"https://github.com/ddebeau/zfs_uploader","last_synced_at":"2026-04-10T05:38:47.368Z","repository":{"id":42673956,"uuid":"356651675","full_name":"ddebeau/zfs_uploader","owner":"ddebeau","description":"Simple program for backing up full and incremental ZFS snapshots to Amazon S3.","archived":false,"fork":false,"pushed_at":"2024-01-17T02:27:31.000Z","size":159,"stargazers_count":51,"open_issues_count":5,"forks_count":6,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-03-14T20:04:55.528Z","etag":null,"topics":["backup","s3","zfs"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ddebeau.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-10T17:31:25.000Z","updated_at":"2024-06-14T20:57:51.705Z","dependencies_parsed_at":"2024-01-09T03:25:09.536Z","dependency_job_id":"2086ea2c-57ba-45ab-9538-c651a3c02a0c","html_url":"https://github.com/ddebeau/zfs_uploader","commit_stats":null,"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddebeau%2Fzfs_uploader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddebeau%2Fzfs_uploader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddebeau%2Fzfs_uploader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ddebeau%2Fzfs_uploader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ddebeau","download_url":"https://codeload.github.com/ddebeau/zfs_uploader/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250195033,"owners_count":21390230,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","s3","zfs"],"created_at":"2024-08-02T01:04:01.068Z","updated_at":"2025-12-13T16:16:24.572Z","avatar_url":"https://github.com/ddebeau.png","language":"Python","funding_links":[],"categories":["Python","others"],"sub_categories":[],"readme":"# ZFS Uploader\nZFS Uploader is a simple program for backing up full and incremental ZFS \nsnapshots to Amazon S3. It supports CRON based scheduling and can \nautomatically remove old snapshots and backups. A helpful CLI (`zfsup`) lets \nyou run jobs, restore, and list backups.\n\n### Features\n- Backup/restore ZFS file systems\n- Create incremental and full backups\n- Automatically remove old snapshots and backups\n- Use any S3 storage class type\n- Helpful CLI\n\n### Requirements\n- Python 3.6 or higher\n- ZFS 0.8.1 or higher (untested on earlier versions)\n\n## Install Instructions\nCommands should be run as root.\n\n1. Create a directory and virtual environment\n```bash\nmkdir /etc/zfs_uploader\ncd /etc/zfs_uploader\nvirtualenv --python python3 env\n```\n\n2. Install ZFS Uploader\n```bash\nsource env/bin/activate\npip install zfs_uploader\nln -sf /etc/zfs_uploader/env/bin/zfsup /usr/local/sbin/zfsup\n```\n\n3. Write configuration file\n\nPlease see the [Configuration File](#configuration-file) section below for \nhelpful configuration examples. \n```bash\nvi config.cfg\nchmod 600 config.cfg\n```\n\n4. Start service\n```bash\ncp zfs_uploader.service /etc/systemd/system/zfs_uploader.service\nsudo systemctl enable --now zfs_uploader\n```\n\n5. List backups\n```bash\nzfsup list\n```\n\n## Configuration File\nThe program reads backup job parameters from a configuration file. Default \nparameters may be set which then apply to all backup jobs. Multiple backup \njobs can be set in one file.\n\n### Parameters\n#### bucket_name : str\n   S3 bucket name.\n#### access_key : str\n   S3 access key.\n#### secret_key : str\n   S3 secret key.\n#### filesystem : str\n   ZFS filesystem.\n#### prefix : str, optional\n   Prefix to be prepended to the s3 key.\n#### region : str, default: us-east-1\n   S3 region.\n#### endpoint : str, optional\n   S3 endpoint for alternative services\n#### cron : str, optional\n   Cron schedule. Example: `* 0 * * *`\n#### max_snapshots : int, optional\n   Maximum number of snapshots.\n#### max_backups : int, optional\n   Maximum number of full and incremental backups.\n#### max_incremental_backups_per_full : int, optional\n   Maximum number of incremental backups per full backup.\n#### storage_class : str, default: STANDARD\n   S3 storage class.\n#### max_multipart_parts : int, default: 10000\n   Maximum number of parts to use in a multipart S3 upload.\n\n### Examples\n#### Multiple full backups\n```ini\n[DEFAULT]\nbucket_name = BUCKET_NAME\nregion = us-east-1\naccess_key = ACCESS_KEY\nsecret_key = SECRET_KEY\nstorage_class = STANDARD\n\n[pool/filesystem]\ncron = 0 2 * * *\nmax_snapshots = 7\nmax_incremental_backups_per_full = 6\nmax_backups = 7\n```\n\nFilesystem is backed up at 02:00 daily. Only the most recent 7 snapshots\nare kept. The oldest backup without dependents is removed once there are\nmore than 7 backups.\n\n#### Backblaze B2 S3-compatible endpoint, full backups\n```ini\n[DEFAULT]\nbucket_name = BUCKET_NAME\nregion = eu-central-003\naccess_key = ACCESS_KEY\nsecret_key = SECRET_KEY\nstorage_class = STANDARD\nendpoint = https://s3.eu-central-003.backblazeb2.com\n\n[pool/filesystem]\ncron = 0 2 * * *\nmax_snapshots = 7\nmax_incremental_backups_per_full = 6\nmax_backups = 7\n```\n\n##### Structure\nfull backup (f), incremental backup (i)\n\n1.  f\n2.  f i\n3.  f i i\n4.  f i i i\n5.  f i i i i\n6.  f i i i i i\n7.  f i i i i i i\n8.  f i i i i i f\n9.  f i i i i f i\n10. f i i i f i i\n11. f i i f i i i\n12. f i f i i i i\n13. f f i i i i i\n14. f i i i i i i\n\n#### Single full backup\n```ini\n[DEFAULT]\nbucket_name = BUCKET_NAME\nregion = us-east-1\naccess_key = ACCESS_KEY\nsecret_key = SECRET_KEY\nstorage_class = STANDARD\n\n[pool/filesystem]\ncron = 0 2 * * *\nmax_snapshots = 7\nmax_backups = 7\n```\n\nFilesystem is backed up at 02:00 daily. Only the most recent 7 snapshots\nare kept. The oldest incremental backup is removed once there are\nmore than 7 backups. The full backup is never removed.\n\n##### Structure\nfull backup (f), incremental backup (i)\n\n1.  f\n2.  f i\n3.  f i i\n4.  f i i i\n5.  f i i i i\n6.  f i i i i i\n7.  f i i i i i i\n\n#### Only full backups\n```ini\n[DEFAULT]\nbucket_name = BUCKET_NAME\nregion = us-east-1\naccess_key = ACCESS_KEY\nsecret_key = SECRET_KEY\nstorage_class = STANDARD\n\n[pool/filesystem]\ncron = 0 2 * * *\nmax_snapshots = 7\nmax_incremental_backups_per_full = 0\nmax_backups = 7\n```\n\nFilesystem is backed up at 02:00 daily. Only the most recent 7 snapshots\nare kept. The oldest full backup is removed once there are\nmore than 7 backups. No incremental backups are taken.\n\n##### Structure\nfull backup (f)\n\n1.  f\n2.  f f\n3.  f f f\n4.  f f f f\n5.  f f f f f\n6.  f f f f f f\n7.  f f f f f f f\n\n## Miscellaneous\n### Storage class codes\n- STANDARD\n- REDUCED_REDUNDANCY\n- STANDARD_IA\n- ONEZONE_IA\n- INTELLIGENT_TIERING\n- GLACIER\n- DEEP_ARCHIVE\n- OUTPOSTS\n\n## Release Instructions\n1. Increment version in `__init__.py` file\n\n2. Update `CHANGELOG.md` with new version\n\n3. Tag release in GitHub when ready. Add changelog items to release \n   description. GitHub Action workflow will automatically build and push \n   the release to PyPi.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fddebeau%2Fzfs_uploader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fddebeau%2Fzfs_uploader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fddebeau%2Fzfs_uploader/lists"}