https://github.com/knighthat/zstd-backup
Cross-platform python script that compresses your files/folder using Zstandard algorithm
https://github.com/knighthat/zstd-backup
backup-script linux macos windows zstandard
Last synced: 5 months ago
JSON representation
Cross-platform python script that compresses your files/folder using Zstandard algorithm
- Host: GitHub
- URL: https://github.com/knighthat/zstd-backup
- Owner: knighthat
- License: mit
- Created: 2023-06-01T17:05:03.000Z (about 3 years ago)
- Default Branch: dev
- Last Pushed: 2024-07-08T20:24:26.000Z (almost 2 years ago)
- Last Synced: 2024-07-09T01:21:19.046Z (almost 2 years ago)
- Topics: backup-script, linux, macos, windows, zstandard
- Language: Python
- Homepage:
- Size: 200 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
ZSTD BACKUP
Backup your files and folders using ZSTD compression algorithm
# Why Zstandard (ZSTD)
Zstandard (ZSTD) provides the best compression ratio (as far as I know). \
Furthermore, the algorithm also allows multithreading for faster compression while being easy to use
and is compatible with tar command
For more details, visit [facebook/zstd](https://github.com/facebook/zstd)
---
# Table of contents
* [Requirements](#requirements)
1. Python
2. Git
* [Setup](#setup)
1. Clone repo
2. Create virtual environment
3. Install dependencies
4. Edit configuration
5. Execute program
* [config.yml](#configyml)
* [Issues](#issues)
* [License](#license)
---
# Installation
## Requirements
1. Python (3.8 - 3.13)
- Linux: Included in most modern distros
- [Windows](https://www.python.org/downloads/windows/)
- [MacOS](https://www.python.org/downloads/macos/)
2. Git
* Debian-based distros (Debian, Ubuntu, Mint, etc.): `apt install git`
* RedHat-based distros (Red Hat, Fedora, CentOS, etc.): `dnf install git`
* Arch-based distros (Arch Linux, Manjaro): `pacman -S git`
* [Git for Windows](https://git-scm.com/download/win)
* [Git for macOS](https://git-scm.com/download/mac)
## Setup
1. Clone this repository
```shell
git clone --depth 1 https://github.com/knighthat/zstd-backup && cd zstd-backup
```
2. Create virtual environment (recommended, or use global python)
* **Windows**
> Run this in your command prompt or powershell
```shell
python -m venv venv
```
* **Linux**
> Some distros require you to install `venv` module before executing this command
```shell
/usr/bin/python3 -m venv venv
```
* MacOS
```shell
python -m venv venv
```
3. Install required packages
```shell
venv/bin/pip install -r requirements.txt
```
4. Edit configuration [config.yml](#configyml)
5. Run program by executing `main.py` script
* Windows
```shell
venv\Scripts\python zstd_backup.py
```
* Others
```shell
venv/bin/python3 main.py
```
# config.yml
> This is where you specify the files/folders that are included in the compressed file.
> Also the place where compressed file saved to.
```yaml
# Available levels: DEBUG, INFO, WARN, ERROR, FATAL
console_log_level: INFO
# All files and folders will be included
# in the final compressed file.
include:
- '/path/to/compress'
# Where to save this backup.
# Must be a path (program creates dirs if path not exist)
destination: "./backups"
# Ignore all files that start with 1 of the paths below.
# Must be in absolute form.
# If '/path/to/ignore' is listed, any sub-folders and files
# within that directory will be ignored.
# Since v0.0.3, you can use regular expression to ignore files.
# For regex to match the entire path, use colons:
# - ':ignore-\d+:'
# 'ignore-1' matches the pattern but not '/path/to/ignore-1'
# For regex to find matches in a string, use semi-colon:
# - ';ignore-\d+;'
# will match both 'ignore-1' and '/path/to/ignore-1'
ignore: []
old_backups:
# How many backups should I keep (default: 5, 0 means unlimited)
keep: 5
# If a backup's creation date exceeds this number, it'll be deleted.
# Set this to 0 to skip this process
retention: 30 # Days
# Should this program delete older backups (even if it was
# within retention days and total files less than 'keep')
# to save space for new backup.
# This option always keeps the most recent backup
# unless 'aggressive' is set to True.
remove_old_backups_for_space: true
# If True, delete to the last backup to make some space for new backup
aggressive: false
arguments:
# Compression level.
# Higher values are slower but yield smaller size.
# Default level is 3
# More at: https://python-zstandard.readthedocs.io/en/latest/compressor.html#zstdcompressor
level: 3
# How many cores should the algo use.
# More threads equals faster compression time.
# 0 will use all threads (also the default)
threads: 4
settings:
# Program will attempt to write to compressed
# file this many bytes per cycle.
# Higher number can cause I/O bottleneck
# If you're unsure, leave it at default 1024
write_chunk: 1024 # bytes
# Settings related to progress bar
# This function is still in Beta
progress_bar:
# Enabling progress bar may
# result in slower write.
enabled: true
# Send compressed file to remote server
remote_storage:
enabled: false
# As of version 0.1.0, only SFTP is supported
type: SFTP
server:
host: '127.0.0.1'
port: 22
credentials:
# Available protocols: PASSWORD, DSS, ED25519, RSA, ECDSA
# PASSWORD requires presence of 'password' field,
# others require 'keypath' and 'passphrase'
# Leave 'passphrase' empty if key isn't encrypted with one.
protocol: 'ED25519'
# Automatically switch to public/private key
# once 'private_key' is filled out.
username: username
keypath: ''
passphrase: ''
# Path must contain name you want the file to be
# '{}' will be replaced by the original name.
remote_path: '/remote/location/compressed.zstd'
# File won't be deleted if error occurs during transfer
delete_after_transfer: false
```
# Issues
If you have any question, please open a ticket [HERE](https://github.com/knighthat/linux-server-backup/issues)
# License
This repo is licensed under MIT. Feel free to use, modify, and re-distribute without restrictions.