Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/rafsaf/ogion
A tool for performing scheduled database backups and transferring encrypted data to secure clouds, for home labs, hobby projects, etc., in environments such as k8s, docker, vms.
https://github.com/rafsaf/ogion
aws-s3 azure backup backup-tool backup-utility cron docker googlecloudstorage kubernetes mariadb mysql postgresql python
Last synced: about 2 months ago
JSON representation
A tool for performing scheduled database backups and transferring encrypted data to secure clouds, for home labs, hobby projects, etc., in environments such as k8s, docker, vms.
- Host: GitHub
- URL: https://github.com/rafsaf/ogion
- Owner: rafsaf
- License: gpl-3.0
- Created: 2022-07-10T13:35:29.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-11-15T11:20:20.000Z (about 2 months ago)
- Last Synced: 2024-11-15T12:26:54.898Z (about 2 months ago)
- Topics: aws-s3, azure, backup, backup-tool, backup-utility, cron, docker, googlecloudstorage, kubernetes, mariadb, mysql, postgresql, python
- Language: Python
- Homepage: https://ogion.rafsaf.pl
- Size: 11 MB
- Stars: 3
- Watchers: 3
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/rafsaf/ogion/blob/main/LICENSE)
[](https://docs.python.org/3/whatsnew/3.13.html)
[](https://github.com/astral-sh/ruff)
[](https://github.com/rafsaf/ogion/actions/workflows/tests.yml)
[](https://github.com/rafsaf/ogion/actions/workflows/type_check.yml)
[](https://github.com/rafsaf/ogion/actions/workflows/dev_build.yml)
[](https://github.com/rafsaf/ogion/actions/workflows/release_build.yml)
[](https://github.com/rafsaf/ogion/actions/workflows/update_compose_dbs.yml)
# Ogion
A tool for performing scheduled database backups and transferring encrypted data to secure public clouds, for home labs, hobby projects, etc., in environments such as k8s, docker, vms.
Backups are in `age` format using [age](https://github.com/FiloSottile/age), with strong encryption under the hood. Why age? it's modern replacement for GnuPG, available for most architectures and systems.
This project is more or less well tested cron-like runtime with predefined supported providers and backup targets (see below) with sensible defaults for backup commands. It has rich integration tests using providers container replacements: fake gcs, azurite, minio. Goal was to make 100% sure it will work in the wild.
There is **no compression before age encryption** step whatsoever. This is intentional, prepare for large backups size (compared to ogion 6.0 where 7zip was used, some backups that were 300MB now are 2.2GB). There are known exploits when mixing compression with encryption, and for small systems compression this just seems unnecessary. See:
- [CRIME](https://en.wikipedia.org/wiki/CRIME)
- [BREACH](https://en.wikipedia.org/wiki/BREACH)
- [Known plaintext attack](https://en.wikipedia.org/wiki/Known-plaintext_attack)
- [A Known Plaintext Attack on the PKZIP](https://link.springer.com/content/pdf/10.1007/3-540-60590-8_12.pdf)
- [TLSv1.3 removes compression](https://blog.cloudflare.com/tls-1-3-overview-and-q-and-a/)
## Documentation
- [https://ogion.rafsaf.pl](https://ogion.rafsaf.pl)
## Alternatives
There are better tools for big corporate databases and systems:
- [pgBackRest - Reliable PostgreSQL Backup & Restore](https://pgbackrest.org/)
- [postgres operator for k8s based on pgBackRest from crunchydata](https://access.crunchydata.com/documentation/postgres-operator/latest)
## Supported backup targets
- PostgreSQL ([all currently supported versions](https://endoflife.date/postgresql))
- MariaDB ([all currently supported versions](https://endoflife.date/mariadb))
- MySQL ([currently supported versions after 8.0](https://endoflife.date/mysql), version 8.0 does not work with current mariadb-client)
- Single file
- Directory
## Supported upload providers
- Google Cloud Storage bucket
- AWS S3 bucket
- Azure Blob Storage
- Debug (local)
## Notifications
- Discord
- Email (SMTP)
- Slack
## Deployment strategies
Using docker image: `rafsaf/ogion:latest`, see all tags on [dockerhub](https://hub.docker.com/r/rafsaf/ogion/tags)
- docker (docker compose) container
- kubernetes deployment
## Architectures
- linux/amd64
- linux/arm64
## Example
Everyday 5am backup of PostgreSQL database defined in the same file and running in docker container.
```yml
# docker-compose.yml
services:
db:
image: postgres:16
environment:
- POSTGRES_PASSWORD=pwd
ogion:
image: rafsaf/ogion:latest
environment:
- POSTGRESQL_PG16=host=db password=pwd cron_rule=0 0 5 * * port=5432
- AGE_RECIPIENTS=age1q5g88krfjgty48thtctz22h5ja85grufdm0jly3wll6pr9f30qsszmxzm2
- BACKUP_PROVIDER=name=debug
```
(NOTE this will use provider [debug](https://ogion.rafsaf.pl/latest/providers/debug/) that store backups locally in the container).
## Real world usage
The author actively uses ogion (with GCS) for one production project [plemiona-planer.pl](https://plemiona-planer.pl) postgres database (both PRD and STG) and for bunch of homelab projects including self hosted Firefly III mariadb, Grafana postgres, KeyCloak postgres, Nextcloud postgres and configuration file, Minecraft server files, and two other postgres dbs for some demo projects.
See how it looks for ~2GB size database:
