Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/sikalabs/tergum

Tergum is simple tool provides centralized backup solution with multiple sources (databases, files, S3, ...) and multiple backup storages (S3, filesystem, ...)
https://github.com/sikalabs/tergum

backup backup-database backup-tool mysql restore restore-database s3

Last synced: 5 months ago
JSON representation

Tergum is simple tool provides centralized backup solution with multiple sources (databases, files, S3, ...) and multiple backup storages (S3, filesystem, ...)

Awesome Lists containing this project

README 

          


  Tergum

  
Tergum: Universal Backup Tool


  


    SikaLabs

    SikaLabs

    SikaLabs

  



## Why Tergum?



Tergum is simple tool provides centralized backup solution with multiple sources (databases, files, S3, ...) and multiple backup storages (S3, filesystem, ...). Tergum has native backup monitoring and alerts you when backup fails. Tergum also support backup encryption, compression and automatic recovery testing.



__Tergum is under active development, not all features are already implemented. Check [current project state](#current-project-state)__



## Do you want to start using Tergum? Give us a call



Let's discuss Tergum in your project in [30 min call](https://calendly.com/ondrejsika/tergum-intro)



## What "Tergum" means?



Tergum means backup in latin.



## Tergum Cloud: Bring Your Backups into Cloud



Tergum Cloud allow you to manage your backup using UI & Terraform and store your backups securely in our AWS.



Are you interested in our public beta? Drop us email [hi@sikalabs.com](mailto:hi@sikalabs.com?subject=Tergum_Cloud)



## Tergum Enterprise: Use Tergum Cloud in Your Private Infrastructure



Tergum Enterprise brings our cloud platform behind your filewall. For an inquiry, contact our sales [sales@sikalabs.com](mailto:sales@sikalabs.com?subject=Tergum_Enterprise)



## Install



Install using Brew:



```

brew install sikalabs/tap/tergum

```



On Linux (amd64):



```

curl -fsSL https://raw.githubusercontent.com/sikalabs/tergum/master/install.sh | sudo sh

```



Using [scoop](https://scoop.sh/) on Windows:



```

scoop install https://raw.githubusercontent.com/sikalabs/scoop-bucket/master/tergum.json

```



### Autocomplete



See: `tergum completion`



#### Bash



```

source <(tergum completion bash)

```



## CLI Usage



### Generated CLI Docs on Github



See: 



## Generate CLI Docs



Generate Markdown CLI docs to `./cobra-docs`



```

tergum generate-docs

```



## Tergum Config File



Tergum supports only JSON config file, but we're working on YAML support.



Config file examples are in [misc/example/config](./misc/example/config) directory



#### Basic Config Structure



```yaml

Meta:

  SchemaVersion: 3

Settings: 

Cloud: 

Notification: 

Telemetry: 

Backups:

  - 

  - 

  - ...

```



#### Backup Block



```yaml

ID: 

Source:

  Mysql: 

  MysqlServer: 

  Postgres: 

  PostgresServer: 

  Mongo: 

  SingleFile: 

  Dir: 

  KubernetesTLSSecret: 

  Kubernetes: 

  Notion: 

  FTP: 

  Redis: 

  Vault: 

  Dummy: 

  Gitlab: 

  Consul: 

Middlewares:

  - 

  - ...

Destinations:

  - ID: 

    Middlewares:

      - 

      - ...

    FilePath: 

    File: 

    S3: 

    AzureBlob: 

    Telegram: 

  - ...

SleepBefore: 

```



#### GzipMiddlewareConfiguration



```yaml

Gzip: {}

SymmetricEncryption:

  Passphrase: "passphrase"

```



#### Example BackupSourceMysqlConfiguration Block



```yaml

Host: "127.0.0.1"

Port: "3306"

User: "root"

Password: "root"

Database: "default"

```



With extra args



```yaml

Host: "127.0.0.1"

Port: "3306"

User: "root"

Password: "root"

Database: "default"

MysqldumpExtraArgs:

  - --column-statistics=0

```



#### Example BackupSourceMysqlServerConfiguration Block



```yaml

Host: "127.0.0.1"

Port: "3306"

User: "root"

Password: "root"

```



With extra args



```yaml

Host: "127.0.0.1"

Port: "3306"

User: "root"

Password: "root"

MysqldumpExtraArgs:

  - --column-statistics=0

```



#### Example BackupSourcePostgresConfiguration Block



```yaml

Host: "127.0.0.1"

Port: "15432"

User: "postgres"

Password: "pg"

Database: "postgres"

```



With extra args



```yaml

Host: "127.0.0.1"

Port: "15432"

User: "postgres"

Password: "pg"

Database: "postgres"

PgdumpExtraArgs:

  - --ignore-version

```



With SSL mode



```yaml

Host: "127.0.0.1"

Port: "15432"

User: "postgres"

Password: "pg"

Database: "postgres"

SSLMode: "require"

```



#### Example BackupSourcePostgresServerConfiguration Block



```yaml

Host: "127.0.0.1"

Port: "15432"

User: "postgres"

Password: "pg"

```



With extra args



```yaml

Host: "127.0.0.1"

Port: "15432"

User: "postgres"

Password: "pg"

PgdumpallExtraArgs:

  - --ignore-version

```



With SSL mode



```yaml

Host: "127.0.0.1"

Port: "15432"

User: "postgres"

Password: "pg"

SSLMode: "require"

```



#### Example BackupSourceMongoConfiguration Block



Dump all dbs & no auth



```yaml

Host: "127.0.0.1"

Port: "27017"

```



Dump all dbs with auth



```yaml

Host: "127.0.0.1"

Port: "27017"

User: "root"

Password: "root"

```



Dump single db with auth



```yaml

Host: "127.0.0.1"

Port: "27017"

User: "root"

Password: "root"

Database: "test"

```



Dump single db with auth and custom Authentication Database



```yaml

Host: "127.0.0.1"

Port: "27017"

User: "root"

Password: "root"

AuthenticationDatabase: "test" # default is admin

Database: "test"

```



#### Example BackupSourceKubernetesTLSSecret Block



Backup all TLS secrets



```yaml

Server: https://kubernetes-api.example.com

Token: foo-bar-baz

Namespace: default

```



Backup single TLS secret



```yaml

Server: https://kubernetes-api.example.com

Token: foo-bar-baz

Namespace: default

SecretName: tls-example-com

```



#### Example BackupSourceKubernetes Block



Backup all resources (pods)



```yaml

Server: https://kubernetes-api.example.com

Token: foo-bar-baz

Namespace: default

Resource: pod

```



Backup single resource (hello-world pod)



```yaml

Server: https://kubernetes-api.example.com

Token: foo-bar-baz

Namespace: default

Resource: pod

Name: hello-world

```



#### Example BackupSourceSingleFileConfiguration Block



```yaml

Path: /data/export/dump.sql

```



### Example BackupSourceDirConfiguration Block



```yaml

Path: /data

Excludes:

  - /data/tmp

```



### Example BackupSourceNotion Block



```yaml

Token: 

SpaceID: 

Format: 

```



### Example BackupSourceFTP Block



```yaml

Host: 

User: 

Password: 

```



### Example BackupSourceRedis Block



```yaml

Host: 

Port: 

```



### Example BackupSourceVault Block



```yaml

Addr: 

Token: 

Headers: 

```



example with cloudflare access headers



```yaml

Addr: https://vault.corp.com

Token: s.1234567890

Headers:

  CF-Access-Client-ID: xxx1234567890

  CF-Access-Client-Secret: xxx123456789

```



### Example BackupSourceDummy Block



```yaml

Content: 

```



### Example BackupSourceGitlab Block



```yaml

NamePrefix: 

Skip: 

```



- Gitlab Docs about SKIP - 



### Example BackupSourceConsul Block



```yaml

Addr: 

Token: 

```



Example without ACL



```yaml

Addr: http://127.0.0.1:8500

```



Example with ACL requires token



```yaml

Addr: http://127.0.0.1:8500

Token: 51047cd1-c243-a969-2bf1-a845405e4da9

```



#### Example BackupDestinationFilePathConfiguration Block



```yaml

Path: "/backup/mysql-default.sql"

```



#### Example BackupDestinationFileConfiguration Block



```yaml

Dir: "/backup/"

Prefix: "mysql-default"

Suffix: "sql"

```



#### Example BackupDestinationS3Configuration Block



AWS:



```yaml

AccessKey: "admin"

SecretKey: "asdfasdf"

Endpoint: "https://minio.example.com"

BucketName: "tergum-backups"

Prefix: "mysql-default"

Suffix: "sql"

```



Minio:



```yaml

accessKey: "aws_access_key_id"

secretKey: "aws_secret_access_key"

region: "eu-central-1"

bucketName: "tergum-backups"

prefix: "mysql-default"

suffix: "sql"

```



Minio with 3 retries:



You can set `UploadRetries` (default is 0) to retry upload in case of error.



```yaml

accessKey: "aws_access_key_id"

secretKey: "aws_secret_access_key"

region: "eu-central-1"

bucketName: "tergum-backups"

prefix: "mysql-default"

suffix: "sql"

UploadRetries: 3

```



#### Example BackupDestinationAzureBlobConfiguration Block



```yaml

AccountName: account_name

AccountKey: account_key

ContainerName: container_name

Prefix: "mysql-default"

Suffix: "sql"

```



#### Example BackupDestinationTelegramConfiguration Block



```yaml

BotToken: "123456789:ABC-DEF1234ghIkl-zyx57W2v1u123ew11"

ChatID: -123456789

FileName: "backup.sql"

```



[example](./examples/config/telegram)



#### Notification Block



```yaml

Backends: {

  Email:  

Target:

  - 

  - 

  - ...

```



#### Example NotificationBackendEmail Block



```yaml

SmtpHost: "mail.example.com"

SmtpPort: "25"

Usename: "aaa"

Password: "aaa/bbb"

From: "tergum@example.com"

```



#### NotificationTarget Block



```yaml

Email: 

SlackWebhook: 

Telegram: 

```



#### Example NotificationEmailTarget Block



```yaml

Emails:

  - ondrej@example.com

  - monitoring@example.com

SendOK: false

```



- `SendOK=true` will send email notification for all tergum runs (failed & OK runs)



#### Example NotificationSlackWebhookTarget Block



```yaml

URLs:

  - https://hooks.slack.com/services/xxx/yyy/zzz

SendOK: false

```



- `SendOK=true` will send email notification for all tergum runs (failed & OK runs)



#### Example NotificationTelegramTarget Block



```yaml

BotToken: "123456789:ABC-DEF1234ghIkl-zyx57W2v1u123ew11"

ChatIDs: -123456789

SendOK: false

```



[example](./examples/config/telegram)



- `SendOK=true` will send email notification for all tergum runs (failed & OK runs)



#### Cloud Block



```yaml

Email: 

```



### Settings Block



- `UseDoBackupV2` - use new backup processor (default is false)

- `ExtraName` - extra name for backup file (for example "my-backup") - default is empty



```yaml

Settings:

  UseDoBackupV2: true

  ExtraName: "my-backup"

```



### Telemetry Block



- `Origin` - origin of custom telemetry api

- `Disable` - disable telemetry

- `Name` - name of the instance in telemetry

- `CollectHostData` - collect host data (hostname, os, arch, cpu, memory)

- `CollectBackupLog` - collect backup log (backup log is sent to telemetry api)



```yaml

Telemetry:

  Origin: "tergum-telemetry-api.corp.com"

  Disable: true

  Name: "my-tergum"

  CollectHostData: true

  CollectBackupLog: true

```



### Tergum Utils



#### `tergum utils cron`



Simple cron scheduler in Tergum



```

tergum utils cron   [args...]

```



Example usage:



```

tergum utils cron "0 0 * * *" -- tergum backup -c tergum.yaml

```



## Current Project State



### Backup Sources



- [x] SingleFile

- [x] Files (Dir)

- [x] Postgres

- [x] PostgresServer

- [x] MySQL

- [x] MySQLServer

- [ ] Oracle (Enterprise)

- [ ] S3

- [ ] Ceph RBD

- [ ] CephFS

- [x] MongoDB

- [x] Gitlab

- [ ] Proxmox

- [x] Kubernetes Resource

  - [x] Kubernetes TLS Secret

- [ ] Container Image

- [x] Redis

- [x] [Notion](https://notion.so)

- [x] FTP Server (for old school hostings)

- [x] Hashicorp Vault

- [x] Hashicorp Consul

- [x] Dummy (for testing)



### Passwords Sources



- [x] YAML

- [x] Environment Variables

- [ ] Hashicorp Vault

- [ ] AWS Secrets Manager

- [ ] Azure Key Vault



### Backup Processors



- [x] GZIP Compression

- [x] Symmetric Encryption

- [ ] AsymmetricEncryption

- [ ] GPG Encryption

- [ ] GPG Signatures



### Backup Storage



- [x] Files

- [x] S3

- [ ] Tergum Cloud

- [x] Azure Blob

- [ ] GCS (Google Cloud Storage)

- [ ] Container Registry

- [x] Telegram



### Notification



- [x] Email

- [x] Slack

- [x] Telegram

- [ ] Microsoft Teams

- [ ] Pagerduty