Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/AnchorFree/vmbackup-sidecar
Backup sidecar container for VictoriaMetrics
https://github.com/AnchorFree/vmbackup-sidecar
backup docker-image project-monitoring s3 sidecar sidecar-container snapshot sync team-kyiv-devops victoriametrics
Last synced: 2 days ago
JSON representation
Backup sidecar container for VictoriaMetrics
- Host: GitHub
- URL: https://github.com/AnchorFree/vmbackup-sidecar
- Owner: AnchorFree
- License: apache-2.0
- Created: 2019-08-30T15:14:59.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2023-05-08T20:16:28.000Z (over 1 year ago)
- Last Synced: 2024-03-18T11:33:16.537Z (8 months ago)
- Topics: backup, docker-image, project-monitoring, s3, sidecar, sidecar-container, snapshot, sync, team-kyiv-devops, victoriametrics
- Language: Go
- Homepage:
- Size: 45.9 KB
- Stars: 10
- Watchers: 9
- Forks: 2
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/AnchorFree/vmbackup-sidecar/blob/master/LICENSE)
[](https://goreportcard.com/report/github.com/AnchorFree/vmbackup-sidecar)
**Warning:** use this tool _only_ with VictoriaMetrics version < 1.29.0.
Starting from
[v1.29.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.29.0) VictoriaMetrics added
native `vmbackup` and `vmrestore` tools for fast backups on GCS and S3.
vmbackup-sidecar
----------------
Provides a sidecar backup service for [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster) *vmstorage* component.
## How it works
**vmbackup-sidecar** provides `/backup/create` API endpoint. After receiving `GET` request, it creates *VictoriaMetrics* `vmstorage` snapshot via [snapshot API](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/Cluster-VictoriaMetrics#url-format) and syncs it with S3 bucket using [aws s3 sync](https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html). When sync with S3 is completed, all `vmstorage` snapshots are removed to free up space. Depending on the number of `vmstorage` instances (pods) running, snapshots will be stored in S3 bucket in following tree:
```
├──
│ ├── data
│ └── indexdb
├──
│ ├── data
│ └── indexdb
└──
├── data
└── indexdb
```
## Docker image
```
$ docker pull anchorfree/vmbackup-sidecar:latest
```
## Configuration
Parameters for communicating with `vmstorage` instance and credentials for accessing S3 bucket are stored in environment variables:
```bash
VMSTORAGE_HOST: localhost # vmstorage hostname (should be available via localhost as both containers are in the same Pod)
VMSTORAGE_PORT: "8482" # which port vmstorage is listening for API requests
VM_SNAPSHOT_BUCKET: my-bucket # S3 bucket name for syncing snapshot
VM_STORAGE_DATA_PATH: "/vmstorage-data" # Corresponds to --storageDataPath flag in VictoriaMetrics setup
ENVIRONMENT: "dev" # either "prod" or "dev", affects logs output (structlog for prod, plain for dev)
AWS_ACCESS_KEY_ID:
AWS_SECRET_ACCESS_KEY:
```
Command-line options:
```bash
./backup-vm -h
Usage of ./backup-vm:
-help
Show usage
-only-show-errors
Only errors and warnings are displayed. All other output is suppressed
-port int
Port to listen (default 8488)
```
## Deployment
**vmbackup-sidecar** [container](https://hub.docker.com/r/anchorfree/vmbackup-sidecar) is supposed to be running in the same [pod](https://kubernetes.io/docs/concepts/workloads/pods/pod/) as *VictoriaMetrics* **vmstorage** component.
Here is a [helm-chart](https://helm.sh/docs/developing_charts/) which implements such kind of **vmbackup-sidecar** deployment: https://github.com/AnchorFree/helm-charts/blob/master/stable/victoria-metrics/templates/vmstorage-statefulset.yaml