{"id":13581577,"url":"https://github.com/AnchorFree/vmbackup-sidecar","last_synced_at":"2025-04-06T10:32:35.624Z","repository":{"id":79769848,"uuid":"205406545","full_name":"AnchorFree/vmbackup-sidecar","owner":"AnchorFree","description":"Backup sidecar container for VictoriaMetrics","archived":false,"fork":false,"pushed_at":"2023-05-08T20:16:28.000Z","size":47,"stargazers_count":10,"open_issues_count":2,"forks_count":2,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-03-18T11:33:16.537Z","etag":null,"topics":["backup","docker-image","project-monitoring","s3","sidecar","sidecar-container","snapshot","sync","team-kyiv-devops","victoriametrics"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AnchorFree.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-08-30T15:14:59.000Z","updated_at":"2023-02-27T14:08:34.000Z","dependencies_parsed_at":"2024-01-16T20:59:06.266Z","dependency_job_id":"29f40d51-3321-4279-a523-6db15f041888","html_url":"https://github.com/AnchorFree/vmbackup-sidecar","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnchorFree%2Fvmbackup-sidecar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnchorFree%2Fvmbackup-sidecar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnchorFree%2Fvmbackup-sidecar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AnchorFree%2Fvmbackup-sidecar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AnchorFree","download_url":"https://codeload.github.com/AnchorFree/vmbackup-sidecar/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247470366,"owners_count":20944146,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","docker-image","project-monitoring","s3","sidecar","sidecar-container","snapshot","sync","team-kyiv-devops","victoriametrics"],"created_at":"2024-08-01T15:02:06.157Z","updated_at":"2025-04-06T10:32:35.211Z","avatar_url":"https://github.com/AnchorFree.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"[![GitHub license](https://img.shields.io/github/license/AnchorFree/vmbackup-sidecar.svg)](https://github.com/AnchorFree/vmbackup-sidecar/blob/master/LICENSE)\n[![Go Report](https://goreportcard.com/badge/github.com/AnchorFree/vmbackup-sidecar)](https://goreportcard.com/report/github.com/AnchorFree/vmbackup-sidecar)\n![GitHub release (latest by date including pre-releases)](https://img.shields.io/github/v/release/AnchorFree/vmbackup-sidecar?include_prereleases)\n\n**Warning:** use this tool _only_ with VictoriaMetrics version \u003c 1.29.0.\nStarting from\n[v1.29.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.29.0) VictoriaMetrics added\nnative `vmbackup` and `vmrestore` tools for fast backups on GCS and S3.\n\nvmbackup-sidecar\n----------------\n\nProvides a sidecar backup service for [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics/tree/cluster) *vmstorage* component.\n\n\n## How it works\n\n**vmbackup-sidecar** provides `/backup/create` API endpoint. After receiving `GET` request, it creates *VictoriaMetrics* `vmstorage` snapshot via [snapshot API](https://github.com/VictoriaMetrics/VictoriaMetrics/wiki/Cluster-VictoriaMetrics#url-format) and syncs it with S3 bucket using [aws s3 sync](https://docs.aws.amazon.com/cli/latest/reference/s3/sync.html). When sync with S3 is completed, all `vmstorage` snapshots are removed to free up space. Depending on the number of `vmstorage` instances (pods) running, snapshots will be stored in S3 bucket in following tree:\n\n```\n\u003cs3-bucket-name\u003e\n├── \u003cvmstorage-pod-1\u003e\n│   ├── data\n│   └── indexdb\n├── \u003cvmstorage-pod-2\u003e\n│   ├── data\n│   └── indexdb\n└── \u003cvmstorage-pod-3\u003e\n    ├── data\n    └── indexdb\n```\n\n\n## Docker image\n\n```\n$ docker pull anchorfree/vmbackup-sidecar:latest\n```\n\n\n## Configuration\n\nParameters for communicating with `vmstorage` instance and credentials for accessing S3 bucket are stored in environment variables:\n\n```bash\nVMSTORAGE_HOST: localhost      # vmstorage hostname (should be available via localhost as both containers are in the same Pod)\nVMSTORAGE_PORT: \"8482\"         # which port vmstorage is listening for API requests\nVM_SNAPSHOT_BUCKET: my-bucket  # S3 bucket name for syncing snapshot\nVM_STORAGE_DATA_PATH: \"/vmstorage-data\"  # Corresponds to --storageDataPath flag in VictoriaMetrics setup\nENVIRONMENT: \"dev\"            # either \"prod\" or \"dev\", affects logs output (structlog for prod, plain for dev)\nAWS_ACCESS_KEY_ID: \u003caccess_key\u003e\nAWS_SECRET_ACCESS_KEY: \u003csecret_key\u003e\n```\n\nCommand-line options:\n\n```bash\n./backup-vm -h\nUsage of ./backup-vm:\n  -help\n    Show usage\n  -only-show-errors\n    Only errors and warnings are displayed. All other output is suppressed\n  -port int\n    Port to listen (default 8488)\n```\n\n## Deployment\n\n**vmbackup-sidecar** [container](https://hub.docker.com/r/anchorfree/vmbackup-sidecar) is supposed to be running in the same [pod](https://kubernetes.io/docs/concepts/workloads/pods/pod/) as *VictoriaMetrics* **vmstorage** component.\n\nHere is a [helm-chart](https://helm.sh/docs/developing_charts/) which implements such kind of **vmbackup-sidecar** deployment: https://github.com/AnchorFree/helm-charts/blob/master/stable/victoria-metrics/templates/vmstorage-statefulset.yaml\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAnchorFree%2Fvmbackup-sidecar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAnchorFree%2Fvmbackup-sidecar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAnchorFree%2Fvmbackup-sidecar/lists"}