{"id":13563701,"url":"https://github.com/miracle2k/k8s-snapshots","last_synced_at":"2025-12-29T23:52:27.921Z","repository":{"id":47715974,"uuid":"78219308","full_name":"miracle2k/k8s-snapshots","owner":"miracle2k","description":"Automatic Volume Snapshots on Kubernetes.","archived":false,"fork":false,"pushed_at":"2023-09-07T14:19:48.000Z","size":274,"stargazers_count":350,"open_issues_count":24,"forks_count":63,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-11-04T16:45:01.731Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miracle2k.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"issuehunt":"miracle2k/k8s-snapshots"}},"created_at":"2017-01-06T16:07:52.000Z","updated_at":"2024-09-25T15:39:49.000Z","dependencies_parsed_at":"2024-08-01T13:29:40.973Z","dependency_job_id":null,"html_url":"https://github.com/miracle2k/k8s-snapshots","commit_stats":{"total_commits":175,"total_committers":25,"mean_commits":7.0,"dds":0.5142857142857142,"last_synced_commit":"5185ecb707438211e8c24deb5704bb59fdb78077"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracle2k%2Fk8s-snapshots","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracle2k%2Fk8s-snapshots/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracle2k%2Fk8s-snapshots/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracle2k%2Fk8s-snapshots/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miracle2k","download_url":"https://codeload.github.com/miracle2k/k8s-snapshots/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247074643,"owners_count":20879286,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T13:01:22.382Z","updated_at":"2025-12-29T23:52:27.879Z","avatar_url":"https://github.com/miracle2k.png","language":"Python","funding_links":["https://issuehunt.io/r/miracle2k/k8s-snapshots"],"categories":["Python","Python (1887)","Data Protection \u0026 Backup"],"sub_categories":[],"readme":"Interval-based Volume Snapshots and Expiry on Kubernetes\n========================================================\n\n**What you do:** Create a custom `SnapshotRule` resource which defines your desired snapshot intervals.\n**What I do:** Create snapshots of your volumes, and expire old ones using a Grandfather-father-son backup scheme.\n\n**Supported Environments**:\n\n- Google Compute Engine disks.\n- AWS EBS disks.\n- Digital Ocean.\n\nWant to help adding support for other backends? It's pretty straightforward.\nHave a look at the [API that backends need to implement](https://github.com/miracle2k/k8s-snapshots/blob/master/k8s_snapshots/backends/abstract.py).\n\n\nQuickstart\n----------\n\nA persistent volume claim:\n\n```\ncat \u003c\u003cEOF | kubectl apply -f -\napiVersion: \"k8s-snapshots.elsdoerfer.com/v1\"\nkind: SnapshotRule\nmetadata:\n  name: postgres\nspec:\n  deltas: P1D P30D\n  persistentVolumeClaim: postgres-data\nEOF\n```\n\nA specific AWS EC2 volume:\n\n```\ncat \u003c\u003cEOF | kubectl apply -f -\napiVersion: \"k8s-snapshots.elsdoerfer.com/v1\"\nkind: SnapshotRule\nmetadata:\n  name: mysql\nspec:\n  deltas: P1D P30D\n  backend: aws\n  disk:\n     region: eu-west-1\n     volumeId: vol-0aa6f44aad0daf9f2\nEOF\n```\n\n\nYou can also use an annotation instead of the CRDs:\n\n\n```bash\nkubectl patch pv pvc-01f74065-8fe9-11e6-abdd-42010af00148 -p \\\n  '{\"metadata\": {\"annotations\": {\"backup.kubernetes.io/deltas\": \"P1D P30D P360D\"}}}'\n``` \n\n\nUsage\n-----\n\n### How to enable backups\n\nTo backup a volume, you can create a `SnapshotRule` custom resource.\nSee more on this in the section further doiwn below.\n\nAlternatively, you can add an annotation with the name\n``backup.kubernetes.io/deltas`` to either your `PersistentVolume` or\n`PersistentVolumeClaim` resources.\n\nSince ``PersistentVolumes`` are often created automatically for you\nby Kubernetes, you may want to annotate the volume claim in your\nresource definition file. Alternatively, you can ``kubectl edit pv``\na ``PersistentVolume`` created by Kubernetes and add the annotation.\n\nThe value of the annotation are a set of deltas that define how often\na snapshot is created, and how many snapshots should be kept. See\nthe section above for more information on how deltas work.\n\nIn the end, your annotation may look like this:\n\n```\nbackup.kubernetes.io/deltas: PT1H P2D P30D P180D\n```\n\nThere is also the option of manually specifying the volume names\nto be backed up as options to the *k8s-snapshots* daemon. See below\nfor more information.\n\n\n### How the deltas work\n\nThe expiry logic of [tarsnapper](https://github.com/miracle2k/tarsnapper)\nis used.\n\nThe generations are defined by a list of deltas formatted as [ISO 8601\ndurations](https://en.wikipedia.org/wiki/ISO_8601#Durations) (this differs from\ntarsnapper). ``PT60S`` or ``PT1M`` means a minute, ``PT12H`` or ``P0.5D`` is\nhalf a day, ``P1W`` or ``P7D`` is a week. The number of backups in each\ngeneration is implied by it's and the parent generation's delta.\n\nFor example, given the deltas ``PT1H P1D P7D``, the first generation will\nconsist of 24 backups each one hour older than the previous\n(or the closest approximation possible given the available backups),\nthe second generation of 7 backups each one day older than the previous,\nand backups older than 7 days will be discarded for good.\n\nIf the daemon is not running for a while, it will still  try to approximate your desired \nsnapshot scheme as closely as possible.\n\nThe most recent backup is always kept.\n\nThe first delta is the backup interval.\n\n\nSetup\n-----\n\n`k8s-snapshots` needs access to your Kubernetes cluster resources \n(to read the desired snapshot configuration) and access to your cloud infrastructure\n(to make snapshots).\n\nDepending on your environment, it may be able to configure itself. Or, you might need to \nprovide some configuration options.\n\nUse the example deployment file given below to start off.\n\n```bash\ncat \u003c\u003cEOF | kubectl create -f -\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: k8s-snapshots\n  namespace: kube-system\nspec:\n  replicas: 1\n  selector:\n    matchLabels:\n      app: k8s-snapshots\n  template:\n    metadata:\n      labels:\n        app: k8s-snapshots\n    spec:\n      containers:\n      - name: k8s-snapshots\n        image: elsdoerfer/k8s-snapshots:latest\nEOF\n```\n\n### 1. Based on your cluster.\n\nSee the [docs/](docs/) folder for platform-specific instructions.\n\n\n### 2. For Role-based Access Control (RBAC) enabled clusters\n\nIn Kubernetes clusters with RBAC, the required permissions need to be provided to the `k8s-snapshots` pods to watch and list `persistentvolume` or `persistentvolumeclaims`. We provide a manifest to setup a `ServiceAccount` with a minimal set of permissions in [rbac.yaml](manifests/rbac.yaml).\n\n```\nkubectl apply -f manifests/rbac.yaml\n```\n\nFurthermore, under GKE, \"Because of the way Container Engine checks permissions when you create a Role or ClusterRole, you must first create a RoleBinding that grants you all of the permissions included in the role you want to create.\"\n\nIf the above kubectl apply command produces an error about \"attempt to grant extra privileges\", the following will grant _your_ user the necessary privileges *first*, so that you can then bind them to the service account:\n\n```\n  kubectl create clusterrolebinding your-user-cluster-admin-binding --clusterrole=cluster-admin --user=your.google.cloud.email@example.org\n```\n\nFinally, adjust the deployment by adding ```serviceAccountName: k8s-snapshots``` to the spec (else you'll end up using the \"default\" service account), as follows:\n\n```\n\u003csnip\u003e\n    spec:\n     serviceAccountName: k8s-snapshots\n     containers:\n      - name: k8s-snapshots\n        image: elsdoerfer/k8s-snapshots:v2.0\n\u003c/snip\u003e\n```\n\nFurther Configuration Options\n-----------------------------\n\n\n### Pinging a third party service\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd\u003ePING_URL\u003c/td\u003e\n    \u003ctd\u003e\n      We'll send a GET request to this url whenever a backup completes.\n      This is useful for integrating with monitoring services like\n      Cronitor or Dead Man's Snitch.\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\n### Make snapshot names more readable\n\nIf your persistent volumes are auto-provisioned by Kubernetes, then\nyou'll end up with snapshot names such as\n``pv-pvc-01f74065-8fe9-11e6-abdd-42010af00148``. If you want that\nprettier, set the enviroment variable ``USE_CLAIM_NAME=true``. Instead\nof the auto-generated name of the persistent volume, *k8s-snapshots*\nwill instead use the name that you give to your\n``PersistentVolumeClaim``.\n\n\n### SnapshotRule resources\n\nIt's possible to ask *k8s-snapshots* to create snapshots of volumes\nfor which no `PersistentVolume` object exists within the Kubernetes\ncluster. For example, you might have a volume at your Cloud provider\nthat you use within Kubernetes by referencing it directly.\n\nTo do this, we use a custom Kubernetes resource, `SnapshotRule`.\n\nFirst, you need to create this custom resource.\n\nOn Kubernetes 1.7 and higher:\n\n```\ncat \u003c\u003cEOF | kubectl create -f -\napiVersion: apiextensions.k8s.io/v1beta1\nkind: CustomResourceDefinition\nmetadata:\n  name: snapshotrules.k8s-snapshots.elsdoerfer.com\nspec:\n  group: k8s-snapshots.elsdoerfer.com\n  version: v1\n  scope: Namespaced\n  names:\n    plural: snapshotrules\n    singular: snapshotrule\n    kind: SnapshotRule\n    shortNames:\n    - sr\nEOF\n```\n\nOr on Kubernetes 1.6 and lower:\n\n```\ncat \u003c\u003cEOF | kubectl create -f -\napiVersion: apps/v1\nkind: ThirdPartyResource\nmetadata:\n  name: snapshot-rule.k8s-snapshots.elsdoerfer.com\ndescription: \"Defines snapshot management rules for a disk.\"\nversions:\n- name: v1\nEOF\n```\n\nYou can then create `SnapshotRule` resources:\n\n```\ncat \u003c\u003cEOF | kubectl apply -f -\napiVersion: \"k8s-snapshots.elsdoerfer.com/v1\"\nkind: SnapshotRule\nmetadata:\n  name: mysql\nspec:\n  deltas: P1D P30D\n  backend: aws\n  disk:\n     region: eu-west-1\n     volumeId: vol-0aa6f44aad0daf9f2\nEOF\n```\n\nThis is an example for backing up an EBS disk on the Amazon cloud. The\n`disk` option requires different keys, depending on the backend. See\nthe [examples folder](https://github.com/miracle2k/k8s-snapshots/tree/master/examples).\n\nYou may also point `SnapshotRule` resources to PersistentVolumes (or\nPersistentVolumeClaims). This is intended as an alternative to adding\nan annotation; it may be desirable for some to separate the snapshot\nfunctionality from the resource.\n\n```\ncat \u003c\u003cEOF | kubectl apply -f -\napiVersion: \"k8s-snapshots.elsdoerfer.com/v1\"\nkind: SnapshotRule\nmetadata:\n  name: mysql\nspec:\n  deltas: P1D P30D\n  persistentVolumeClaim: datadir-mysql\nEOF\n```\n\n\n### Backing up the etcd volumes of a kops cluster\n\nAfter setting up the custom resource definitions (see previous section), use\nsnapshot rules as defined in the `examples/backup-kops-etcd.yml` file. Reference\nthe volume ids of your etcd volumes.\n\n\n### Other environment variables\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eLOG_LEVEL\u003c/td\u003e\n    \u003ctd\u003e**Default: INFO**. Possible values: DEBUG, INFO, WARNING, ERROR\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eJSON_LOG\u003c/td\u003e\n    \u003ctd\u003e**Default: False**. Output the log messages as JSON objects for\n        easier processing.\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eTZ\u003c/td\u003e\n    \u003ctd\u003e**Default: UTC**. Used to change the timezone. ie. TZ=America/Montreal\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n\nFAQ\n----\n\n**What if I manually create snapshots for the same volumes that\n*k8s-snapshots* manages?**\n\nStarting with v0.3, when *k8s-snapshots* decides when to create the\nnext snapshot, and which snapshots it deletes, it no longer considers\nsnapshots that are not correctly labeled by it.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiracle2k%2Fk8s-snapshots","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiracle2k%2Fk8s-snapshots","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiracle2k%2Fk8s-snapshots/lists"}