{"id":18431952,"url":"https://github.com/jessebot/k8s-backups-tutorial","last_synced_at":"2026-04-28T18:31:28.002Z","repository":{"id":37881321,"uuid":"505892585","full_name":"jessebot/k8s-backups-tutorial","owner":"jessebot","description":"Backup pvcs in k8s using k8up","archived":false,"fork":false,"pushed_at":"2022-06-27T13:50:16.000Z","size":34,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-16T09:25:44.419Z","etag":null,"topics":["k8s","k8up","kubernetes","kubernetes-backup","persistent-volumes","postgresql","restic"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jessebot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-21T15:06:29.000Z","updated_at":"2024-04-17T15:10:48.000Z","dependencies_parsed_at":"2022-09-26T22:11:31.501Z","dependency_job_id":null,"html_url":"https://github.com/jessebot/k8s-backups-tutorial","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessebot%2Fk8s-backups-tutorial","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessebot%2Fk8s-backups-tutorial/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessebot%2Fk8s-backups-tutorial/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessebot%2Fk8s-backups-tutorial/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jessebot","download_url":"https://codeload.github.com/jessebot/k8s-backups-tutorial/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248804901,"owners_count":21164149,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["k8s","k8up","kubernetes","kubernetes-backup","persistent-volumes","postgresql","restic"],"created_at":"2024-11-06T05:26:57.396Z","updated_at":"2026-04-28T18:31:22.976Z","avatar_url":"https://github.com/jessebot.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Backups of Kubernetes Stateful Applications\nInstructions for getting your backups up and running for persistent data from stateful apps on k8s. I'll be using PostgreSQL running on a small kind cluster, with a Persistent Volume, as the example we're backing up. This was tutorial was designed from my personal lab notes while backing up Postgres for NextCloud. Really you just need the bitnami postgres helm chart installed and you should be fine.\n\nI was going to use velero, but then I read a bunch of posts saying it was [slower](https://www.reddit.com/r/kubernetes/comments/u1uqip/comment/i4fflnc/?utm_source=share\u0026utm_medium=web2x\u0026context=3), and often incompatible with alternative block storage, and since I want to backup to Backblaze b2, that leaves me in a bit of a pickle, until I found [k8up](https://github.com/k8up-io/getting-started). I'm so sorry you have to memorize another 3-4 letter k8s app... I'm also in pain.\n\nK8up seems pretty awesome though, and it wraps Restic. [Restic](https://restic.net/) has a good reputation, and consistent mention in the k8s community as well, so much so that k8up is not the only app written around restic. There's also [Stash](https://stash.run/), but stash is expensive for production workloads, and gates you out of a number of features, so we're going to ignore them for now, because k8up is free and we love free things.\n\n# Getting started\nI'm assuming you already have a cluster up and running and at least one namespace for an additional stateful app you want to monitor, which in my case is nextcloud. If you don't have a cluster, check out my basic cluster intro [here](https://github.com/jessebot/argo-vault-example).\n\nInstall k8up on your cluster\n```bash\n# Create a namespace to house it all\nkubectl create namespace k8up\n\n# need the CRDs\nkubectl apply -f https://github.com/k8up-io/k8up/releases/download/v2.3.0/k8up-crd.yaml --namespace k8up\n\nhelm repo add appuio https://charts.appuio.ch\nhelm repo update\n\n# `k8sup-backups` can be anything; can also omit \u0026 use `--generate-name` instead\nhelm install k8sup-backups appuio/k8up --namespace k8up\n```\n\n## Backblaze b2 bucket\nIf you haven't already, create a bucket for these backups, which you can do via Backblaze's api/cli, but I did through the [Web UI](https://help.backblaze.com/hc/en-us/articles/1260803542610-Creating-a-B2-Bucket-using-the-Web-UI), and there's no shame in it. This isn't an interview.\n\nThen, also if you haven't already, create an application key for restic, also using the [Web UI](https://help.backblaze.com/hc/en-us/articles/360052129034-Creating-and-Managing-Application-Keys). No Shame :triumph:\n\n## Create K8s Secrets for Restic Access to your Bucket\nK8up will run the restic commands for you, so you don't even actually need the restic cli tool locally to initialize the repo like you would noramlly need to with restic.\n\n*Note *: the following secrets need to live in the same namespace as the application you want to backup. That's why I'm putting my secrets in the nextcloud namespace, because my postgres pod runs there.\n\nCreate a k8s secret for the Restic repo secret specifically:\n```bash\n# k8up-restic-b2-repo-pw is just the name I used in my `backup.yaml`/`schedule.yaml`\n# can be anything as long as all match in secret and backup/schedule resources\n kubectl create secret generic k8up-restic-b2-repo-pw  --from-literal=password=$YOUR_PASSWORD_HERE --namespace nextcloud\n```\n*tip*: if you put a space before the command, bash won't save it in history\n\nCreate a secret with your Backblaze b2 application key id and application key for your bucket like:\n```bash\n# k8up-restic-b2-creds-nextcloud-pg is just the name I used, but\n# can be anything as long as all match in secret and backup/schedule resources\n kubectl create secret generic k8up-restic-b2-creds-nextcloud-pg --from-literal=application-key-id=$YOUR_KEY_ID_HERE --from-literal=application-key=$YOUR_KEY_HERE --namespace nextcloud\n```\nThere's definitely cooler vault based ways to get some of this done, but my vault docs are under construction, so we're doing it the old fashion way, less great, but not the worst.\n\n## Schedule Restic Backups with k8up Schedule CRD\n\n### Application Aware Backups\nAccording to the [k8up docs](https://k8up.io/k8up/2.3/how-tos/application-aware-backups.html#_postgresql), You'll need to annotate your pods you want backed up with `k8up.io/backup` and in my case, since I'm using postgres, I'd need to add this to my annotations on the pod:\n```yaml\nmetadata:\n  annotations:\n      k8up.io/backupcommand: sh -c 'PGDATABASE=\"$POSTGRES_DB\" PGUSER=\"$POSTGRES_USER\" PGPASSWORD=\"$POSTGRES_PASSWORD\" pg_dump --clean'\n      k8up.io/file-extension: .sql\n```\n\n#### Annotate via kubectl\nYou could literally do a `kubectl edit` on this, which throws you into your default text editor, normally vi/vim, or you could knock it out entirely via the command line. For the edit, assuming nextcloud with postgres, you want something like:\n```\nkubectl edit pod nextcloud-postgresql-0 --namespace nextcloud\n```\nThen find the metadata section, and past the above yaml annotations directly in in there before saving and exiting.\n\nAnother example for annotating a postgres nextcloud pod, assuming you have `POSTGRES_DB`, `POSTGRES_PASSWORD`, and `POSTGRES_USER` as env variables configured for the pod already.\n```bash\nkubectl annotate pods nextcloud-postgresql-0 k8up.io/file-extension=.sql --namespace nextcloud\nkubectl annotate pods nextcloud-postgresql-0 k8up.io/backupcommand=\"sh -c 'PGDATABASE=\"$POSTGRES_DB\" PGUSER=\"$POSTGRES_USER\" PGPASSWORD=\"$POSTGRES_PASSWORD\" pg_dump --clean'\" --namespace nextcloud\n```\n\n#### Annotate via Helm\nIf you're using the postgres helm chart, or a helm chart that chains postgres, you can do something like this in your `values.yaml`:\n```yaml\n## PostgreSQL chart configuration\n## for more options see https://github.com/bitnami/charts/tree/master/bitnami/postgresql\npostgresql:\n  enabled: true\n  global:\n    postgresql:\n      auth:\n        username: nextcloud\n        password: areallycoolpassword\n        database: nextcloud\n  primary:\n    podAnnotations:\n      k8up.io/backupcommand: \"sh -c 'PGDATABASE=\\\"$POSTGRES_DB\\\" PGUSER=\\\"$POSTGRES_USER\\\" PGPASSWORD=\\\"$POSTGRES_PASSWORD\\\" pg_dump --clean'\"\n      k8up.io/file-extension: .sql\n    persistence:\n      enabled: true\n```\n\n### Create a one time backup to b2\nGrab the [`backup.yaml`](https://raw.githubusercontent.com/jessebot/k8s-backups-tutorial/main/backup-yamls/backup.yaml) from the `backup-yamls` directory in this repo. Make sure the bucket parameters in `backup.yaml` (and `schedule.yaml`, if you use it) are set to your b2 bucket. I've left `nextcloud` here as an example:\n```yaml\napiVersion: k8up.io/v1\nkind: Backup\nmetadata:\n  name: backup-nextcloud-b2\n  namespace: nextcloud\nspec:\n  failedJobsHistoryLimit: 2\n  successfulJobsHistoryLimit: 2\n  backend:\n    repoPasswordSecretRef:\n      name: k8up-restic-b2-repo-pw\n      key: password\n    b2:\n      bucket: nextcloud-pgsql\n      accountIDSecretRef:\n        name: k8up-restic-b2-creds-nextcloud-pg\n        key: application-key-id\n      accountKeySecretRef:\n        name: k8up-restic-b2-creds-nextcloud-pg\n        key: application-key\n```\n\n```bash\n# create the backup `schedule` resource\nkubectl apply -f backup-yamls/backup.yaml\n```\nYou can find a further explanation on how to do this with minio in the [k8up docs](https://k8up.io/k8up/2.3/how-tos/backup.html).\n\n### Create a scheduled backup to b2\nCreate the aforementioned `schedule.yaml` to backup once a day and prune monthly.\n```bash\n# create the backup `schedule` resource\nkubectl apply -f backup-yamls/schedule.yaml\n```\n\n:blue-heart: Success! You should be good to go :)\n\n## Possible issues\nIf you run into an issue with scheduled backups or repeat backups on b2 specifically, you may be hitting this [k8up issue #690](https://github.com/k8up-io/k8up/issues/690), for which a fix has been merged. You should be able to just change your `values.yaml` to point to the latest release, link this:\n\n```yaml\n# values.yaml\nimage:\n  tag: \"v2.3.3\"\n\nk8up:\n  backupImage:\n    tag: \"v2.3.3\"\n```\n\nYou can then apply that fix via helm with:\n```bash\n# NOTE: k8sup-backups is the name of *your* release that you want to upgrade\n# Can be found by doing a `helm list -A`, -A will list all namespaces\nhelm upgrade k8sup-backups appuio/k8up --namespace k8up --values values.yaml\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessebot%2Fk8s-backups-tutorial","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjessebot%2Fk8s-backups-tutorial","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessebot%2Fk8s-backups-tutorial/lists"}