{"id":24638075,"url":"https://github.com/upbound/configuration-aws-eks-velero","last_synced_at":"2025-03-20T09:26:18.201Z","repository":{"id":196041162,"uuid":"694210700","full_name":"upbound/configuration-aws-eks-velero","owner":"upbound","description":"This repository offers a configuration for uxp backup","archived":false,"fork":false,"pushed_at":"2024-01-22T21:48:58.000Z","size":520,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-01-25T10:12:52.403Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/upbound.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-09-20T14:42:33.000Z","updated_at":"2024-08-01T21:17:34.000Z","dependencies_parsed_at":"2023-12-16T00:04:08.972Z","dependency_job_id":null,"html_url":"https://github.com/upbound/configuration-aws-eks-velero","commit_stats":null,"previous_names":["haarchri/configuration-backup-controller","upbound/configuration-aws-eks-velero"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-aws-eks-velero","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-aws-eks-velero/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-aws-eks-velero/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/upbound%2Fconfiguration-aws-eks-velero/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/upbound","download_url":"https://codeload.github.com/upbound/configuration-aws-eks-velero/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244584629,"owners_count":20476578,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-25T10:12:54.952Z","updated_at":"2025-03-20T09:26:18.177Z","avatar_url":"https://github.com/upbound.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Backup Controller base configuration\n\nThis repository offers a configuration for a Backup Mechanism with Velero.\n\n[Velero](https://velero.io/) is a widely used open-source solution in the Kubernetes ecosystem for performing backup and restore operations on Kubernetes cluster resources. It is strongly recommended to incorporate Velero into any disaster recovery plan for your Crossplane control planes. This guide provides best practices for using Velero in the context of Crossplane.\n\n[Disaster Recovery When Using Crossplane for Infrastructure Provision on AWS](https://aws.amazon.com/blogs/opensource/disaster-recovery-when-using-crossplane-for-infrastructure-provisioning-on-aws/) in this blog post, we discuss different backup considerations when employing GitOps and the use of Kubernetes-native infrastructure provisioning tools on Amazon Elastic Kubernetes Service (Amazon EKS).\n\n## In the Context of Crossplane\n\nIt's important to understand that disaster recovery for Crossplane does not equate to disaster recovery for your business-critical state, such as database or cloud storage data. Here are key points to keep in mind:\n\n1. Crossplane ensures the configuration of your resources.\n2. Crossplane does not concern itself with the internal state of your resources.\n\nCrossplane enforces two primary aspects:\n\n1. Ensuring that the declared resources exist.\n2. Ensuring that the configuration of those resources matches what you declared. If there is a mismatch, Crossplane attempts to reconcile it to the desired state.\n\nCrossplane does not delve into the internal state of the resource.\n\nLikewise, if your control plane fails and goes offline, it does not necessarily mean that the resources managed by that control plane also fail. It simply means that the configuration of those resources stops reconciling while the control plane is offline, which could result in configuration drift.\n\nIt is advisable to configure Velero to store captured backup states for all your control planes in a single global bucket or blob.\n\nFor each control plane you create, this configuration creates three Velero schedules with the following intervals:\n\n- Hourly\n- Daily\n- Monthly\n\nYou can adjust the backup frequency based on your platform's Recovery Point Objective (RPO). If you need to restore state more frequently, you must take more frequent snapshots.\n\n## Backup and restore workflow\n### Backup\nVelero consists of two components:\n\n- A Velero server pod that runs in your Amazon EKS cluster\n- A command-line client (Velero CLI) that runs locally\n\nWhenever we issue a backup against an Amazon EKS cluster, Velero performs a backup of cluster resources in the following way:\n\n1. The Velero CLI makes a call to the Kubernetes API server to create a backup CRD object.\n2. Checks the scope of the backup CRD object, namely if we set filters.\n3. Queries the API server for the resources that need a backup.\n4. Compresses the retrieved Kubernetes objects into a .tar file and saves it in Amazon S3.\n\n![backup](backup.png)\n\n### Restore\n\nTo restore the backup state into a new cluster, follow these steps:\n\n1. Ensure that Crossplane and providers are not running in the new cluster. Otherwise, the order of managed resources, composites, and claims becomes critical. Failure to do this may result in race conditions and, for example, duplication of managed resources.\n2. Install Velero and configure it to use the backup data source.\n3. Restore the backed-up resources to the new cluster.\n4. Scale up crossplane and provider deployments to one in the new cluster.\n5. During the backup and restoration process, Velero preserves owner references, resource states, and external names of managed resources. The reconciliation process should proceed as if there were no changes.\n\nSimilarly, whenever we issue a restore operation:\n\n1. The Velero CLI makes a call to Kubernetes API server to create a restore CRD that will restore from an existing backup.\n2. Validates the restore CRD object.\n3. Makes a call to Amazon S3 to retrieve backup files.\n4. Initiates restore operation.\n\n![restore](restore.png)\n\n## Walkthrough\n\n### Prerequisites\n\nTo set up Velero, make sure you have the following prerequisites in place:\n\n1. Ensure you have a Crossplane control plane running.\n2. Set up an Amazon EKS cluster.\n3. Ensure you can connect to the EKS cluster using a Provider Helm with the corresponding ProviderConfig.\n\nOnce these prerequisites are met, you can proceed with the following steps to install Velero:\n\n1. Apply the configuration by running the following command:\n   \n   ```\n   kubectl apply -f examples/configuration.yaml\n   ```\n\n2. Apply the backup configuration by running the following command:\n\n   ```\n   kubectl apply -f backup.yaml\n   ```\n\nThese steps will help you bring up Velero and configure it for your environment.\n\n### Install the Velero CLI\n\nInstallation instructions vary depending on your operating system. Follow the instructions to install Velero [here](https://velero.io/docs/v1.6/basic-install/#install-the-cli)\n\n### Create Manual Backup\n\nCreate a backup of the crossplane controlplane. Be sure your\nkubectl context is set to the crossplane controlplane before\nrunning the command below.\n\nTriggering a backup as shown below may include items\nthat will already exist on the cluster where the backup will be\nrestored. These items can be skipped during restore. Consult the\n[Velero backup reference documentation](https://velero.io/docs/v1.8/backup-reference/)\nfor available inclusion and exclusion options.\nCrossplane managed resources are cluster scoped. Indicating\nthe `--include-cluster-resources` will ensure their\ninclusion in the backup.\n\n```bash\nvelero backup create uxp-backup --include-cluster-resources=true\nBackup request \"uxp-backup\" submitted successfully.\nRun `velero backup describe uxp-backup` or `velero backup logs uxp-backup` for more details.\n```\n\nLet’s check on the status of the backup and validate that the backup has been completed successfully.\n\nLook for the field `Phase`: in the output of the command. If the current `Phase` is `InProgress`, then wait a few moments and try again until you see the `Phase`: `Completed`. You can see additional details of the backup, including information such as the start time and completion time, along with the number of items backed up.\n\n```bash\nvelero backup describe uxp-backup\n\nName:         uxp-backup\nNamespace:    velero\nLabels:       velero.io/storage-location=velero-backup-default\nAnnotations:  velero.io/source-cluster-k8s-gitversion=v1.27.8-eks-8cb36c9\n              velero.io/source-cluster-k8s-major-version=1\n              velero.io/source-cluster-k8s-minor-version=27+\n\nPhase:  Completed\n\n\nNamespaces:\n  Included:  *\n  Excluded:  \u003cnone\u003e\n\nResources:\n  Included:        *\n  Excluded:        \u003cnone\u003e\n  Cluster-scoped:  included\n\nLabel selector:  \u003cnone\u003e\n\nStorage Location:  velero-backup-default\n\nVelero-Native Snapshot PVs:  auto\n\nTTL:  720h0m0s\n\nCSISnapshotTimeout:    10m0s\nItemOperationTimeout:  1h0m0s\n\nHooks:  \u003cnone\u003e\n\nBackup Format Version:  1.1.0\n\nStarted:    2023-12-15 23:28:27 +0100 CET\nCompleted:  2023-12-15 23:28:35 +0100 CET\n\nExpiration:  2024-01-14 23:28:26 +0100 CET\n\nTotal items to be backed up:  648\nItems backed up:              648\n\nVelero-Native Snapshots: \u003cnone included\u003e\n```\n\nWe can also see the backup files created by Velero in the Amazon S3 bucket we previously created:\n\n```bash\naws s3 ls s3-velero-service-configuration-aws-eks-velero/backups/uxp-backup/\n[...]\n2023-12-15 23:28:36         29 uxp-backup-csi-volumesnapshotclasses.json.gz\n2023-12-15 23:28:36         29 uxp-backup-csi-volumesnapshotcontents.json.gz\n2023-12-15 23:28:35         29 uxp-backup-csi-volumesnapshots.json.gz\n2023-12-15 23:28:35         27 uxp-backup-itemoperations.json.gz\n2023-12-15 23:28:35      20991 uxp-backup-logs.gz\n2023-12-15 23:28:35         29 uxp-backup-podvolumebackups.json.gz\n2023-12-15 23:28:35       4733 uxp-backup-resource-list.json.gz\n2023-12-15 23:28:36         49 uxp-backup-results.gz\n2023-12-15 23:28:35         29 uxp-backup-volumesnapshots.json.gz\n[...]\n2023-12-15 23:28:35     305615 uxp-backup.tar.gz\n2023-12-15 23:28:36       2905 velero-backup.json\n```\n### Create manual Restore\n\nUse the following command to restore only the upbound-system and\ndefault namespace - Note:\nthis will not include cluster scoped resources like XRDs, Compositions\nunless specified.\n\nScale Crossplane and provider pods down prior to the restore.\n```bash\nkubectl -n upbound-system get deployments —-no-headers|\\\n    awk '{print $1}'|\\\n    while read pod;\\\n    do kubectl -n upbound-system scale \\\n        --replicas=0 deployment/$pod;\\\n    done\n```\n\n```bash\nvelero restore create uxp-restore \\\n    --from-backup uxp-backup \\\n    --include-namespaces upbound-system,default \\\n    --include-cluster-resources\nRestore request \"uxp-restore\" submitted successfully.\nRun `velero restore describe uxp-restore` or `velero restore logs uxp-restore` for more details.\n```\n\nScale Crossplane and provider pods up after the restore concluded.\n```bash\nkubectl -n upbound-system get deployments —-no-headers|\\\n    awk '{print $1}'|\\\n    while read pod;\\\n    do kubectl -n upbound-system scale \\\n        --replicas=0 deployment/$pod;\\\n    done\n```\n\nWhen backups include items that exist on the restore cluster\nthere is an option to specify `--existing-resource-policy none`\nfor the restore command to skip them. Consult the\n[Velero restore reference](https://velero.io/docs/v1.12/restore-reference/)\nfor more information.\n\n#### Validate the restore was successful\nLet’s check on the status of the restore and validate that the restore has been completed successfully.\n\nLook for `Phase`: `Completed` in the output. If you see `Phase`: `InProgress`, then wait a few moments and run the command again. \n\n```bash\nvelero restore describe uxp-restore\nName:         uxp-restore\nNamespace:    velero\nLabels:       \u003cnone\u003e\nAnnotations:  \u003cnone\u003e\n\nPhase:                       Completed\nTotal items to be restored:  18\nItems restored:              18\n\nStarted:    2023-12-15 23:32:38 +0100 CET\nCompleted:  2023-12-15 23:32:40 +0100 CET\n\nWarnings:\n  Velero:     \u003cnone\u003e\n  Cluster:    \u003cnone\u003e\n  Namespaces:\n    upbound-system:  *\n\nBackup:  uxp-backup\n\nNamespaces:\n  Included:  upbound-system, default\n  Excluded:  \u003cnone\u003e\n\nResources:\n  Included:        *\n  Excluded:        nodes, events, events.events.k8s.io, backups.velero.io, restores.velero.io, resticrepositories.velero.io, csinodes.storage.k8s.io, volumeattachments.storage.k8s.io, backuprepositories.velero.io\n  Cluster-scoped:  included\n\nNamespace mappings:  \u003cnone\u003e\n\nLabel selector:  \u003cnone\u003e\n\nRestore PVs:  auto\n\nExisting Resource Policy:   \u003cnone\u003e\nItemOperationTimeout:       1h0m0s\n\nPreserve Service NodePorts:  auto\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fupbound%2Fconfiguration-aws-eks-velero","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fupbound%2Fconfiguration-aws-eks-velero","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fupbound%2Fconfiguration-aws-eks-velero/lists"}