{"id":13644552,"url":"https://github.com/FairwindsOps/gemini","last_synced_at":"2025-04-21T10:33:29.276Z","repository":{"id":37905314,"uuid":"250851187","full_name":"FairwindsOps/gemini","owner":"FairwindsOps","description":"Automated backups of PersistentVolumeClaims in Kubernetes using VolumeSnapshots","archived":false,"fork":false,"pushed_at":"2025-03-17T13:43:48.000Z","size":503,"stargazers_count":393,"open_issues_count":6,"forks_count":28,"subscribers_count":18,"default_branch":"main","last_synced_at":"2025-04-08T10:35:52.948Z","etag":null,"topics":["backup","backup-utility","backups","csi","fairwinds-incubator","kubernetes","persistent-storage","pvc","schedule-backups","volumesnapshot-api","volumesnapshots"],"latest_commit_sha":null,"homepage":"https://fairwinds.com","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FairwindsOps.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-28T17:13:22.000Z","updated_at":"2025-04-06T18:39:51.000Z","dependencies_parsed_at":"2024-01-13T16:23:44.472Z","dependency_job_id":"2513017b-e97c-4962-8638-55e485c95a81","html_url":"https://github.com/FairwindsOps/gemini","commit_stats":{"total_commits":179,"total_committers":16,"mean_commits":11.1875,"dds":0.7206703910614525,"last_synced_commit":"520cda0d78d4643010ce3003c472fd1204d022bb"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FairwindsOps%2Fgemini","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FairwindsOps%2Fgemini/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FairwindsOps%2Fgemini/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FairwindsOps%2Fgemini/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FairwindsOps","download_url":"https://codeload.github.com/FairwindsOps/gemini/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250040483,"owners_count":21365113,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","backup-utility","backups","csi","fairwinds-incubator","kubernetes","persistent-storage","pvc","schedule-backups","volumesnapshot-api","volumesnapshots"],"created_at":"2024-08-02T01:02:07.132Z","updated_at":"2025-04-21T10:33:29.269Z","avatar_url":"https://github.com/FairwindsOps.png","language":"Go","readme":"\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"/logo.png\" alt=\"Gemini Logo\" /\u003e\n  \u003cbr\u003e\n\n  [![Version][version-image]][version-link] [![CircleCI][circleci-image]][circleci-link] [![Go Report Card][goreport-image]][goreport-link] [![Codecov][codecov-image]][codecov-link]\n\u003c/div\u003e\n\n[version-image]: https://img.shields.io/static/v1.svg?label=Version\u0026message=0.0.1\u0026color=239922\n[version-link]: https://github.com/FairwindsOps/gemini\n\n[goreport-image]: https://goreportcard.com/badge/github.com/FairwindsOps/gemini\n[goreport-link]: https://goreportcard.com/report/github.com/FairwindsOps/gemini\n\n[circleci-image]: https://circleci.com/gh/FairwindsOps/gemini.svg?style=svg\n[circleci-link]: https://circleci.com/gh/FairwindsOps/gemini\n\n[codecov-image]: https://codecov.io/gh/FairwindsOps/gemini/branch/master/graph/badge.svg?token=7C20K7SYNR\n[codecov-link]: https://codecov.io/gh/FairwindsOps/gemini\n\nGemini is a Kubernetes CRD and operator for managing `VolumeSnapshots`. This allows you\nto create a snapshot of the data on your `PersistentVolumes` on a regular schedule,\nretire old snapshots, and restore snapshots with minimal downtime.\n\n## Installation\nThe Gemini Helm chart will install both the CRD and the operator into your cluster\n\n```bash\nkubectl create ns gemini\nhelm repo add fairwinds-stable https://charts.fairwinds.com/stable\nhelm install gemini fairwinds-stable/gemini --namespace gemini\n```\n\n### Prerequisites\nYou'll need to have the `VolumeSnapshot` API available in your cluster. This API is in\n[beta as of Kubernetes 1.17](https://kubernetes.io/docs/concepts/storage/volume-snapshots/),\nand was introduced as alpha in 1.12.\n\nTo check if your cluster has `VolumeSnapshots` available, you can run\n```bash\nkubectl api-resources | grep volumesnapshots\n```\n\n* To enable on v1.12-16, set the flag `--feature-gates=VolumeSnapshotDataSource=true` on the API server binary [source](https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/#kubernetes-snapshots-requirements)\n* To enable VolumeSnapshots on kops, see our [instructions here](/examples/bash)\n* Depending on your environment, you may need to configure the VolumeSnapshot API as well as the CSI. Fortunately, some managed Kubernetes providers like DigitalOcean support VolumeSnapshots by default, even on older versions\n\nBefore getting started with Gemini, it's a good idea to make sure you're able to\n[create a VolumeSnapshot manually](https://kubernetes.io/docs/concepts/storage/volume-snapshots/#volumesnapshots).\n\n### Upgrading to V2\nVersion 2.0 of Gemini updates the CRD from `v1beta1` to `v1`. There are no substantial\nchanges, but `v1` adds better support for PersistentVolumeClaims on Kubernetes 1.25.\n\nIf you want to keep the v1beta1 CRD available, you can run:\n```\nkubectl apply -f https://raw.githubusercontent.com/FairwindsOps/gemini/main/pkg/types/snapshotgroup/v1beta1/crd-with-beta1.yaml\n```\nbefore upgrading, and add `--skip-crds` when running `helm install`.\n\n## Usage\n\n### Snapshots\nGemini can schedule snapshots for an existing PVC, or create a new PVC to back up.\n\n#### Schedules\n\nThe `schedule` parameter tells Gemini how often to create snapshots, and how many historical snapshots to keep.\n\nFor example, the following schedule tells Gemini to create a snapshot every day, keeping two weeks worth of history:\n```yaml\napiVersion: gemini.fairwinds.com/v1\nkind: SnapshotGroup\nmetadata:\n  name: test-volume\nspec:\n  persistentVolumeClaim:\n    claimName: postgres\n  schedule:\n    - every: day\n      keep: 14\n```\n\nFor a more complex example, Gemini can create new snapshots every 10 minutes,\nalways keep the last 3 snapshots, and preserve historical hourly, daily, monthly, and yearly snapshots.\n\n```yaml\napiVersion: gemini.fairwinds.com/v1\nkind: SnapshotGroup\nmetadata:\n  name: test-volume\nspec:\n  persistentVolumeClaim:\n    claimName: postgres\n  schedule:\n    - every: 10 minutes\n      keep: 3\n    - every: hour\n      keep: 1\n    - every: day\n      keep: 1\n    - every: month\n      keep: 1\n    - every: year\n      keep: 1\n```\n\nNote that `keep` specifies how many historical snapshots you want, _in addition_ to the most recent snapshot.\nThis way the schedule\n```yaml\n- every: 10 minutes\n  keep: 3\n```\nwill always give you _at least_ 30 minutes of snapshot coverage. But you will see four snapshots at any given time.\nE.g. right after a new snapshot is created, you'll see snapshots for\n* 0m ago\n* 10m ago\n* 20m ago\n* 30m ago\n\n\n#### Using an Existing PVC\n\u003e See the [extended example](/examples/codimd/README.md)\nThe following example schedules snapshots every 10 minutes for a pre-existing PVC named `postgres`.\n\n```yaml\napiVersion: gemini.fairwinds.com/v1\nkind: SnapshotGroup\nmetadata:\n  name: test-volume\nspec:\n  persistentVolumeClaim:\n    claimName: postgres\n  schedule:\n    - every: 10 minutes\n      keep: 3\n```\n\n#### Creating a New PVC\nYou can also specify an entire PVC spec inside the SnapshotGroup if you'd like Gemini to create\nthe PVC for you.\n```yaml\napiVersion: gemini.fairwinds.com/v1\nkind: SnapshotGroup\nmetadata:\n  name: test-volume\nspec:\n  persistentVolumeClaim:\n    spec:\n      accessModes:\n        - ReadWriteOnce\n      resources:\n        requests:\n          storage: 1Gi\n  schedule:\n    - every: 10 minutes\n      keep: 3\n```\n\nThe PVC will have the same name as the SnapshotGroup, (in this example, `test-volume`)\n\n#### Snapshot Spec\nYou can use the `spec.template` field to set the template for any `VolumeSnapshots` that get created,\nmost notably the name of the [snapshot class](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/)\nyou want to use.\n\n```yaml\napiVersion: gemini.fairwinds.com/v1\nkind: SnapshotGroup\nmetadata:\n  name: test-volume\nspec:\n  persistentVolumeClaim:\n    claimName: postgres\n  schedule:\n    - every: \"10 minutes\"\n      keep: 3\n  template:\n    spec:\n      volumeSnapshotClassName: test-snapshot-class\n```\n\n### Restore\n\u003e Caution: you cannot alter a PVC without some downtime!\nYou can restore your PVC to a particular point in time using an annotation.\n\nFirst, check out what `VolumeSnapshots` are available:\n```bash\n$ kubectl get volumesnapshot\nNAME                           AGE\ntest-volume-1585945609         15s\n```\n\nNext, you'll need to remove any Pods that are using the PVC:\n```bash\n$ kubectl scale all --all --replicas=0\n```\n\nThen, copy the timestamp from the first step, and use that to annotate the `SnapshotGroup`:\n```bash\n$ kubectl annotate snapshotgroup/test-volume --overwrite \\\n  \"gemini.fairwinds.com/restore=1585945609\"\n```\n\nFinally, you can scale your Pods back up:\n```bash\n$ kubectl scale all --all --replicas=1\n```\n\n## End-to-End Example\nTo see gemini working end-to-end, check out [the CodiMD example](examples/codimd)\n\n## Caveats\n* Like the VolumeSnapshot API it builds on, Gemini is **currently in beta**\n* Be sure to test out both the snapshot and restore process to ensure Gemini is working properly\n* VolumeSnapshots simply grab the current state of the volume, without respect for things like in-flight database transactions. You may find you need to stop the application in order to get a consistently usable VolumeSnapshot.\n\n\u003c!-- Begin boilerplate --\u003e\n## Join the Fairwinds Open Source Community\n\nThe goal of the Fairwinds Community is to exchange ideas, influence the open source roadmap,\nand network with fellow Kubernetes users.\n[Chat with us on Slack](https://join.slack.com/t/fairwindscommunity/shared_invite/zt-e3c6vj4l-3lIH6dvKqzWII5fSSFDi1g)\n[join the user group](https://www.fairwinds.com/open-source-software-user-group) to get involved!\n\n\u003ca href=\"https://www.fairwinds.com/t-shirt-offer?utm_source=gemini\u0026utm_medium=gemini\u0026utm_campaign=gemini-tshirt\"\u003e\n  \u003cimg src=\"https://www.fairwinds.com/hubfs/Doc_Banners/Fairwinds_OSS_User_Group_740x125_v6.png\" alt=\"Love Fairwinds Open Source? Share your business email and job title and we'll send you a free Fairwinds t-shirt!\" /\u003e\n\u003c/a\u003e\n\n## Other Projects from Fairwinds\n\nEnjoying Gemini? Check out some of our other projects:\n* [Polaris](https://github.com/FairwindsOps/Polaris) - Audit, enforce, and build policies for Kubernetes resources, including over 20 built-in checks for best practices\n* [Goldilocks](https://github.com/FairwindsOps/Goldilocks) - Right-size your Kubernetes Deployments by compare your memory and CPU settings against actual usage\n* [Pluto](https://github.com/FairwindsOps/Pluto) - Detect Kubernetes resources that have been deprecated or removed in future versions\n* [Nova](https://github.com/FairwindsOps/Nova) - Check to see if any of your Helm charts have updates available\n* [rbac-manager](https://github.com/FairwindsOps/rbac-manager) - Simplify the management of RBAC in your Kubernetes clusters\n","funding_links":[],"categories":["kubernetes","Go","OPS"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FFairwindsOps%2Fgemini","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FFairwindsOps%2Fgemini","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FFairwindsOps%2Fgemini/lists"}