{"id":19416986,"url":"https://github.com/quobyte/quobyte-csi-driver","last_synced_at":"2025-04-24T13:33:14.012Z","repository":{"id":43398709,"uuid":"146604348","full_name":"quobyte/quobyte-csi-driver","owner":"quobyte","description":"Quobyte CSI driver","archived":false,"fork":false,"pushed_at":"2024-05-10T19:13:30.000Z","size":7502,"stargazers_count":18,"open_issues_count":0,"forks_count":11,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-05-22T08:31:46.022Z","etag":null,"topics":["container-storage","container-storage-interface","csi","go","k8s-cluster","kubernetes","quobyte"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/quobyte.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-29T13:30:33.000Z","updated_at":"2024-06-13T06:25:49.043Z","dependencies_parsed_at":"2024-01-14T08:33:13.545Z","dependency_job_id":"87c9d8d8-b917-48a8-99a3-26516675eb05","html_url":"https://github.com/quobyte/quobyte-csi-driver","commit_stats":{"total_commits":149,"total_committers":10,"mean_commits":14.9,"dds":0.1476510067114094,"last_synced_commit":"9a5370c2528d33f227ad35584dd63a16add31d7e"},"previous_names":["quobyte/quobyte-csi"],"tags_count":35,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quobyte%2Fquobyte-csi-driver","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quobyte%2Fquobyte-csi-driver/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quobyte%2Fquobyte-csi-driver/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/quobyte%2Fquobyte-csi-driver/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/quobyte","download_url":"https://codeload.github.com/quobyte/quobyte-csi-driver/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250636261,"owners_count":21463037,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["container-storage","container-storage-interface","csi","go","k8s-cluster","kubernetes","quobyte"],"created_at":"2024-11-10T13:06:34.697Z","updated_at":"2025-04-24T13:33:13.650Z","avatar_url":"https://github.com/quobyte.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Quobyte CSI\n\nQuobyte CSI is the implementation of\n [Container Storage Interface (CSI)](https://github.com/container-storage-interface/spec/tree/release-1.0).\n Quobyte CSI enables easy integration of Quobyte Storage into Kubernetes. Current Quobyte CSI driver\n supports the following functionality\n\n* Dynamic Volume Create\n* Volume Delete\n* Pre-provisioned volumes (Delete policy does not apply to these volumes)\n* Volume Expansion (Only dynamically provisioned volumes can be expanded)\n  * Quobyte supports volumes with unlimited size, expanding an unlimited sized\n    volume restricts the volume size to expanded size.\n* Volume snapshots\n\n## Select Quobyte CSI Driver Release\n\n1. Choose a Quobyte CSI release from [available releases](https://github.com/quobyte/quobyte-csi/releases)\n\n2. Follow the instructions specific to that release\n\n## Index\n\n* [Requirements](#requirements)\n* [Deploy Quobyte clients](docs/install_client)\n  * [Quobyte 2.x](docs/install_client/deploy_clients_2_x.md)\n  * [Quobyte 3.x](docs/install_client/deploy_clients_3_x.md)\n* [Deploy Quobyte CSI](#deploy-quobyte-csi-driver)\n* [Quobyte Version Compatibility](docs/quobyte_versions.md)\n* [Snapshotter Setup](#snapshotter-setup) (**required only if snapshots are enabled**)\n* [Usage Examples](#use-quobyte-volumes-in-kubernetes)\n  * [Use Quobyte volumes in Kubernetes](#use-quobyte-volumes-in-kubernetes)\n    * [Dynamic volume provisioning](#dynamic-volume-provisioning)\n    * [Use existing volumes](#use-existing-volumes)\n  * [Volume Snapshots](#volume-snapshots)\n    * [Dynamic Snapshots](#dynamic-snapshots)\n    * [Pre-provisioned Snapshots](#pre\\-provisioned-snapshots)\n  * [Shared Volumes](usage-example/shared-volume/README.md)\n* [Update Quobyte CSI or Client](docs/update_quobyte_csi_or_clients.md#index)\n  * [Update Quobyte CSI Driver](docs/update_quobyte_csi_or_clients.md#update-quobyte-csi-driver)\n  * [Update Quobyte Client](docs/update_quobyte_csi_or_clients.md#update-quobyte-client)\n    * [Application pod recovery](docs/update_quobyte_csi_or_clients.md#application-pod-recovery)\n* [Harden Access with Quobyte Access Keys](docs/quobyte_access_keys.md) (**requires Quobyte 3.1 or later**)\n* [Uninstall Quobyte CSI](#uninstall-quobyte-csi)\n* [Quobyte Client Upgrade Example](docs/client_update_example.md)\n* [Multi-cluster setup](docs/multi-cluster-setup.md)\n* [Collect Quobyte CSI logs](docs/collect_quobyte_csi_logs.md)\n* [Run e2e tests on kind cluster](docs/Kind-cluster-for-e2e-tests.md)\n\n## Requirements\n\n* Requires [Helm](https://helm.sh/docs/intro/install/#from-script) and [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/)\n* Requires at least Kubernetes v1.22\n* Quobyte installation with reachable registry and api services from the Kubernetes nodes and pods\n* Quobyte client with mount path as\n  \u003c[values.clientMountPoint](https://github.com/quobyte/quobyte-csi/blob/4671450b0dec5fe162f78f9e35c6c6fe90e3f86b/quobyte-csi-driver/values.yaml#L18)\u003e`/mounts`. Please see [Deploy Quobyte clients](docs/install_client) for Quobyte client installation instructions.\n  * To use Quobyte access keys, the Quobyte client (requires Quobyte version 3.0 or above) should\n   be deployed with **--enable-access-contexts**. Additionally, the metadata cache (global policy)\n   should be disabled.\n* If you have load balancer for Quobyte API, the load balancer must be configured with sticky\n  sessions.\n* Requires [additional setup](#snapshotter-setup) to use volume snapshots\n\n## Deploy Quobyte CSI Driver\n\n**Note:** Quobyte CSI driver automatically deletes all the application pods with\n [stale](https://github.com/kubernetes/kubernetes/issues/70013) Quobyte CSI volumes\n and leaves the new pod creation to kubernetes. To reschedule a new pod\n automatically by k8s, applications should be deployed with `Deployment/ReplicaSet/StatefulSets`\n but not as a plain `Pod`.\n\n1. Add `quobyte-csi-driver` helm repository to your `helm` repos\n\n    ```bash\n    helm repo add quobyte-csi-driver https://quobyte.github.io/quobyte-csi-driver/helm\n    ```\n\n    If the `quobyte-csi-driver` helm repo already exists in your helm repositories, you should update\n    the repo to get the new Quobyte CSI Driver releases. Update the repo\n\n    ```bash\n    helm repo update quobyte-csi-driver\n    ``` \n\n2. List all available Quobyte CSI versions\n\n    ```bash\n    helm search repo quobyte-csi-driver/quobyte-csi-driver -l\n    ```\n\n3. List all customization options for Quobyte CSI driver\n\n    ```bash\n    helm show values quobyte-csi/quobyte-csi [--version \u003cchart-version\u003e] # or use other \"show \u003csubcommands\u003e\"\n    ```\n\n4. Edit [Quobyte CSI driver configuration](quobyte-csi-driver/values.yaml) (./quobyte-csi-driver/values.yaml) and configure CSI driver\n   with Quobyte API, other required information.\n\n5. (optional) generate driver deployment `.yaml` and verify the configuration.\n\n    ```bash\n    helm template ./quobyte-csi-driver --debug \u003e csi-driver.yaml\n    ```\n\n6. Deploy the Quobyte CSI driver with customizations\n\n    ```bash\n    # Deploys helm chart with name \"quobyte-csi\".\n    # Please change quobyte-csi as required\n    helm install quobyte-csi quobyte-csi-driver/quobyte-csi-driver [--version \u003cchart-version\u003e]\n      \\ --set quobyte.apiURL=\"\u003cyour-api-url\u003e\" ....\n    ```\n\n    or\n\n    ```bash\n    helm install quobyte-csi quobyte-csi-driver/quobyte-csi-driver [--version \u003cchart-version\u003e]\n      \\ -f \u003cyour-customized-values.yaml\u003e [--set quobyte.apiURL=\"\u003cyour-api-url\u003e\" .. other overrides]\n    ```\n\n7. Verify the status of Quobyte CSI driver pods\n\n    Deploying Quobyte CSI driver should create a CSIDriver object\n     with your `csiProvisionerName` (this may take few seconds)\n\n    ```bash\n    CSI_PROVISIONER=\"\u003cYOUR-csiProvisionerName\u003e\"\n    kubectl get CSIDriver | grep ^${CSI_PROVISIONER}\n    ```\n\n    The Quobyte CSI driver is ready for use, if you see `quobyte-csi-controller-x`\n    pod running on any one node and `quobyte-csi-node-xxxxx`\n    running on every node of the Kubernetes cluster.\n\n    ```bash\n    CSI_PROVISIONER=$(echo $CSI_PROVISIONER | tr \".\" \"-\")\n    kubectl -n kube-system get po -owide | grep ^quobyte-csi-.*-${CSI_PROVISIONER}\n    ```\n\n8. Make sure your CSI driver is running against the expected Quobyte API endpoint\n\n    ```bash\n    kubectl -n kube-system exec -it \\\n    \"$(kubectl get po -n kube-system | grep -m 1 ^quobyte-csi-node-$CSI_PROVISIONER \\\n    |  cut -f 1 -d' ')\" -c quobyte-csi-driver -- env | grep QUOBYTE_API_URL\n    ```\n\n    The above command should print your Quobyte API endpoint. Otherwise, uninstall\n    Quobyte CSI driver and install again with the correct Quobyte API URL.\n\n## Examples\n\n`Note:` [k8s storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/) is\n  immutable. Do not delete existing definitions, such a deletion could cause issues for existing\n  PV/PVCs.\n\n### Use Quobyte volumes in Kubernetes\n\n`Note:` This section uses `example/` deployment files for demonstration. These should be modified\n  with your deployment configurations such as `namespace`, `quobyte registry`, `Quobyte API user credentials` etc.\n\nWe use `quobyte` namespace for the examples. Create the namespace\n\n  ```bash\n  kubectl create ns quobyte\n  ```\n\nQuobyte requires a secret to authenticate volume create and delete requests. Create this secret with\n your Quobyte API login credentials (Kubernetes requires base64 encoding for secret data which can be obtained\n with the command `echo -n \"value\" | base64`). Please encode your user name, password (and optionally access key\n information) in base64 and update [example/quobyte-admin-credentials.yaml](example/quobyte-admin-credentials.yaml). If provided, access key\n ensures only authorized user can access the tenant and volumes (users must be restricted to their own namespace in k8s cluster).\n\n  ```bash\n  kubectl create -f example/quobyte-admin-credentials.yaml\n  ```\n\nCreate a [storage class](example/StorageClass.yaml) with the `provisioner` set to `csi.quobyte.com` along with other configuration\n parameters. You could create multiple storage classes by varying `parameters` such as\n  `quobyteTenant` etc.\n\n  ```bash\n  kubectl create -f example/StorageClass.yaml\n  ```\n\n#### Dynamic volume provisioning\n\nCreating a PVC referencing the storage class created in the previous step would provision dynamic\n volume. The secret `csi.storage.k8s.io/provisioner-secret-name` from the namespace `csi.storage.k8s.io/provisioner-secret-namespace`\n in the referenced StorageClass will be used to authenticate volume creation and deletion.\n\n1. Create [PVC](example/pvc-dynamic-provision.yaml) to trigger dynamic provisioning\n\n    ```bash\n    kubectl create -f example/pvc-dynamic-provision.yaml\n    ```\n\n2. Mount the PVC in a [pod](example/nginx-demo-pod-with-dynamic-vol.yaml) as shown in the following example\n\n    ```bash\n    kubectl create -f example/nginx-demo-pod-with-dynamic-vol.yaml\n    ```\n\n3. Wait for the pod to be in running state\n\n    ```bash\n    kubectl get po -w | grep 'nginx-dynamic-vol'\n    ```\n\n4. Once the pod is running, copy the [index file](example/index.html) to the deployed nginx pod\n\n    ```bash\n    kubectl cp example/index.html nginx-dynamic-vol:/tmp\n    kubectl exec -it nginx-dynamic-vol -- mv /tmp/index.html /usr/share/nginx/html/\n    kubectl exec -it nginx-dynamic-vol -- chown -R nginx:nginx /usr/share/nginx/html/\n    ```\n\n5. Access the home page served by nginx pod from the command line\n\n    ```bash\n    curl http://$(kubectl get pods nginx-dynamic-vol -o yaml | grep ' podIP:' | awk '{print $2}'):80\n    ```\n\n    Above command should retrieve the Quobyte CSI welcome page (in raw html format). If encountered\n    error, see if you need to forward your local port to pod.\n\n    NOTE: Depending on your cluster setup (for example, kind clusters), you may need to forward your\n    local port to container to access the nginx pod port. In such case, you could use\n\n    ```bash\n    kubectl port-forward nginx-dynamic-vol 8086:80\n    ```\n\n  and then try `curl localhost:8086`\n\n#### Use existing volumes\n\nQuobyte CSI requires the volume UUID to be passed on to the PV as `VolumeHandle`  \n\n* Quobyte-csi supports both volume name and UUID\n  * **To use Volume Name** `VolumeHandle` should be of the format `\u003cTenant_Name/UUID\u003e|\u003cVolume_Name\u003e`\n   and `nodePublishSecretRef` with Quobyte API login credentials should be specified as shown in the\n   example PV `example/pv-existing-vol.yaml`\n  * **To use Volume UUID** `VolumeHandle` can be `|\u003cVolume_UUID\u003e`.\n\nIn order to use the pre-provisioned `test` volume belonging to the tenant `My Tenant`, user needs to create\n a PV with `volumeHandle: My Tenant|test` as shown in the [example PV](example/pv-existing-vol.yaml).\n\n1. Edit [example/pv-existing-vol.yaml](example/pv-existing-vol.yaml) and point it to the the pre-provisioned volume in Quobyte\n storage through `volumeHandle`. Create the PV with pre-provisioned volume.\n\n    ```bash\n    kubectl create -f example/pv-existing-vol.yaml\n    ```\n\n2. Create a [PVC](example/pvc-existing-vol.yaml) that matches the storage requirements with the above PV (make sure both PV and PVC refer\n to the same storage class). The created PVC will automatically binds to the PV.\n\n    ```bash\n    kubectl create -f example/pvc-existing-vol.yaml\n    ```\n\n3. Create a [pod](example/nginx-demo-pod-with-existing-vol.yaml) referring the PVC as shown in the below example\n\n    ```bash\n    kubectl create -f example/nginx-demo-pod-with-existing-vol.yaml\n    ```\n\n4. Wait for the pod to be in running state\n\n    ```bash\n    kubectl get po -w | grep 'nginx-existing-vol'\n    ```\n\n5. Once the pod is running, copy the [index file](example/index.html) to the deployed nginx pod\n\n    ```bash\n    kubectl cp example/index.html nginx-existing-vol:/tmp\n    kubectl exec -it nginx-existing-vol -- mv /tmp/index.html /usr/share/nginx/html/\n    kubectl exec -it nginx-existing-vol -- chown -R nginx:nginx /usr/share/nginx/html/\n    ```\n\n6. Access the home page served by nginx pod from the command line\n\n    ```bash\n    curl http://$(kubectl get pods nginx-existing-vol -o yaml | grep ' podIP:' | awk '{print $2}'):80\n    ```\n\n    Above command should retrieve the Quobyte CSI welcome page (in raw html format). If encountered\n    error, see if you need to forward your local port to pod.\n\n    NOTE: Depending on your cluster setup (for example, kind clusters), you may need to forward your\n    local port to container to access the nginx pod port. In such case, you could use\n\n    ```bash\n    kubectl port-forward nginx-dynamic-vol 8086:80\n    ```\n\n  and then try `curl localhost:8086`\n\n### Volume snapshots\n\n#### Snapshot Requirements\n\n1. [Quobyte CSI Driver](./quobyte-csi-driver/values.yaml) is deployed with `enableSnapshots: true`\n\n2. [Snapshotter setup](#snapshotter-setup)\n\n##### Dynamic Snapshots\n\n  1. Provision a PVC for a Quobyte volume by following the [instructions](#use-quobyte-volumes-in-kubernetes)\n\n  2. Populate backing volume with [nginx index file](example/index.html)\n\n      ```bash\n      VOLUME=\"\u003cQuobyte-Volume\u003e\" # volume for which snapshot will be taken\n      wget https://raw.githubusercontent.com/quobyte/quobyte-csi/master/example/index.html -P \u003cvalues.clientMountPoint\u003e/mounts/$VOLUME\n      ```\n\n  3. Create [volume snapshot secrets](example/quobyte-admin-credentials.yaml)\n\n     Our examples use same secret in all the places wherever secret is required. Please create and\n     configure secrets as per your requirements.\n\n        ```bash\n        kubectl create -f example/quobyte-admin-credentials.yaml\n        ```\n\n  4. Create volume [snapshot class](example/volume-snapshot-class.yaml)\n\n        ```bash\n        kubectl create -f example/volume-snapshot-class.yaml\n        ```\n\n  5. Create [dynamic volume snapshot](example/volume-snapshot-dynamic-provision.yaml)\n\n        ```bash\n        kubectl create -f example/volume-snapshot-dynamic-provision.yaml\n        ```\n\n     The above command should create required `volumesnapshotcontent` object dynamically\n\n  6. (optional) verify created `volumesnapshot` and `volumesnapshotcontent` objects\n\n        ```bash\n        kubectl get volumesnapshot\n        kubectl get volumesnapshotcontent\n        ```\n\n  7. [Restore snapshot](example/restore-snapshot-pvc-dynamic-provision.yaml) and create PVC\n\n        ```bash\n        kubectl create -f example/restore-snapshot-pvc-dynamic-provision.yaml\n        ```\n\n     This should create a PVC and a PV for the restored snapshot\n\n  8. Create pod with [restored snapshot](example/nginx-demo-pod-with-dynamic-snapshot-vol.yaml)\n\n        ```bash\n        kubectl create -f example/nginx-demo-pod-with-dynamic-snapshot-vol.yaml\n        ```\n\n##### Pre-provisioned Snapshots\n\n  1. Create volume [snapshot class](example/volume-snapshot-class.yaml)\n\n        ```bash\n        kubectl create -f example/volume-snapshot-class.yaml\n        ```\n\n  2. Create volume snapshot secrets\n\n     Our examples use same secret in all the places wherever secret is required.\n      Please create and configure secrets as per your requirements.\n\n      ```bash\n      kubectl create -f example/quobyte-admin-credentials.yaml\n      ```\n\n  3. Create `VolumeSnapshotContent` object for pre-provisioned volume with\n   [required configuration](example/volume-snapshot-content-pre-provisioned.yaml)\n\n        ```bash\n        kubectl create -f example/volume-snapshot-content-pre-provisioned.yaml\n        ```\n\n  4. Create `VolumeSnapshot` object by adjusting the [example snapshot object](example/volume-snapshot-pre-provisioned.yaml)\n\n     **name and namespace must match** `volumeSnapshotRef` **details** from the step 2\n\n        ```bash\n        kubectl create -f example/volume-snapshot-pre-provisioned.yaml\n        ```\n\n  5. (optional) verify created `volumesnapshot` and `volumesnapshotcontent` objects\n\n        ```bash\n        kubectl get volumesnapshot\n        kubectl get volumesnapshotcontent\n        ```\n\n  6. [Restore snapshot](example/restore-snapshot-pvc-pre-provisioned.yaml)\n\n        ```bash\n        kubectl create -f example/restore-snapshot-pvc-pre-provisioned.yaml\n        ```\n\n  7. Create pod with [restored snapshot](example/nginx-demo-pod-with-pre-provisioned-snapshot-vol.yaml)\n\n        ```bash\n        kubectl create -f example/nginx-demo-pod-with-pre-provisioned-snapshot-vol.yaml\n        ```\n\n## Uninstall Quobyte CSI\n\n1. Delete Quobyte CSI containers and corresponding RBAC\n\n    List available helm charts\n\n    ```bash\n    helm list\n    ```\n\n    Delete intended chart\n\n    ```bash\n    helm delete \u003cQuobyte-CSI-chart-name\u003e\n    ```\n\n## Snapshotter Setup\n\n### Install Snapshotter\n\nThe below setup is required once per k8s cluster\n\n  ```bash\n    kubectl create -f quobyte-csi-driver/k8s-snapshot-crd.yaml\n    kubectl create -f quobyte-csi-driver/k8s-snapshot-controller.yaml\n\n  ```\n\n### Remove Snapshotter\n\n  ```bash\n    kubectl delete -f quobyte-csi-driver/k8s-snapshot-controller.yaml\n    kubectl delete -f quobyte-csi-driver/k8s-snapshot-crd.yaml\n\n  ```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquobyte%2Fquobyte-csi-driver","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fquobyte%2Fquobyte-csi-driver","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fquobyte%2Fquobyte-csi-driver/lists"}