{"id":13583605,"url":"https://github.com/democratic-csi/democratic-csi","last_synced_at":"2025-05-14T20:09:15.679Z","repository":{"id":38240329,"uuid":"223211268","full_name":"democratic-csi/democratic-csi","owner":"democratic-csi","description":"csi storage for container orchestration systems","archived":false,"fork":false,"pushed_at":"2025-04-06T17:02:49.000Z","size":1450,"stargazers_count":1010,"open_issues_count":60,"forks_count":88,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-13T17:46:41.921Z","etag":null,"topics":["centos","csi","csi-driver","debian","freebsd","freenas","iscsi","k8s","kubernetes","nfs","rhel","storage","truenas","ubuntu","zfs","zfsonlinux"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/democratic-csi.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-11-21T15:59:16.000Z","updated_at":"2025-04-13T12:01:24.000Z","dependencies_parsed_at":"2022-07-11T05:46:09.480Z","dependency_job_id":"37f3c0a0-8f23-4ecf-9d29-f1e972866b41","html_url":"https://github.com/democratic-csi/democratic-csi","commit_stats":null,"previous_names":[],"tags_count":42,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/democratic-csi%2Fdemocratic-csi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/democratic-csi%2Fdemocratic-csi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/democratic-csi%2Fdemocratic-csi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/democratic-csi%2Fdemocratic-csi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/democratic-csi","download_url":"https://codeload.github.com/democratic-csi/democratic-csi/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254219374,"owners_count":22034397,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["centos","csi","csi-driver","debian","freebsd","freenas","iscsi","k8s","kubernetes","nfs","rhel","storage","truenas","ubuntu","zfs","zfsonlinux"],"created_at":"2024-08-01T15:03:37.919Z","updated_at":"2025-05-14T20:09:15.659Z","avatar_url":"https://github.com/democratic-csi.png","language":"JavaScript","funding_links":[],"categories":["JavaScript","ubuntu"],"sub_categories":[],"readme":"![Image](https://img.shields.io/docker/pulls/democraticcsi/democratic-csi.svg)\n![Image](https://img.shields.io/github/actions/workflow/status/democratic-csi/democratic-csi/main.yml?branch=master\u0026style=flat-square)\n[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/democratic-csi)](https://artifacthub.io/packages/search?repo=democratic-csi)\n\n# Introduction\n\n`democratic-csi` implements the `csi` (container storage interface) spec\nproviding storage for various container orchestration systems (ie: Kubernetes).\n\nThe current focus is providing storage via iscsi/nfs from zfs-based storage\nsystems, predominantly `FreeNAS / TrueNAS` and `ZoL` on `Ubuntu`.\n\nThe current drivers implement the depth and breadth of the `csi` spec, so you\nhave access to resizing, snapshots, clones, etc functionality.\n\n`democratic-csi` is 2 things:\n\n- several implementations of `csi` drivers\n  - `freenas-nfs` (manages zfs datasets to share over nfs)\n  - `freenas-iscsi` (manages zfs zvols to share over iscsi)\n  - `freenas-smb` (manages zfs datasets to share over smb)\n  - `freenas-api-nfs` experimental use with SCALE only (manages zfs datasets to share over nfs)\n  - `freenas-api-iscsi` experimental use with SCALE only (manages zfs zvols to share over iscsi)\n  - `freenas-api-smb` experimental use with SCALE only (manages zfs datasets to share over smb)\n  - `zfs-generic-nfs` (works with any ZoL installation...ie: Ubuntu)\n  - `zfs-generic-iscsi` (works with any ZoL installation...ie: Ubuntu)\n  - `zfs-generic-smb` (works with any ZoL installation...ie: Ubuntu)\n  - `zfs-generic-nvmeof` (works with any ZoL installation...ie: Ubuntu)\n  - `zfs-local-ephemeral-inline` (provisions node-local zfs datasets)\n  - `zfs-local-dataset` (provision node-local volume as dataset)\n  - `zfs-local-zvol` (provision node-local volume as zvol)\n  - `synology-iscsi` experimental (manages volumes to share over iscsi)\n  - `objectivefs` (manages objectivefs volumes)\n  - `lustre-client` (crudely provisions storage using a shared lustre\n    share/directory for all volumes)\n  - `nfs-client` (crudely provisions storage using a shared nfs share/directory\n    for all volumes)\n  - `smb-client` (crudely provisions storage using a shared smb share/directory\n    for all volumes)\n  - `local-hostpath` (crudely provisions node-local directories)\n  - `node-manual` (allows connecting to manually created smb, nfs, lustre,\n    oneclient, nvmeof, and iscsi volumes, see sample PVs in the `examples`\n    directory)\n- framework for developing `csi` drivers\n\nIf you have any interest in providing a `csi` driver, simply open an issue to\ndiscuss. The project provides an extensive framework to build from making it\nrelatively easy to implement new drivers.\n\n# Installation\n\nPredominantly 3 things are needed:\n\n- node prep (ie: your kubernetes cluster nodes)\n- server prep (ie: your storage server)\n- deploy the driver into the cluster (`helm` chart provided with sample\n  `values.yaml`)\n\n## Community Guides\n\n- https://jonathangazeley.com/2021/01/05/using-truenas-to-provide-persistent-storage-for-kubernetes/\n- https://www.lisenet.com/2021/moving-to-truenas-and-democratic-csi-for-kubernetes-persistent-storage/\n- https://gist.github.com/admun/4372899f20421a947b7544e5fc9f9117 (migrating\n  from `nfs-client-provisioner` to `democratic-csi`)\n- https://gist.github.com/deefdragon/d58a4210622ff64088bd62a5d8a4e8cc\n  (migrating between storage classes using `velero`)\n- https://github.com/fenio/k8s-truenas (NFS/iSCSI over API with TrueNAS Scale)\n\n## Node Prep\n\nYou should install/configure the requirements for both nfs and iscsi.\n\n### cifs\n\n```bash\n# RHEL / CentOS\nsudo yum install -y cifs-utils\n\n# Ubuntu / Debian\nsudo apt-get install -y cifs-utils\n```\n\n### nfs\n\n```bash\n# RHEL / CentOS\nsudo yum install -y nfs-utils\n\n# Ubuntu / Debian\nsudo apt-get install -y nfs-common\n```\n\n### iscsi\n\nNote that `multipath` is supported for the `iscsi`-based drivers. Simply setup\nmultipath to your liking and set multiple portals in the config as appropriate.\n\nIf you are running Kubernetes with rancher/rke please see the following:\n\n- https://github.com/rancher/rke/issues/1846\n\n#### RHEL / CentOS\n\n```bash\n# Install the following system packages\nsudo yum install -y lsscsi iscsi-initiator-utils sg3_utils device-mapper-multipath\n\n# Enable multipathing\nsudo mpathconf --enable --with_multipathd y\n\n# Ensure that iscsid and multipathd are running\nsudo systemctl enable iscsid multipathd\nsudo systemctl start iscsid multipathd\n\n# Start and enable iscsi\nsudo systemctl enable iscsi\nsudo systemctl start iscsi\n```\n\n#### Ubuntu / Debian\n\n```\n# Install the following system packages\nsudo apt-get install -y open-iscsi lsscsi sg3-utils multipath-tools scsitools\n\n# Enable multipathing\nsudo tee /etc/multipath.conf \u003c\u003c-'EOF'\ndefaults {\n    user_friendly_names yes\n    find_multipaths yes\n}\nEOF\n\nsudo systemctl enable multipath-tools.service\nsudo service multipath-tools restart\n\n# Ensure that open-iscsi and multipath-tools are enabled and running\nsudo systemctl status multipath-tools\nsudo systemctl enable open-iscsi.service\nsudo service open-iscsi start\nsudo systemctl status open-iscsi\n```\n\n#### [Talos](https://www.talos.dev/)\n\nTo use iscsi storage in kubernetes cluster in talos these steps are needed which are similar to the ones explained in https://www.talos.dev/v1.1/kubernetes-guides/configuration/replicated-local-storage-with-openebs-jiva/#patching-the-jiva-installation\n\n##### Patch nodes\n\nsince talos does not have iscsi support by default, the iscsi extension is needed\ncreate a `patch.yaml` file with\n\n```yaml\n- op: add\n  path: /machine/install/extensions\n  value:\n    - image: ghcr.io/siderolabs/iscsi-tools:v0.1.1\n```\n\nand apply the patch across all of your nodes\n\n```bash\ntalosctl -e \u003cendpoint ip/hostname\u003e -n \u003cnode ip/hostname\u003e patch mc -p @patch.yaml\n```\n\nthe extension will not activate until you \"upgrade\" the nodes, even if there is no update, use the latest version of talos installer.\nVERIFY THE TALOS VERSION IN THIS COMMAND BEFORE RUNNING IT AND READ THE [OpenEBS Jiva](https://www.talos.dev/v1.1/kubernetes-guides/configuration/replicated-local-storage-with-openebs-jiva/#patching-the-jiva-installation).\nupgrade all of the nodes in the cluster to get the extension\n\n```bash\ntalosctl -e \u003cendpoint ip/hostname\u003e -n \u003cnode ip/hostname\u003e upgrade --image=ghcr.io/siderolabs/installer:v1.1.1\n```\n\nin your `values.yaml` file make sure to enable these settings\n\n```yaml\nnode:\n  hostPID: true\n  driver:\n    extraEnv:\n      - name: ISCSIADM_HOST_STRATEGY\n        value: nsenter\n      - name: ISCSIADM_HOST_PATH\n        value: /usr/local/sbin/iscsiadm\n    iscsiDirHostPath: /usr/local/etc/iscsi\n    iscsiDirHostPathType: \"\"\n```\n\nand continue your democratic installation as usuall with other iscsi drivers.\n\n#### Privileged Namespace\n\ndemocratic-csi requires privileged access to the nodes, so the namespace should allow for privileged pods. One way of doing it is via [namespace labels](https://kubernetes.io/docs/tasks/configure-pod-container/enforce-standards-namespace-labels/).\nAdd the followin label to the democratic-csi installation namespace `pod-security.kubernetes.io/enforce=privileged`\n\n```\nkubectl label --overwrite namespace democratic-csi pod-security.kubernetes.io/enforce=privileged\n```\n\n### nvmeof\n\n```bash\n# not required but likely helpful (tools are included in the democratic images\n# so not needed on the host)\napt-get install -y nvme-cli\n\n# get the nvme fabric modules\napt-get install linux-generic\n\n# ensure the nvmeof modules get loaded at boot\ncat \u003c\u003cEOF \u003e /etc/modules-load.d/nvme.conf\nnvme\nnvme-tcp\nnvme-fc\nnvme-rdma\nEOF\n\n# load the modules immediately\nmodprobe nvme\nmodprobe nvme-tcp\nmodprobe nvme-fc\nmodprobe nvme-rdma\n\n# nvme has native multipath or can use DM multipath\n# democratic-csi will gracefully handle either configuration\n# RedHat recommends DM multipath (nvme_core.multipath=N)\ncat /sys/module/nvme_core/parameters/multipath\n\n# kernel arg to enable/disable native multipath\nnvme_core.multipath=N\n```\n\n### zfs-local-ephemeral-inline\n\nThis `driver` provisions node-local ephemeral storage on a per-pod basis. Each\nnode should have an identically named zfs pool created and avaialble to the\n`driver`. Note, this is _NOT_ the same thing as using the docker zfs storage\ndriver (although the same pool could be used). No other requirements are\nnecessary.\n\n- https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190122-csi-inline-volumes.md\n- https://kubernetes-csi.github.io/docs/ephemeral-local-volumes.html\n\n### zfs-local-{dataset,zvol}\n\nThis `driver` provisions node-local storage. Each node should have an\nidentically named zfs pool created and avaialble to the `driver`. Note, this is\n_NOT_ the same thing as using the docker zfs storage driver (although the same\npool could be used). Nodes should have the standard `zfs` utilities installed.\n\nIn the name of ease-of-use these drivers by default report `MULTI_NODE` support\n(`ReadWriteMany` in k8s) however the volumes will implicity only work on the\nnode where originally provisioned. Topology contraints manage this in an\nautomated fashion preventing any undesirable behavior. So while you may\nprovision `MULTI_NODE` / `RWX` volumes, any workloads using the volume will\nalways land on a single node and that node will always be the node where the\nvolume is/was provisioned.\n\n### local-hostpath\n\nThis `driver` provisions node-local storage. Each node should have an\nidentically name folder where volumes will be created.\n\nIn the name of ease-of-use these drivers by default report `MULTI_NODE` support\n(`ReadWriteMany` in k8s) however the volumes will implicity only work on the\nnode where originally provisioned. Topology contraints manage this in an\nautomated fashion preventing any undesirable behavior. So while you may\nprovision `MULTI_NODE` / `RWX` volumes, any workloads using the volume will\nalways land on a single node and that node will always be the node where the\nvolume is/was provisioned.\n\nThe nature of this `driver` also prevents the enforcement of quotas. In short\nthe requested volume size is generally ignored.\n\n### windows\n\nSupport for Windows was introduced in `v1.7.0`. Currently support is limited\nto kubernetes nodes capabale of running `HostProcess` containers. Support was\ntested against `Windows Server 2019` using `rke2-v1.24`. Currently any of the\n`-smb` and `-iscsi` drivers will work. Support for `ntfs` was added to the\nlinux nodes as well (using the `ntfs3` driver) so volumes created can be\nutilized by nodes with either operating system (in the case of `cifs` by both\nsimultaneously).\n\nIf using any `-iscsi` driver be sure your iqns are always fully lower-case by\ndefault (https://github.com/PowerShell/PowerShell/issues/17306).\n\nDue to current limits in the kubernetes tooling it is not possible to use the\n`local-hostpath` driver but support is implemented in this project and will\nwork as soon as kubernetes support is available.\n\n```powershell\n# ensure all updates are installed\n\n# enable the container feature\nEnable-WindowsOptionalFeature -Online -FeatureName Containers –All\n\n# install a HostProcess compatible kubernetes\n\n# smb support\n# If using with Windows based machines you may need to enable guest access\n# (even if you are connecting with credentials)\nSet-ItemProperty HKLM:\\SYSTEM\\CurrentControlSet\\Services\\LanmanWorkstation\\Parameters AllowInsecureGuestAuth -Value 1\nRestart-Service LanmanWorkstation -Force\n\n# iscsi\n# enable iscsi service and mpio as appropriate\nGet-Service -Name MSiSCSI\nSet-Service -Name MSiSCSI -StartupType Automatic\nStart-Service -Name MSiSCSI\nGet-Service -Name MSiSCSI\n\n# mpio\nGet-WindowsFeature -Name 'Multipath-IO'\nAdd-WindowsFeature -Name 'Multipath-IO'\n\nEnable-MSDSMAutomaticClaim -BusType \"iSCSI\"\nDisable-MSDSMAutomaticClaim -BusType \"iSCSI\"\n\nGet-MSDSMGlobalDefaultLoadBalancePolicy\nSet-MSDSMGlobalLoadBalancePolicy -Policy RR\n```\n\n- https://kubernetes.io/blog/2021/08/16/windows-hostprocess-containers/\n- https://kubernetes.io/docs/tasks/configure-pod-container/create-hostprocess-pod/\n\n## Server Prep\n\nServer preparation depends slightly on which `driver` you are using.\n\n### FreeNAS (freenas-nfs, freenas-iscsi, freenas-smb, freenas-api-nfs, freenas-api-iscsi, freenas-api-smb)\n\nThe recommended version of FreeNAS is 12.0-U2+, however the driver should work\nwith much older versions as well.\n\nThe various `freenas-api-*` drivers are currently EXPERIMENTAL and can only be\nused with SCALE 21.08+. Fundamentally these drivers remove the need for `ssh`\nconnections and do all operations entirely with the TrueNAS api. With that in\nmind, any ssh/shell/etc requirements below can be safely ignored. The minimum\nvolume size through the api is `1G` so beware that requested volumes with a\nsize small will be increased to `1G`. Also note the following known issues:\n\n- https://jira.ixsystems.com/browse/NAS-111870\n- https://github.com/democratic-csi/democratic-csi/issues/112\n- https://github.com/democratic-csi/democratic-csi/issues/101\n\nEnsure the following services are configurged and running:\n\n- ssh (if you use a password for authentication make sure it is allowed)\n  - https://www.truenas.com/community/threads/ssh-access-ssh-rsa-not-in-pubkeyacceptedalgorithms.101715/\n  - `PubkeyAcceptedAlgorithms +ssh-rsa`\n- ensure `zsh`, `bash`, or `sh` is set as the root shell, `csh` gives false errors due to quoting\n- nfs\n- iscsi\n\n  - (fixed in 12.0-U2+) when using the FreeNAS API concurrently the\n    `/etc/ctl.conf` file on the server can become invalid, some sample scripts\n    are provided in the `contrib` directory to clean things up ie: copy the\n    script to the server and directly and run - `./ctld-config-watchdog-db.sh | logger -t ctld-config-watchdog-db.sh \u0026`\n    please read the scripts and set the variables as appropriate for your server.\n  - ensure you have pre-emptively created portals, initatior groups, auths\n    - make note of the respective IDs (the true ID may not reflect what is\n      visible in the UI)\n    - IDs can be visible by clicking the the `Edit` link and finding the ID in the\n      browser address bar\n    - Optionally you may use the following to retrieve appropiate IDs:\n      - `curl --header \"Accept: application/json\" --user root:\u003cpassword\u003e 'http(s)://\u003cip\u003e/api/v2.0/iscsi/portal'`\n      - `curl --header \"Accept: application/json\" --user root:\u003cpassword\u003e 'http(s)://\u003cip\u003e/api/v2.0/iscsi/initiator'`\n      - `curl --header \"Accept: application/json\" --user root:\u003cpassword\u003e 'http(s)://\u003cip\u003e/api/v2.0/iscsi/auth'`\n  - The maximum number of volumes is limited to 255 by default on FreeBSD (physical devices such as disks and CD-ROM drives count against this value).\n    Be sure to properly adjust both [tunables](https://www.freebsd.org/cgi/man.cgi?query=ctl\u0026sektion=4#end) `kern.cam.ctl.max_ports` and `kern.cam.ctl.max_luns` to avoid running out of resources when dynamically provisioning iSCSI volumes on FreeNAS or TrueNAS Core.\n\n- smb\n\nIf you would prefer you can configure `democratic-csi` to use a\nnon-`root` user when connecting to the FreeNAS server:\n\n- Create a non-`root` user (e.g., `csi`)\n\n- Ensure that user has passwordless `sudo` privileges:\n\n  ```\n  csi ALL=(ALL) NOPASSWD:ALL\n\n  # if on CORE 12.0-u3+ you should be able to do the following\n  # which will ensure it does not get reset during reboots etc\n  # at the command prompt\n  cli\n\n  # after you enter the truenas cli and are at that prompt\n  account user query select=id,username,uid,sudo_nopasswd\n\n  # find the `id` of the user you want to update (note, this is distinct from the `uid`)\n  account user update id=\u003cid\u003e sudo=true\n  account user update id=\u003cid\u003e sudo_nopasswd=true\n  # optional if you want to disable password\n  #account user update id=\u003cid\u003e password_disabled=true\n\n  # exit cli by hitting ctrl-d\n\n  # confirm sudoers file is appropriate\n  cat /usr/local/etc/sudoers\n  ```\n\n  (note this can get reset by FreeNAS if you alter the user via the\n  GUI later)\n\n- Instruct `democratic-csi` to use `sudo` by adding the following to\n  your driver configuration:\n\n  ```\n  zfs:\n    cli:\n      sudoEnabled: true\n  ```\n\nStarting with TrueNAS CORE 12 it is also possible to use an `apiKey` instead of\nthe `root` password for the http connection.\n\nIssues to review:\n\n- https://jira.ixsystems.com/browse/NAS-108519\n- https://jira.ixsystems.com/browse/NAS-108520\n- https://jira.ixsystems.com/browse/NAS-108521\n- https://jira.ixsystems.com/browse/NAS-108522\n- https://jira.ixsystems.com/browse/NAS-107219\n\n### ZoL (zfs-generic-nfs, zfs-generic-iscsi, zfs-generic-smb, zfs-generic-nvmeof)\n\nEnsure ssh and zfs is installed on the nfs/iscsi server and that you have installed\n`targetcli`.\n\nThe driver executes many commands over an ssh connection. You may consider\ndisabling all the `motd` details for the ssh user as it can spike the cpu\nunecessarily:\n\n- https://askubuntu.com/questions/318592/how-can-i-remove-the-landscape-canonical-com-greeting-from-motd\n- https://linuxconfig.org/disable-dynamic-motd-and-news-on-ubuntu-20-04-focal-fossa-linux\n- https://github.com/democratic-csi/democratic-csi/issues/151 (some notes on\n  using delegated zfs permissions)\n\n```bash\n####### nfs\nyum install -y nfs-utils\nsystemctl enable --now nfs-server.service\n\napt-get install -y nfs-kernel-server\nsystemctl enable --now nfs-kernel-server.service\n\n####### iscsi\nyum install targetcli -y\napt-get -y install targetcli-fb\n\n####### smb\napt-get install -y samba smbclient\n\n# create posix user\ngroupadd -g 1001 smbroot\nuseradd -u 1001 -g 1001 -M -N -s /sbin/nologin smbroot\npasswd smbroot (optional)\n\n# create smb user and set password\nsmbpasswd -L -a smbroot\n\n####### nvmeof\n# ensure nvmeof target modules are loaded at startup\ncat \u003c\u003cEOF \u003e /etc/modules-load.d/nvmet.conf\nnvmet\nnvmet-tcp\nnvmet-fc\nnvmet-rdma\nEOF\n\n# load the modules immediately\nmodprobe nvmet\nmodprobe nvmet-tcp\nmodprobe nvmet-fc\nmodprobe nvmet-rdma\n\n# install nvmetcli and systemd services\ngit clone git://git.infradead.org/users/hch/nvmetcli.git\ncd nvmetcli\n\n## install globally\npython3 setup.py install --prefix=/usr\npip install configshell_fb\n\n## install to root home dir\npython3 setup.py install --user\npip install configshell_fb --user\n\n# prevent log files from filling up disk\nln -sf /dev/null ~/.nvmetcli/log.txt\nln -sf /dev/null ~/.nvmetcli/history.txt\n\n# install systemd unit and enable/start\n## optionally to ensure the config file is loaded before we start\n## reading/writing to it add an ExecStartPost= to the unit file\n##\n## ExecStartPost=/usr/bin/touch /var/run/nvmet-config-loaded\n##\n## in your dirver config set nvmeof.shareStrategyNvmetCli.configIsImportedFilePath=/var/run/nvmet-config-loaded\n## which will prevent the driver from making any changes until the configured\n## file is present\nvi nvmet.service\n\ncp nvmet.service /etc/systemd/system/\nmkdir -p /etc/nvmet\nsystemctl daemon-reload\nsystemctl enable --now nvmet.service\nsystemctl status nvmet.service\n\n# create the port(s) configuration manually\necho \"\ncd /\nls\n\" | nvmetcli\n\n# do this multiple times altering as appropriate if you have/want multipath\n# change the port to 2, 3.. each additional path\n# the below example creates a tcp port listening on all IPs on port 4420\necho \"\ncd /ports\ncreate 1\ncd 1\nset addr adrfam=ipv4 trtype=tcp traddr=0.0.0.0 trsvcid=4420\n\nsaveconfig /etc/nvmet/config.json\n\" | nvmetcli\n\n# if running TrueNAS SCALE you can skip the above and simply copy\n# contrib/scale-nvmet-start.sh to your machine and add it as a startup script\n# to launch POSTINIT type COMMAND\n# and then create the port(s) as mentioned above\n```\n\n### Synology (synology-iscsi)\n\nEnsure iscsi manager has been installed and is generally setup/configured. DSM 6.3+ is supported.\n\n### objectivefs (objectivefs)\n\nObjectiveFS requires the use of an _Admin Key_ to properly automate the\nlifecycle of filesystems. Each deployment of the driver will point to a single\n`pool` (bucket) and create individual `filesystems` within that bucket\non-demand.\n\nEnsure the config value used for `pool` is an existing bucket. Be sure the\nbucket is _NOT_ being used in fs mode (ie: the whole bucket is a single fs).\n\nThe `democratic-csi` `node` container will host the fuse mount process so\nbe careful to only upgrade when all relevant workloads have been drained from\nthe respective node. Also beware that any cpu/memory limits placed on the\ncontainer by the orchestration system will impact any ability to use the\ncaching, etc features of objectivefs.\n\n- https://objectivefs.com/howto/csi-driver-objectivefs\n- https://objectivefs.com/howto/csi-driver-objectivefs-kubernetes-managed\n- https://objectivefs.com/howto/objectivefs-admin-key-setup\n- https://objectivefs.com/features#filesystem-pool\n- https://objectivefs.com/howto/how-to-create-a-filesystem-with-an-existing-empty-bucket\n\n## Helm Installation\n\n```bash\nhelm repo add democratic-csi https://democratic-csi.github.io/charts/\nhelm repo update\n# helm v2\nhelm search democratic-csi/\n\n# helm v3\nhelm search repo democratic-csi/\n\n# copy proper values file from https://github.com/democratic-csi/charts/tree/master/stable/democratic-csi/examples\n# edit as appropriate\n# examples are from helm v2, alter as appropriate for v3\n\n# add --create-namespace for helm v3\nhelm upgrade \\\n--install \\\n--values freenas-iscsi.yaml \\\n--namespace democratic-csi \\\nzfs-iscsi democratic-csi/democratic-csi\n\nhelm upgrade \\\n--install \\\n--values freenas-nfs.yaml \\\n--namespace democratic-csi \\\nzfs-nfs democratic-csi/democratic-csi\n```\n\n### A note on non standard kubelet paths\n\nSome distrobutions, such as `minikube` and `microk8s` use a non-standard\nkubelet path. In such cases it is necessary to provide a new kubelet host path,\nmicrok8s example below:\n\n```bash\nmicrok8s helm upgrade \\\n  --install \\\n  --values freenas-nfs.yaml \\\n  --set node.kubeletHostPath=\"/var/snap/microk8s/common/var/lib/kubelet\"  \\\n  --namespace democratic-csi \\\n  zfs-nfs democratic-csi/democratic-csi\n```\n\n- microk8s - `/var/snap/microk8s/common/var/lib/kubelet`\n- pivotal - `/var/vcap/data/kubelet`\n- k0s - `/var/lib/k0s/kubelet`\n\n### openshift\n\n`democratic-csi` generally works fine with openshift. Some special parameters\nneed to be set with helm (support added in chart version `0.6.1`):\n\n```bash\n# for sure required\n--set node.rbac.openshift.privileged=true\n--set node.driver.localtimeHostPath=false\n\n# unlikely, but in special circumstances may be required\n--set controller.rbac.openshift.privileged=true\n```\n\n### Nomad\n\n`democratic-csi` works with Nomad in a functioning but limted capacity. See the\n[Nomad docs](docs/nomad.md) for details.\n\n### Docker Swarm\n\n- https://github.com/moby/moby/blob/master/docs/cluster_volumes.md\n- https://github.com/olljanat/csi-plugins-for-docker-swarm\n\n## Multiple Deployments\n\nYou may install multiple deployments of each/any driver. It requires the\nfollowing:\n\n- Use a new helm release name for each deployment\n- Make sure you have a unique `csiDriver.name` in the values file (within the\n  same cluster)\n- Use unqiue names for your storage classes (per cluster)\n- Use a unique parent dataset (ie: don't try to use the same parent across\n  deployments or clusters)\n- For `iscsi` and `smb` be aware that the names of assets/shares are _global_\n  and so collisions are possible/probable. Appropriate use of the respective\n  `nameTemplate`, `namePrefix`, and `nameSuffix` configuration options will\n  mitigate the issue [#210](https://github.com/democratic-csi/democratic-csi/issues/210).\n\n# Snapshot Support\n\nInstall snapshot controller (once per cluster):\n\n- https://github.com/democratic-csi/charts/tree/master/stable/snapshot-controller\n\nOR\n\n- https://github.com/kubernetes-csi/external-snapshotter/tree/master/client/config/crd\n- https://github.com/kubernetes-csi/external-snapshotter/tree/master/deploy/kubernetes/snapshot-controller\n\nInstall `democratic-csi` as usual with `volumeSnapshotClasses` defined as appropriate.\n\n- https://kubernetes.io/docs/concepts/storage/volume-snapshots/\n- https://github.com/kubernetes-csi/external-snapshotter#usage\n- https://github.com/democratic-csi/democratic-csi/issues/129#issuecomment-961489810\n\n# Migrating from freenas-provisioner and freenas-iscsi-provisioner\n\nIt is possible to migrate all volumes from the non-csi freenas provisioners\nto `democratic-csi`.\n\nCopy the `contrib/freenas-provisioner-to-democratic-csi.sh` script from the\nproject to your workstation, read the script in detail, and edit the variables\nto your needs to start migrating!\n\n# Related\n\n- https://github.com/nmaupu/freenas-provisioner\n- https://github.com/travisghansen/freenas-iscsi-provisioner\n- https://datamattsson.tumblr.com/post/624751011659202560/welcome-truenas-core-container-storage-provider\n- https://github.com/dravanet/truenas-csi\n- https://github.com/SynologyOpenSource/synology-csi\n- https://github.com/openebs/zfs-localpv\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdemocratic-csi%2Fdemocratic-csi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdemocratic-csi%2Fdemocratic-csi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdemocratic-csi%2Fdemocratic-csi/lists"}