{"id":19389273,"url":"https://github.com/bwolf/gluster-k8s-playground","last_synced_at":"2026-04-13T03:36:03.861Z","repository":{"id":75889925,"uuid":"231628636","full_name":"bwolf/gluster-k8s-playground","owner":"bwolf","description":"Playground to experiment with Gluster and Kubernetes","archived":false,"fork":false,"pushed_at":"2020-01-03T16:53:38.000Z","size":6,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-08-25T02:32:44.821Z","etag":null,"topics":["gluster","glusterfs","kubernetes","linux","storage","vagrant"],"latest_commit_sha":null,"homepage":null,"language":"Nix","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bwolf.png","metadata":{"files":{"readme":"README.org","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-01-03T16:52:37.000Z","updated_at":"2020-01-03T16:54:26.000Z","dependencies_parsed_at":"2023-07-11T23:45:59.700Z","dependency_job_id":null,"html_url":"https://github.com/bwolf/gluster-k8s-playground","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bwolf/gluster-k8s-playground","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bwolf%2Fgluster-k8s-playground","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bwolf%2Fgluster-k8s-playground/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bwolf%2Fgluster-k8s-playground/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bwolf%2Fgluster-k8s-playground/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bwolf","download_url":"https://codeload.github.com/bwolf/gluster-k8s-playground/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bwolf%2Fgluster-k8s-playground/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31739050,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-13T03:27:07.512Z","status":"ssl_error","status_checked_at":"2026-04-13T03:26:53.610Z","response_time":93,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gluster","glusterfs","kubernetes","linux","storage","vagrant"],"created_at":"2024-11-10T10:15:46.139Z","updated_at":"2026-04-13T03:36:03.852Z","avatar_url":"https://github.com/bwolf.png","language":"Nix","funding_links":[],"categories":[],"sub_categories":[],"readme":"#+TITLE: Gluster file system with Kubernetes\n#+AUTHOR: Marcus Geiger\n#+DATE: \u003c2019-12-28 Sat\u003e\n#+LANGUAGE: en\n#+OPTIONS: ':nil *:t -:t ::t \u003c:t H:3 \\n:nil ^:t arch:headline author:t\n#+OPTIONS: broken-links:nil c:nil creator:nil d:(not \"LOGBOOK\") date:t e:t\n#+OPTIONS: email:nil f:t inline:t num:t p:nil pri:nil prop:nil stat:t tags:t\n#+OPTIONS: tasks:t tex:t timestamp:t title:t toc:t todo:t |:t\n\nThis project is a playground to experiment with Gluster and Kubernetes. It\nprovides a virtual environment using Vagrant (and VirtualBox) to create three\nstorage VMs (building the storage cluster) and a single Kubernetes instance and\nhow to setup Gluster storage and how to consume it from Kubernetes.\n\n* Bringing up Gluster\n#+BEGIN_SRC sh\n  vagrant up stor1 stor2 stor3\n#+END_SRC\n\nAdd nodes to the Gluster peer:\n#+BEGIN_SRC sh\n  vagrant ssh stor1\n  sudo gluster peer probe stor2\n  sudo gluster peer probe stor3\n#+END_SRC\n\nSetup the Gluster volume using 3 replicas:\n#+BEGIN_SRC sh\n  sudo gluster volume create gv_first replica 3 \\\n       stor1:/data/glusterfs/lv_first/brick \\\n       stor2:/data/glusterfs/lv_first/brick \\\n       stor3:/data/glusterfs/lv_first/brick\n#+END_SRC\n\nGluster prints ~volume create: gv_first: success: please start the volume to access\ndata~ if successful.\n\nDisplay the volume info:\n#+BEGIN_SRC sh\n  sudo gluster volume info\n#+END_SRC\n\n:Output:\n#+BEGIN_EXAMPLE\n  Volume Name: gv_first\n  Type: Replicate\n  Volume ID: f624417f-0cdb-4783-842a-f5a69f0f30b9\n  Status: Created\n  Snapshot Count: 0\n  Number of Bricks: 1 x 3 = 3\n  Transport-type: tcp\n  Bricks:\n  Brick1: stor1:/data/glusterfs/lv_first/brick\n  Brick2: stor2:/data/glusterfs/lv_first/brick\n  Brick3: stor3:/data/glusterfs/lv_first/brick\n  Options Reconfigured:\n  transport.address-family: inet\n  storage.fips-mode-rchecksum: on\n  nfs.disable: on\n  performance.client-io-threads: off\n#+END_EXAMPLE\n:END:\n\nStart the volume:\n#+BEGIN_SRC sh\n  sudo gluster volume start gv_first\n#+END_SRC\n\nGluster prints ~volume start: gv_first: success~ if successful.\n\nAgain show the volume info:\n#+BEGIN_SRC sh\n  sudo gluster volume info\n#+END_SRC\n\n:Output:\n#+BEGIN_EXAMPLE\n  Volume Name: gv_first\n  Type: Replicate\n  Volume ID: f624417f-0cdb-4783-842a-f5a69f0f30b9\n  Status: Started\n  Snapshot Count: 0\n  Number of Bricks: 1 x 3 = 3\n  Transport-type: tcp\n  Bricks:\n  Brick1: stor1:/data/glusterfs/lv_first/brick\n  Brick2: stor2:/data/glusterfs/lv_first/brick\n  Brick3: stor3:/data/glusterfs/lv_first/brick\n  Options Reconfigured:\n  transport.address-family: inet\n  storage.fips-mode-rchecksum: on\n  nfs.disable: on\n  performance.client-io-threads: off\n#+END_EXAMPLE\n:END:\n\nMounting the volume (the server specified in the mount command is used to fetch\nthe cluster configuration and subsequent communication from the client is\nperformed ac cross the whole cluster with fail over):\n#+BEGIN_SRC sh\n  mount -t glusterfs stor1:/gv_first /mnt\n#+END_SRC\n\nNow play around, put and remove files from the mount point and watch the files\ncoming and going in other bricks.\n* Bringing up Kubernetes\n#+BEGIN_SRC sh\n  vagrant ssh kube\n#+END_SRC\n\nPull the images, bootstrap the master for use with /Weave Net/,\n#+BEGIN_SRC sh\n  sudo kubeadm config images pull\n  sudo kubeadm init --apiserver-advertise-address=192.168.202.245\n#+END_SRC\n\nCopy credentials to the regular user account:\n#+BEGIN_SRC sh\n  mkdir -p $HOME/.kube\n  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config\n  sudo chown $(id -u):$(id -g) $HOME/.kube/config\n#+END_SRC\n\nInstall the /Weave Net/ pod network add-on:\n#+BEGIN_SRC sh\n  kubectl apply -f \"https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\\n')\"\n#+END_SRC\n\nTaint the (master) node:\n#+BEGIN_SRC sh\n  kubectl taint nodes --all node-role.kubernetes.io/master-\n#+END_SRC\n* Using Gluster from Kubernetes\nFollowing this [[https://github.com/kubernetes/examples/tree/master/volumes/glusterfs][guide]] , the prerequisite for mounting Gluster volumes, is to have\nthe ~glusterfs-client~ package installed on the Kubernetes nodes (already done\nvia Ansible playbooks).\n\ncreate the file ~glusterfs-endpoints.json~ in the VM ~kube~:\n:glusterfs-endpoints:\n#+BEGIN_SRC json\n  {\n    \"kind\": \"Endpoints\",\n    \"apiVersion\": \"v1\",\n    \"metadata\": {\n      \"name\": \"glusterfs-cluster\"\n    },\n    \"subsets\": [\n      {\n        \"addresses\": [\n          {\n            \"ip\": \"192.168.202.201\"\n          }\n        ],\n        \"ports\": [\n          {\n            \"port\": 1\n          }\n        ]\n      },\n      {\n        \"addresses\": [\n          {\n            \"ip\": \"192.168.202.202\"\n          }\n        ],\n        \"ports\": [\n          {\n            \"port\": 1\n          }\n        ]\n      },\n      {\n        \"addresses\": [\n          {\n            \"ip\": \"192.168.202.203\"\n          }\n        ],\n        \"ports\": [\n          {\n            \"port\": 1\n          }\n        ]\n      }\n    ]\n  }\n#+END_SRC\n:END:\n\nApply the Gluster definition to Kubernetes:\n#+BEGIN_SRC sh\n  kubectl create -f glusterfs-endpoints.json\n#+END_SRC\n\nVerify the endpoints:\n#+BEGIN_SRC sh\n  kubectl get endpoints\n#+END_SRC\n\nCreate the file ~glusterfs-service.json~ in the VM ~kube~:\n:glusterfs-service:\n#+BEGIN_SRC json\n  {\n    \"kind\": \"Service\",\n    \"apiVersion\": \"v1\",\n    \"metadata\": {\n      \"name\": \"glusterfs-cluster\"\n    },\n    \"spec\": {\n      \"ports\": [\n        {\"port\": 1}\n      ]\n    }\n  }\n#+END_SRC\n:END:\n\nCreate a service for these endpoints, so that they will persist:\n#+BEGIN_SRC sh\n  kubectl create -f glusterfs-service.json\n#+END_SRC\n\nCreate the file ~demo-pod.yaml~ in the VM ~kube~ to demonstrate how to consume a\nGluster volume in a POD:\n:demo-pod:\n#+BEGIN_SRC yaml\n  ---\n  apiVersion: v1\n  kind: Pod\n  metadata:\n    name: demo-pod\n  spec:\n    containers:\n    - name: demo\n      image: nginx\n      volumeMounts:\n      - mountPath: \"/mnt/glusterfs\"\n        name: glusterfsvol\n    volumes:\n    - name: glusterfsvol\n      glusterfs:\n        endpoints: glusterfs-cluster\n        path: gv_first\n        readOnly: true\n#+END_SRC\n:END:\n\nVerify the volume is mounted:\n#+BEGIN_SRC sh\n  kubectl exec demo-pod -- mount | grep gluster\n#+END_SRC\n\n:Output:\n#+BEGIN_EXAMPLE\n  192.168.202.201:gv_first on /mnt/glusterfs type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)\n#+END_EXAMPLE\n:END:\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbwolf%2Fgluster-k8s-playground","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbwolf%2Fgluster-k8s-playground","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbwolf%2Fgluster-k8s-playground/lists"}