{"id":20181934,"url":"https://github.com/devteds/swarm-portworx-consul","last_synced_at":"2025-04-10T05:11:00.758Z","repository":{"id":132073111,"uuid":"125464202","full_name":"devteds/swarm-portworx-consul","owner":"devteds","description":"Test stateful containers (mysql database) using portworx with consul on docker swarm","archived":false,"fork":false,"pushed_at":"2018-03-17T01:16:25.000Z","size":9,"stargazers_count":5,"open_issues_count":0,"forks_count":3,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-24T06:22:36.495Z","etag":null,"topics":["consul","consul-kv","digitalocean","docker","docker-swarm","mysql","portworx","rails","stateful-containers"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devteds.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-16T04:37:39.000Z","updated_at":"2023-01-31T22:44:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"52d79568-adcf-49d8-a2df-81804059e711","html_url":"https://github.com/devteds/swarm-portworx-consul","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devteds%2Fswarm-portworx-consul","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devteds%2Fswarm-portworx-consul/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devteds%2Fswarm-portworx-consul/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devteds%2Fswarm-portworx-consul/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devteds","download_url":"https://codeload.github.com/devteds/swarm-portworx-consul/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248161276,"owners_count":21057555,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consul","consul-kv","digitalocean","docker","docker-swarm","mysql","portworx","rails","stateful-containers"],"created_at":"2024-11-14T02:37:10.187Z","updated_at":"2025-04-10T05:11:00.744Z","avatar_url":"https://github.com/devteds.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003e Find more examples and short videos on development \u0026 deployments with docker, aws etc on [devteds.com](https://devteds.com)\n\n# Stateful Containers: Docker Swarm + Portworx + Consul\n\n- Create Swarm Cluster of 3 nodes on digitalocean\n- Run Consul Cluster on Swarm\n- Make 2 of 3 nodes the data nodes (label data=true)\n- Attach volumes (block storage) to data nodes\n- Install and setup portworx cluster for data nodes; Use consul KV store\n- Create volume and run app services (api app \u0026 database)\n- Test MySQL state by toggling the data nodes on swarm cluster\n\n*Tested on docker version Docker version 17.12.0-ce, build c97c6d6*\n\n## Swarm Cluster\n\nCreate swarm cluster of those 3 nodes on digitalocean. Optionally, edit `swarm.sh` to change DigitalOcean droplet configs.\n\n- node-1 manager\n- node-2 worker, labelled (data=true)\n- node-3 worker, labelled (data=true)\n\n```\nexport DO_TOKEN=\u003cDigitalOcean API Access Token\u003e\n./swarm.sh\n```\n\n\n## Consul Cluster\n\nRefer https://github.com/devteds/swarm-consul\n\n```\neval $(docker-machine env node-1)\ndocker stack deploy -c consul.yml kv\n```\n\nVerify the stack \u0026 agents/containers. There may be some errors before all the nodes in the consul cluster start up and complete leader election. Give it about 30 seconds to gossip and complete leader election.\n\n```\ndocker stack ps -f \"desired-state=running\" kv\ndocker service logs kv_server-bootstrap\ndocker service logs kv_server\ndocker service logs kv_client\ndocker service inspect kv_server-bootstrap\n```\n\nConsul UI \u0026 Catalog\n\n```\nopen http://$(docker-machine ip node-1):8500/ui\ncurl http://$(docker-machine ip node-1):8500/v1/catalog/datacenters\ncurl http://$(docker-machine ip node-1):8500/v1/catalog/nodes\n```\n\nTest Consul CLI - https://github.com/devteds/swarm-consul/blob/master/README.md#test-consul-cli\n\n## Cloud (Digital Ocean)\n\n- Add volumes of 5G to the data nodes (node-2 \u0026 node-3)\n  - Login to DO \u003e Droplets \u003e node-2|node-3 \u003e Volumes \u003e Add Volume \u003e 5GB \u003e Create Volume\n- Test new volume mount on the nodes using `lsblk`\n- New volumes should appear as /dev/sda (or `sda` with type as disk) in the output of `lsblk`\n\n```\ndocker-machine ssh node-2 lsblk\ndocker-machine ssh node-3 lsblk\n```\n\nSample output\n\n```\nNAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT\nsda       8:0    0    5G  0 disk\nvda     253:0    0   40G  0 disk\n├─vda1  253:1    0 39.9G  0 part /\n├─vda14 253:14   0    4M  0 part\n└─vda15 253:15   0  106M  0 part /boot/efi\n```\n\n## Portworx\n\n- Install NTP on all swarm nodes\n- Install portworx on the nodes that are labelled `data`\n- Create new px volume cluster\n- Restart daemon and enable/start px\n\n```\n#./px_setup.sh \u003cCLUSTER ID\u003e \u003cBLOCK DEVICE\u003e\n./px_setup.sh C123 /dev/sda\n```\n\nGive it a few minutes and verify,\n\n```\ndocker-machine ssh node-2 /opt/pwx/bin/pxctl status\ndocker-machine ssh node-3 /opt/pwx/bin/pxctl status\n```\n\nOutput should start with `Status: PX is operational` and the details of the block device.\n\nAlso verify consul key/value,\n\n```\nopen \"http://$(docker-machine ip node-1):8500/ui/#/dc1/kv/\"\n```\n\n## Application Stack\n\nAPI service and a datbase service that use portworx volume. Check the `db` service in `app.yml` to learn how the portworx volume is created and used.\n\n```\neval $(docker-machine env node-1)\ndocker stack deploy -c app.yml app\n```\n\nVerify the stack \u0026 containers. Current State should be \"Running x ....s ago\"\n\n```\ndocker stack ps -f \"desired-state=running\" app\ndocker service logs app_db\ndocker service logs app_api\n\ndocker service inspect app_db\ndocker service inspect app_api\n\n# Verify node-2 \u0026 node-3 has volume 'app_mydb'\ndocker-machine ssh node-2 docker volume ls\ndocker-machine ssh node-3 docker volume ls\n```\n\nWhen all the 3 containers of `app` stack are running, create tables and populate some data (db:migrate \u0026 db:seed). Run on one of the app containers.\n\n```\neval $(docker-machine env node-1)\napi_node=$(docker stack ps -f \"desired-state=running\" app | grep app_api | awk '{print $4}' | head -1)\neval $(docker-machine env \"$api_node\")\napi_container=$(docker ps -a | grep app_api | awk '{print $1}' | head -1)\ndocker exec -it $api_container rails db:migrate\ndocker exec -it $api_container rails db:seed\n```\n\nVerify the application\n\n```\nopen http://$(docker-machine ip node-1):3000/notes\n```\n\nWebpage should list all the notes records and you should be able view or edit.\n\n## Test PX behavior\n\nLet the node that run mysql container (or db service container), leave swarm cluster\n\n```\n# Switch to swarm manager, if not already on.\neval $(docker-machine env node-1)\ndocker stack ps -f \"desired-state=running\" app\n\n# Let's make the db node leave swarm\nnode=$(docker stack ps -f \"desired-state=running\" app | grep app_db | awk '{print $4}' | head -1)\necho $node\n./swarm_leave.sh $node\n\n# Check swarm nodes \u0026 status\ndocker node ls\n\n# Give it a couple of seconds and see the sttus.\ndocker stack ps -f \"desired-state=running\" app\n```\n\nKeep checking the stack status. \n\nThe app_db service container will have a brief outage and start up on the other data node. That will initially be in \"Preparing ..\" and the application should work when the Current State of that new container changes to \"Running x ....s ago\"\n\n```\ndocker stack ps -f \"desired-state=running\" app\nopen http://$(docker-machine ip node-1):3000/notes\n```\n\nNow try switching the data node\n\n```\ndocker stack ps -f \"desired-state=running\" app\n\n# Join back to swarm\n./swarm_join.sh $node\n\n# Check swarm nodes \u0026 status\ndocker node ls\n\n# Let's make the current active db node leave swarm\nnode=$(docker stack ps -f \"desired-state=running\" app | grep app_db | awk '{print $4}' | head -1)\necho $node\n./swarm_leave.sh $node\n\n# Check swarm nodes \u0026 status\ndocker node ls\n\n# Give it a couple of seconds and see the sttus.\ndocker stack ps -f \"desired-state=running\" app\n```\n\nDatabase container now should switch to other data node and application should still work.\n\n\n## Verify Portworx \u0026 useful commands\n\n```\n# journalctl\ndocker-machine ssh node-2 journalctl -f -u portworx\n\ndocker-machine ssh node-2 systemctl daemon-reload\n\n# status | stop | enable | reenable | start\ndocker-machine ssh node-2 systemctl status portworx\n\n# pxctl\ndocker-machine ssh node-2 /opt/pwx/bin/pxctl status\ndocker-machine ssh node-2 /opt/pwx/bin/pxctl c l\ndocker-machine ssh node-2 /opt/pwx/bin/pxctl v l\n```\n\n# Resources\n\n- https://docs.portworx.com/scheduler/docker/swarm.html\n- https://docs.portworx.com/runc/index.html\n- https://docs.portworx.com/manage/volumes.html\n- https://docs.portworx.com/scheduler/docker/install-swarm.html\n- https://docs.portworx.com/maintain/etcd.html\n- https://github.com/portworx/px-docs/blob/gh-pages/etcd/ansible/index.md\n- http://install.portworx.com:8080\n- http://install.portworx.com:8080?type=dock\u0026stork=false\n\n**Fetch latest enterprise version**\n\n```\ncurl -fsSL 'http://install.portworx.com:8080?type=dock\u0026stork=false' | awk '/image: / {print $2}'\n``` \n\n## Create Volume on one of the PX cluster nodes\n\n```\nCreating stack will create the new volume. ANother option is to create manually and make the volume external true.\ndocker volume create -d pxd --name mydb --opt size=3G\n```\n\n# More\n\nFor more code samples and short video tutorials on development \u0026 development with docker \u0026 cloud,\n\n- https://devteds.com\n- https://github.com/devteds\n- https://www.youtube.com/c/ChandraShettigar\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevteds%2Fswarm-portworx-consul","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevteds%2Fswarm-portworx-consul","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevteds%2Fswarm-portworx-consul/lists"}