{"id":13581122,"url":"https://github.com/nmarus/docker-swarm-ansible","last_synced_at":"2025-07-17T06:33:50.301Z","repository":{"id":89134529,"uuid":"224499144","full_name":"nmarus/docker-swarm-ansible","owner":"nmarus","description":"Ansible Playbook that deploys 3 node Docker Swarm with GlusterFS, Traefik, and Portainer Services","archived":false,"fork":false,"pushed_at":"2019-11-29T20:24:56.000Z","size":41,"stargazers_count":73,"open_issues_count":2,"forks_count":25,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-05-01T09:38:48.328Z","etag":null,"topics":["ansible","docker-swarm","glusterfs","haproxy","portainer","traefik"],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nmarus.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-11-27T19:05:32.000Z","updated_at":"2025-01-07T22:28:33.000Z","dependencies_parsed_at":"2023-03-12T04:01:16.867Z","dependency_job_id":null,"html_url":"https://github.com/nmarus/docker-swarm-ansible","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nmarus/docker-swarm-ansible","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmarus%2Fdocker-swarm-ansible","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmarus%2Fdocker-swarm-ansible/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmarus%2Fdocker-swarm-ansible/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmarus%2Fdocker-swarm-ansible/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nmarus","download_url":"https://codeload.github.com/nmarus/docker-swarm-ansible/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nmarus%2Fdocker-swarm-ansible/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265573545,"owners_count":23790459,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","docker-swarm","glusterfs","haproxy","portainer","traefik"],"created_at":"2024-08-01T15:01:58.328Z","updated_at":"2025-07-17T06:33:50.260Z","avatar_url":"https://github.com/nmarus.png","language":"Shell","funding_links":[],"categories":["Shell"],"sub_categories":[],"readme":"# Docker Swarm with GlusterFS, Traefik, HAProxy, and Portainer\n\nThe goal of this repo is to demonstrate using Ansible to build out Docker Swarm architecture that can be used to simply and reliably deploy and manage container workloads. This setup is intended for a small Development Lab and to get familiar with how high availability, scaling, and routing works with Docker. This can be scaled to use in production and there are some notes at the end of this README that will point you in the right direction if you desire to do so.\n\nThis setup will provide a 3 node Docker Swarm with Layer 7 routing, Web UI management, and a distributed file system for Container volumes. A HAProxy node is setup to load balance traffic between each of the Layer 7 routers on the Docker Swarm nodes.\n\n### Technologies in Use\n\n#### Docker\nContainers, nuff said.\n\n#### Docker Swarm\nUses Docker to create a swarm of Docker hosts where you can deploy application services. A Swarm is managed like a single Docker node and allows service networks to span across multiple hosts.\n\n#### Gluster\nGluster is a Distributed Filesystem to allow shared persistent storage volumes across Docker Swarm Cluster. This is used as an alternative to NFS or other shared storage technologies for simplicity and minimal hardware footprint. This example will install the Gluster Storage Plugin for Docker.\n\n#### Traefik\nTraefik is a Layer 7 Router that automatically discovers services in Docker Swarm and routes traffic to the appropriate container(s) in the cluster. Similar to the functionality provided by Docker EE UCP with Interlock. The nice thing about Traefik is that it integrates into the Docker Swarm API and will dynamically create the Layer 7 routes based on labels you add to you containers or services. There is no administrative action in Traefik itself.\n\n#### Portainer\nPortainer is a web UI for managing Docker Swarm. Functionality is similar to Docker EE UCP but provides more features and is an open source project.\n\n#### HAProxy\nHAProxy is a load balancer and SSL off loader. This is typically placed in front of the application servers in a Cluster. In this setup, HAProxy does SSL Offloading and load balances requests to each of the Traefik services running on the Docker Swarm nodes. It additionally monitors each of the hosts and if one is no longer available, will remove that host from the distribution of traffic until it is brought back online.\n\n## Deployment\n\n### Requirements\n\n**Setup Ansible**\n\nThis makes use of Ansible to automate the deployment of this setup. To run the Ansible commands, your workstation should have the Ansible CLI utilities installed.\n\n_For OSX with Homebrew:_\n\n```bash\nbrew install ansible\n```\n\n**Setup Vagrant**\n\nIf wanting to utilize Vagrant, you will need to make Vagrant and Virtualbox are both installed.\n\n_For OSX with Homebrew:_\n\n```bash\nbrew cask install vagrant\nbrew cask install virtualbox\n```\n\n### Setup Machines to Host the Swarm Cluster\n\n#### Vagrant\n\nIf you wish to try this setup with Vagrant the following steps can be used to get up and running fast. The Vagrant file deploys 4 hosts.\n\n* 1 HAParoxy Host\n* 3 Docker Swarm Hosts\n\nTo bring up the infrastructure, run:\n\n```bash\n# install disk resize plugin\nvagrant plugin install vagrant-disksize\n\n# launch vagrant environment\nvagrant up\n```\n\nTo shut down the infrastructure in Vagrant, run:\n\n```bash\nvagrant halt\n```\n\nTo tear down all the infrastructure in Vagrant, run:\n\n```bash\nvagrant destroy -f\n```\n\n#### External Hosts or Manual Setup\n\nIf wanting to deploy on your own defined infrastructure you will need to provision 4 hosts (ie, VMWare Fusion / Workstation / ESXi / AWS / etc...)\n\nThis has been tested with Ubuntu 18.04, but should* work with any Debian flavor of linux.\n\n* 4 Hosts running Ubuntu 18.04\n* Primary User of 'ubuntu' with password 'ubuntu' (modifiable in hosts definition)\n* SSH Service available that above credentials can log in to\n* The 'ubuntu' user should have access to run `sudo`\n* Each host should have:\n  * 1 Core\n  * 1 GB Ram\n  * Primary drive of 20GB where OS is installed\n* The 3 Docker Swarm hosts should also have a econdary non-initialized 10GB+ Drive. This will be used for XFS file system and Gluster Volumes.\n\n## Initialize Infrastructure\n\nAnsible is used to provision the infrastructure.\n\nAll customizations are found in the Ansible Inventory that is defined in the files:\n\n* `hosts` - Inventory of hosts\n* `host-vagrant` - Inventory of hosts for vagrant setup\n* `playbooks/config.yml` - Application config options\n\nInventory Groups:\n\n* `[haproxy]` SINGLE node that will be configured as the front end load balancer with HAProxy\n* `[swarm_managers]` group in hosts MUST define at minimum 3 Nodes\n* `[swarm_workers]` group in hosts is optional and can include any number of non Manager Nodes\n* `[gluster_nodes]` group in host MUST define ONLY 3 Nodes\n\n### Deployment Overview\n\nThe following high level actions are performed by the Ansible script:\n\n* Updates hosts packages\n* Installs required dependencies\n* Installs Docker CE on swarm nodes\n* Sets up Ansible user to have permission to run Docker without sudo access\n* The first node defined in `[swarm_managers]` is setup as the leader in the Swarm Cluster\n* The remaining nodes are setup as Swarm Managers (3 required for clustering, ie 1 Leader, 2 managers)\n* If `[swarm_workers]` is defined, joins these to the Swarm Cluster with the Worker role\n* On `[gluster_nodes]`, configures the second drive (sdb) with a XFS file system\n* Sets up a GlusterFS Cluster on nodes in `[gluster_nodes]` (Ansible will assume there are 3 total, others in this group will not be initialized)\n* Install Docker Gluster Storage Plugin on all `[swarm_managers]` and `[swarm_workers]`\n* Installs [Traefik](https://traefik.io) stack as a global Service on all `[swarm_managers]`\n* Install [Portainer](https://www.portainer.io) Agents as a global service on all `[swarm_managers]` and `[swarm_workers]`\n* Install Portainer UI with a replica set of 1 on `[swarm_managers]`\n* Install HAProxy on dedicated node\n* Creates a self signed certificate for SSL offloading with HA Proxy\n* Routes traffic received on port 80 and 443 to the Traefic Services running on the Swarm manager Nodes.\n\n### Setup DNS\n\nFor the Layer 7 routing to work, the DNS names must be used when accessing the apps through HAProxy and in turn, the Traefik Layer 7 Router.\n\nEach of the deployed applications must resolve to the host running HAProxy. You can accomplish this in one of two ways.\n\n1. DNS - If you a local DNS server, or are deploying into a public cloud, create a DNS entry that points to the HAProxy Host.\n\n2. Local Dev via Hosts File - If you are testing on a local network or in a dev environment, it may be easier to simply modify your hosts file. Example entries below will match the default names defined in the variables of the Ansible hosts file.\n\n**Example Entries in local Hosts File:**\n\n```\n10.10.10.10 portainer.docker.local\n10.10.10.10 traefik.docker.local\n10.10.10.10 wordpress.docker.local\n```\n\n### Test Connectivity to Hosts\n\nThis should return ok and ensure that Ansible config is able to login and execute commands on the hosts.\n\n**For Vagrant Infrastructure**\n\n```bash\nansible all --inventory-file=hosts-vagrant -a \"/bin/echo hello\"\n```\n\n**For External Hosts or Manual Setup on Ubuntu 18.04**\n\n```bash\nansible all -a \"/bin/echo hello\"\n```\n\n### Run Ansible Playbook\n\nOnce everything is configured and verified, run the playbook to configure the clusters.\n\n**For Vagrant Infrastructure**\n\n```bash\nansible-playbook --inventory-file=hosts-vagrant playbooks/install.yml\n```\n\n**For External Hosts or Manual Setup on Ubuntu 18.04**\n\n```bash\nansible-playbook playbooks/install.yml\n```\n\n### Test Connectivity to Traefik\n\nIf using the default Ansible variables defined in the hosts file, you can navigate to:\n\n* http://traefik.docker.local\n\nUsername/Password = admin:password1234\n\n### Test Connectivity to Portainer\n\nIf using the default Ansible variables defined in the hosts file, you can navigate to:\n\n* http://portainer.docker.local\n\nUsername/Password = admin:password1234\n\n### Test Connectivity to HAPRoxy Status Page\n\nIf using the default Ansible variables defined in the hosts file, you can navigate to:\n\n* http://10.10.10.10/stats\n\nUsername/Password = admin:password1234\n\n### Test Deploying a Simple Application Stack to the Swarm\n\n**wordpress-stack.yml**\n\n_Note: This assumes that you are using the default Ansible variables in the hosts file._\n\n```yaml\nversion: '3.5'\nservices:\n  wordpress:\n    image: wordpress\n    environment:\n      WORDPRESS_DB_HOST: db\n      WORDPRESS_DB_USER: exampleuser\n      WORDPRESS_DB_PASSWORD: examplepass\n      WORDPRESS_DB_NAME: exampledb\n    ports:\n      - \"80\"\n    networks:\n      - wordpress\n      - web\n    volumes:\n      - wordress_data:/var/www/html/wp-content\n    deploy:\n      labels:\n        - \"traefik.enable=true\"\n        - \"traefik.http.routers.wordpress.rule=Host(`wordpress.docker.local`)\"\n        - \"traefik.http.services.wordpress.loadbalancer.server.port=80\"\n        - \"traefik.docker.network=web\"\n\n  db:\n    image: mysql:5.7\n    environment:\n      MYSQL_DATABASE: exampledb\n      MYSQL_USER: exampleuser\n      MYSQL_PASSWORD: examplepass\n      MYSQL_RANDOM_ROOT_PASSWORD: '1'\n    networks:\n      - wordpress\n    volumes:\n      - mysql_data:/var/lib/mysql\n\nnetworks:\n  wordpress:\n    driver: overlay\n    attachable: true\n    name: wordpress\n  web:\n    external: true\n    name: web\n\nvolumes:\n  wordress_data:\n    driver: glusterfs\n    name: \"gfs/wordpress_data\"\n  mysql_data:\n    driver: glusterfs\n    name: \"gfs/wordpress_mysql\"\n```\n\n**Deploying stack from CLI**\n\nFrom a Docker Swarm Node (Manager or Worker):\n\n```bash\n# Create Directories on Mounted GFS volume for storage\nsudo mkdir /mnt/gfs/wordpress_data\nsudo mkdir /mnt/gfs/wordpress_mysql\n\n# Deploy above wordpress stack\ndocker stack deploy --compose-file=wordpress-stack.yml wordpress\n```\n\n**Deploying Stack from Portainer UI**\n\n1. Access Portainer UI at http://portainer.docker.local\n2. Select Stacks\n3. Name the stack 'wordpress'\n4. Copy and Paste above yaml into Web Editor\n5. Click 'Deploy Stack'\n\n**Test Wordpress**\n\nNavigate to: http://wordpress.docker.local\n\n_Note: This may take a few minutes to respond after creating stack due to docker downloading images referenced and them starting up._\n\n### Breakdown of Required Elements Defined in Stack File\n\n**Services That Will Require a Eternal Web Route (Layer 7 via Traefik)**\n\n_Note: For a full example reference Wordpress config above._\n\nEnsure the following:\n\n1. The service declaration in your stack yaml file should have the port that will be used to access the web application exposed. This does not require it to be mapped to a host port.\n\n   Example:\n\n   ```yaml\n   services:\n     myservice:\n       [...]\n       ports:\n          - \"80\"\n       [...]\n   ```\n\n2. The deploy option on the exposed service must have the Traefik network assigned and Traefik labels added. The router name and service are arbitrary, but must be unique in Traefik.\n\n   Example:\n\n   ```yaml\n   services:\n     myservice:\n       [...]\n       networks:\n         \u003ctraefik_network_name\u003e:\n           external: true\n           name: \u003ctraefik_network_name\u003e\n       [...]\n       deploy:\n         labels:\n           - \"traefik.enable=true\"\n           - \"traefik.http.routers.\u003crouter_name\u003e.rule=Host(`\u003cweb_url\u003e`)\"\n           - \"traefik.http.services.\u003cservice_name\u003e.loadbalancer.server.port=\u003ccontainer_port\u003e\"\n           - \"traefik.docker.network=\u003ctraefik_network_name\u003e\"\n   ```\n\n3. The Traefik Network must be defined in your compose file in order to assign it to a service. This network has already been created if you have already installed Traefik so it must be reference as an external service.\n\n   Example:\n\n   ```yaml\n   [...]\n   networks:\n     \u003ctraefik_network_name\u003e:\n       external: true\n       name: \u003ctraefik_network_name\u003e\n   ```\n\n**Services That Will Require Persistent Storage (via GlusterFS)**\n\n_Note: For a full example reference Wordpress config above._\n\nEnsure the following:\n\n1. The volumes are defined that use the Gluster Storage Driver.\n\n   Example:\n\n   ```yaml\n   volumes:\n     \u003cvolume_name\u003e:\n       driver: glusterfs\n       name: \"gfs/\u003cvolume_dir\u003e\"\n   ```\n\n2. The volumes of the service use the defined volumes.\n\n   Example:\n\n   ```yaml\n   services:\n     myservice:\n       [...]\n       volumes:\n         - \u003cvolume_name\u003e:\u003ccontainer_path\u003e\n   ```\n\n3. The volume dir path has been created on in Gluster volume.\n\n   Example:\n\n   ```bash\n   sudo mkdir \u003cgluster_mount_path\u003e/\u003cvolume_dir\u003e\n   ```\n\n## Re-running Ansible Playbook\n\nIt is possible to re-run the playbook. Most of the tasks check to see if they were already completed and will be skipped. The most applicable use case to re-run the playbook is when adding additional workers or managers. Re-running the playbook after changing the Gluster node assignment and/or most of the environment variables however is not supported and will likely break stuff or cause the Ansible playbook to fail.\n\nLook to the playbooks for `upgrade-docker`, `upgrade-packages`, and `redeploy-apps` for some maintenance operations.\n\n## Semi Production Setup\n\nIf deploying this public cloud, or are looking for a more robust infrastruture, you will probably want to increase the total number of nodes, make use of a load balancer, and implement SSL. This is not a complete guide on how to do this, but offers some initial insights on where to start. When referencing Public Cloud below, this will refer to Amazon Web Services, however this can also be applied to a hybrid (on premise data center) or other cloud provider with minor changes.\n\n**Ansible Modifications**\n\nWhen deploying this to public cloud and/or in an environment with DNS, you will want to switch out the inventory from using ip address to using host names. Additionally, you will probably want to not use a username / password method of connecting over SSH and should rather use SSH keys.\n\nFor the DNS name modifications, change the entries in the hosts file to use those names. These names MUST also be resolvable from each host internally. For example if you can access hosts at myhost1.mycorp.net, myhost2.mycorp.net, etc. Each of those hosts internally should also be able to access the other hosts by the same DNS name.\n\nTo setup the Ansible script to use SSH keys vs password, you will want to remove `ansible_ssh_pass` and `ansible_become_pass` variable declaration and add `ansible_ssh_private_key_file` in the Ansible host file:\n\n```\nansible_ssh_private_key_file=/path/to/ssh_key.pem\n```\n\nIn addition to this, you want to make sure that the user can run `sudo` without a password. This will be the typical setup when using an Ubuntu AMI from AWS with EC2.\n\n**Load Balancer and SSL**\n\nWhile Traefik can be configured to do SSL offloading, often a better better approach is to make use of an AWS Application Load Balancer (ALB). The advantages to this is that it can do the SSL offloading as well as distribute traffic between each of your Traefik router nodes.\n\nFor each domain you wish to host Docker applications under:\n\n1. Create a AWS Route 53 DNS domain and assign it as the name server from your domain registrar.\n2. Create a AWS ALB and specify each one of your Traefik Nodes (Docker Managers) as targets.\n3. Apply Healthchecks to the ALB so that it can determine if a specific host is up or down.\n4. Add your SSL cert (or optionally a wildcard cert if hosting multiple apps on the same domain) to the ALB\n4. In your Route 53 DNS Domain, create DNS CNAME records to point the application(s) name to you ALB. (i.e `www.mydomain.com`, `file.mydomain.com`)\n5. Alternatively, if you are hosting all your applications for a domain in Docker Swarm you can create a DNS CNAME wildcard record (i.e. `*.mydomain.com`) and point that at your ALB. The advantage to this is that you will not need to create individual Route 53 records each time you add a new application. However, this assumes that all application records are under the same domain.\n\n**Scaling Docker Swarm**\n\nWhile this 3 node setup can handle a decent workload, you may eventually want to increase the number of nodes that run your application, or split out the Gluster Cluster.\n\nTo do this you can modify the Ansible hosts file. For example, to split out Gluster into its own dedicated compute and add 3 worker nodes to the swarm, you can modify the following:\n\n```\n[swarm_managers]\n10.10.10.10\n10.10.10.20\n10.10.10.30\n\n[swarm_workers]\n10.10.10.40\n10.10.10.50\n10.10.10.60\n\n[gluster_nodes]\n10.10.10.70\n10.10.10.80\n10.10.10.90\n```\n\n**Other Considerations**\n\nWhile Gluster is a good way to have a distributed filesystem for use with Swarm, if you are deploying on a Cloud provider, it would be advisable to use another volume storage Driver. What you choose for this will depend on performance, cost, and complexity requirements. Amazons EFS (NFS) is one of many ways to accomplish this.\n\nFor high traffic sites, you will also want to break out the Traefic router to run on dedicated hosts. This is out of the scope of this Ansible playbook, but should be fairly easy to modify. An option would be to setup 2 dedicated Swarm Workers and deploy Traefik services only to those. However, to keep other workloads off these workers, or to make sure Traefik services are only assigned here, you will have to make use of Docker labels and constraints.\n\nIf running in environments where you do not have access to inherent SSL offloading and load balancing that is provided from services like AWS ALB, you could make use of a setup that implements HAProxy as an alternative. A simple example of this is provided with this setup.\n\nTraefik also has this capability as integrations for LetsEncrypt SSL certs (good service if you want free SSL) however, you would still have to load balance between each of your Traefik nodes (assuming you want more than one). Note that if you go down the Traefik route with LetsEncrypt, you will need to also implement a shared key store between the nodes (i.e. Consul).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnmarus%2Fdocker-swarm-ansible","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnmarus%2Fdocker-swarm-ansible","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnmarus%2Fdocker-swarm-ansible/lists"}