{"id":17062887,"url":"https://github.com/bretfisher/dogvscat","last_synced_at":"2025-04-04T18:10:06.579Z","repository":{"id":38185029,"uuid":"133014720","full_name":"BretFisher/dogvscat","owner":"BretFisher","description":"Sample Docker Swarm cluster stack of tools","archived":false,"fork":false,"pushed_at":"2023-01-07T20:07:08.000Z","size":144,"stargazers_count":518,"open_issues_count":10,"forks_count":132,"subscribers_count":43,"default_branch":"main","last_synced_at":"2025-04-02T23:33:10.990Z","etag":null,"topics":["clustering","containers","docker","elk","monitoring","prometheus","rexray","swarm"],"latest_commit_sha":null,"homepage":"http://dogvs.cat","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BretFisher.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-05-11T08:49:18.000Z","updated_at":"2025-03-29T01:49:34.000Z","dependencies_parsed_at":"2023-02-08T00:16:03.552Z","dependency_job_id":null,"html_url":"https://github.com/BretFisher/dogvscat","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BretFisher%2Fdogvscat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BretFisher%2Fdogvscat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BretFisher%2Fdogvscat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BretFisher%2Fdogvscat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BretFisher","download_url":"https://codeload.github.com/BretFisher/dogvscat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247226215,"owners_count":20904465,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clustering","containers","docker","elk","monitoring","prometheus","rexray","swarm"],"created_at":"2024-10-14T10:50:59.538Z","updated_at":"2025-04-04T18:10:06.562Z","avatar_url":"https://github.com/BretFisher.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Dog vs. Cat: Docker Swarm Stacks on Stacks on Stacks\n\nMeanwhile, follow along this presentation I made during DockerCon 2018\nhttps://youtu.be/V9fxU5zJKb4\n\n[![DockerCon EU Swarm Stack](https://img.youtube.com/vi/V9fxU5zJKb4/0.jpg)](https://youtu.be/V9fxU5zJKb4)\n\nThis repo gives a few examples of patterns for how you might build Docker Swarm clusters with all the bells and whistles you would need in a real world setup. Note [I have a course on Swarm for $10 on Udemy](http://swarmmastery.com).\n\nA Docker Swarm cluster needs more then just your app running, it often needs at least these additional services:\n\n- Layer 7 Reverse Proxy (to host multiple HTTP sites on one port)\n- Swarm-aware storage for data persistence\n- Centralized logging of your app containers\n- Centralized monitoring of nodes and containers\n- Cluster management GUI\n- Continuous deployment of updated images\n\nThis demo is meant for you to `git clone` and run locally to help you learn the tools and methods for building a complete Docker Swarm cluster.\n\n# Major To-Do's left (see feature requests in Issues)\n\n- [ ] Show how docker-app could be used for better Continuous Deployment\n- [ ] Show how to deploy servers with simple Terraform and/or Ansible examples\n- [ ] Show how 18.09 SSH makes remote admin so easy\n- [ ] Pull out everything that needs envvars\n- [ ] Use Docker Swarm Secrets for privates\n- [ ] Fix Docker EE Ansible permissions on ELB's and Security groups for port 8080 (app ELB)\n- [x] Better README step-by-step\n- [x] Walkthough videos\n\n## Getting Started\n\nThis repo holds two deployment examples for Docker Swarm\n\n- Docker Swarm CE (Community Edition) open source stack\n- Docker Swarm EE (Enterprise Edition) stack\n\nThe EE stack requires at least a trial license to deploy.\n\n## Deploying the Swarm CE Example\n\nYou can do all this locally on a single node or optionally using Docker Machine to multi-node clusters.\n\n### Step 1: Set needed environment variables\n\nThe scripts and compose/stack files use variables to make this demo easier to get started. Set these at your shell before running commands\n\n```shell\n# for Digital Ocean docker-machine driver\nSSH_FINGERPRINT #fingerprint used to match your SSH key to Digital Ocean's\nDO_SIZE #instance size for Digital Ocean to use for docker-machine\nDO_TOKEN #Digital Ocean API token for creating/deleting droplets\n\n# for Digital Ocean block storage\nREXRAY_DO_TOKEN #Digital Ocean API token so RexRay can create storage volumes, can be same as DO_TOKEN\n\n```\n\n### Step 2: (single node local Swarm)\n\nJust have Docker installed, either via Docker for Windows/Mac or on Linux. See my [YouTube videos on the proper way to setup your OS for Docker](https://www.youtube.com/watch?v=Fc7Rjll30jY\u0026list=PL6cactdCCnTLqhFgmXAVdwLPCM_SZdGYq) using downloads from [store.docker.com](https://store.docker.com).\n\nThen just create a single-node Swarm in that engine:\n\n`docker swarm init`\n\n### Step 2: (multi-node docker-machine Swarm)\n\n`./create-servers.sh` gives example docker-machine commands for creating 3 nodes in various VM environments including locally with VirtualBox, Hyper-V, and in the cloud using Digital Ocean.\n\n### Step 3: Enable Docker Engine Metrics\n\n`./enable-monitoring.sh` simply overwrite `/etc/docker/daemon.json` (we assume it doesn't exist) with two options to enabling the metrics endpoint, which will help Prometheus with more metrics later.\n```json\n{\n  \"metrics-addr\" : \"0.0.0.0:9323\",\n  \"experimental\" : true\n}\n```\n\n### Step 4: Initialize Swarm and Join Nodes\n\n`./create-swarm.sh` gives example docker-machine ssh commands for `docker swarm init` and `join` operations.\n\nAfter this finishes, if you're using my docker-machine example you can connect to Docker TLS endpoint on node1 via:\n\n`docker-machine env dvc1` and then copy/paste the last line of output for your OS.\n\n### Step 5: Enable Persistent Storage with REX-Ray\n\nCreate a docker secret using the `REXRAY_DO_TOKEN` environment variable you set earlier.\n\n`echo $REXRAY_DO_TOKEN | docker secret create rexray_do_token -`\n\nFrom this point on, everything is in stack files! No custom node config's needed. 🎉\n\n`docker stack deploy -c stack-rexray.yml rexray`\n\nThis sets up a global service to run a docker command against the host docker socket via bind-mount to install the storage driver for your cloud. Change the driver name to your cloud or docker volume storage plug-in vendor. This method of wrapping swarm-exec in a global mode service also means any new nodes to join the Swarm later will get the driver installed.\n\n**The above shows off how you can use swarm-exec utility to run a command (even a docker host command) on a set of nodes**\n\n### Step 6: Deploy Reverse Proxy using Traefik\n\nSimple Proxy: `docker stack deploy -c stack-proxy.yml proxy`\n\nThis sets up a simple single-container proxy using Swarms ingress routing mesh to reverse proxy ports 80 and 443. It's good for demos and personal setups but you'll likely want something more as you grow.\n\n**The above shows off how you can use a reverse proxy to control traffic to many web URL's via their DNS name, and also includes Let's Encrypt dynamic config and cert requests**\n\nAdvanced Proxy: `docker stack deploy -c stack-proxy-global.yml proxy`\n\nThis example builds on the simple proxy and adds a global mode Traefik service for HA proxy, and also runs the 80/443 listeners on the host NIC for improved performance and gathering of real client IP's (it then uses overlay networks to talk to app services). For HA Traefik it needs a key/value store so this example uses a single Consul container with RexRay storage. Lastly, it enables a socat container to allow Traefik to run on worker nodes while it uses TCP to talk to the Swarm management API via socat redirect.\n\n**The above shows off how to use host NIC directly in a service to avoid routing mesh, how to encrypt a network with IPSec, and how to use socat to redirect a docker socket to the network so you can void putting management containers on managers.**\n\n### Step 7: Deploy Ops Tools: ELK, Prometheus, and Portainer\n\n`docker stack deploy -c docker-elk/docker-stack.yml -c elk.override.yml elk`\n`docker stack deploy -c swarmprom/docker-compose.yml prom`\n`docker stack deploy -c stack-portainer.yml portainer`\n\n### Step 8: Deploy management tasks like prune\n\n`docker stack deploy -c stack-prune.yml prune`\n\n### Step 9: Deploy sample apps and test\n\n`docker stack deploy -c stack-menu.yml menu`\n`docker stack deploy -c stack-voting.yml vote`\n`docker stack deploy -c stack-ghost.yml ghost`\n\n## Deploying the Swarm EE Example\n\n## Other Notes\n\n### Using Docker Machine? Really???\n- Don't throw out the good in search of the perfect\n- DM works fine solo admins with 3-10 cloud servers\n- Be sure to backup certs from .docker/machine/machines\n- If you're a team of 2-3 and still want to try sticking with DM, maybe try:\n  - https://github.com/bhurlow/machine-share\n  - https://github.com/efrecon/machinery\n\n### Swarm Visualizer\n- You can optionally deploy `stack-visualizer.yml` early on to see how your stacks and services fill out your swarm on port 4040.\n\n`docker stack deploy -c stack-visualizer.yml viz`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbretfisher%2Fdogvscat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbretfisher%2Fdogvscat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbretfisher%2Fdogvscat/lists"}