{"id":15083708,"url":"https://github.com/shazchaudhry/docker-swarm-mode","last_synced_at":"2025-10-05T14:31:47.556Z","repository":{"id":107624974,"uuid":"82408892","full_name":"shazChaudhry/docker-swarm-mode","owner":"shazChaudhry","description":"Setting up a Docker based CI environment. Tools include GitLap, Jenkins, Sonarqube and Nexus","archived":true,"fork":false,"pushed_at":"2019-02-10T17:29:42.000Z","size":1145,"stargazers_count":26,"open_issues_count":3,"forks_count":24,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-09-20T11:29:35.819Z","etag":null,"topics":["continuous-delivery","continuous-integration","devops","docker-swarm-mode","gitlab","jenkins","jenkins-pipeline","nexus3","sonarqube"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shazChaudhry.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2017-02-18T18:53:33.000Z","updated_at":"2024-08-22T19:47:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"f45b807d-4fd6-4c4d-9f5a-da6de296b62b","html_url":"https://github.com/shazChaudhry/docker-swarm-mode","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shazChaudhry%2Fdocker-swarm-mode","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shazChaudhry%2Fdocker-swarm-mode/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shazChaudhry%2Fdocker-swarm-mode/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shazChaudhry%2Fdocker-swarm-mode/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shazChaudhry","download_url":"https://codeload.github.com/shazChaudhry/docker-swarm-mode/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219871465,"owners_count":16554416,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["continuous-delivery","continuous-integration","devops","docker-swarm-mode","gitlab","jenkins","jenkins-pipeline","nexus3","sonarqube"],"created_at":"2024-09-25T06:31:47.970Z","updated_at":"2025-10-05T14:31:42.254Z","avatar_url":"https://github.com/shazChaudhry.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status on Travis](https://travis-ci.org/shazChaudhry/docker-swarm-mode.svg?branch=master)](https://travis-ci.org/shazChaudhry/docker-swarm-mode)\n\n### User story\nAs a member of DevOps team, I want to stand up DevOps tools _(Platform as Code)_ so that projects can run Coninious Integration / Coninious Delivery (CI / CD) pipelines:\n* `Flow Proxy` - The goal of the [Docker Flow Proxy project](https://github.com/docker-flow/docker-flow-proxy) is to provide an easy way to reconfigure proxy every time a new service is deployed, or when a service is scaled. It does not try to \"reinvent the wheel\", but to leverage the existing leaders and combine them through an easy to use integration. It uses HAProxy as a proxy and adds custom logic that allows on-demand reconfiguration.\n* `Flow Swarm Listener` - The goal of the [Docker Flow Swarm Listener project](https://github.com/docker-flow/docker-flow-swarm-listener) is to listen to Docker Swarm events and send requests when a change occurs. At the moment, the only supported option is to send a notification when a new service is created, or an existing service was removed from the cluster.\n* `Sonatype Nexus Repository Manager 3` - Based on CentOS, a free [binary repository manager](https://github.com/sonatype/docker-nexus3) with universal support for popular repository formats such as maven, yum, raw, docker and many other\n* `SonarQube` - [SonarQube](https://github.com/SonarSource/sonarqube) provides the capability to not only show health of an application but also to highlight issues newly introduced. With a Quality Gate in place, you can fix the leak and therefore improve code quality systematically\n* `Jenkins` - As an extensible automation server, [Jenkins](https://hub.docker.com/r/jenkinsci/blueocean/) can be used as a simple CI server or turned into the continuous delivery hub for any project\n* `GitLab Community Edition (CE)` -  [Gitlab](https://github.com/sameersbn/docker-gitlab) is an open source end-to-end software development platform with built-in version control, issue tracking, code review, CI/CD, and more. Self-host GitLab CE on your own servers\n\n![Swarm cluster](pics/CI_Stack.jpg \"Swarm cluster\")\n\nThe architecture of this stack is such that all services are behind an HTTP(S) \u0026 SSH reverse proxy; single point of entry. This reverse proxy in this case is started with self signed certificate _(See ./certs/README.md)_ using docker secrets\n\n### Prerequisite\nDocker swarm mode environment is required\n- Use provided `Vagrantfile` if you are unable to run Docker CE natively on a local machine.\n- *OR* see [Docker for AWS](https://docs.docker.com/docker-for-aws/) documentation on how to create a Docker swarm cluster on AWS\n\n# Deploy CI stack in a VirtualBox with provided Vagrantfile\nThe **assumption** here is that Vagrant, VirtualBox and Gitbash are already install on your machine _(my development environment was a Windows 10 Pro machine)_\n\nExecute the following commands, in gitbash, in order to create a two-node docker swarm mode cluster. The nodes are based on 'ubuntu/xenial64' VM. Once the cluster is created successfully, log in to the master node:\n - `git clone https://github.com/shazChaudhry/docker-swarm-mode.git`\n - `cd docker-swarm-mode`\n - `vagrant up`\n - `vagrant ssh node1` _(Log in to the master node)_\n - `docker node ls` _(confirm that there are two nodes in the cluster; master and worker)_\n - `cd /vagrant`\n\nDeploy stack by running the following commands which will utilize [Docker secrets](https://docs.docker.com/engine/swarm/secrets/) for Jenkins and proxy.\n  * Jenkins secrets are defined in the \"./secrets/jenkins\" directory\n  * Proxy's secrets are defined in the \"./certs\" directory\n  ```\n  docker stack deploy --compose-file docker-compose.portainer.yml portainer\n  docker stack deploy --compose-file docker-compose.yml ci\n  ```\n* Check status of the stack services by running the following command:\n  ```\n  docker stack services ci\n  ```\n* Once all services are up and running, proceed to the next step\n\n#### Service URLs\n* \u003ca href=\"http://node1:9000\"/\u003ehttp://node1:9000\u003c/a\u003e _(Portainer)_\n* \u003ca href=\"https://node1/jenkins\"/\u003ehttps://node1/jenkins\u003c/a\u003e _(Jenkins)_. admin username: `admin`; Password: `admin`\n* \u003ca href=\"https://node1/sonar\"/\u003ehttps://node1/sonar\u003c/a\u003e _(SonarQube)_. admin username: `admin`; Password: `admin`\n* \u003ca href=\"https://node1/nexus\"/\u003ehttps://node1/nexus\u003c/a\u003e _(Nexus)_. admin username: `admin`; Password: `admin123`\n  * Follow these instructions to setup docker registries; at least one should be of type proxy and should point to docker hub: https://help.sonatype.com/repomanager3/private-registry-for-docker/proxy-repository-for-docker\n  * Reserve _\"8082\"_ for docker group repo\n  * Reserve _\"5000\"_ docker hosted repo\n* \u003ca href=\"https://node1/gitlab\"/\u003ehttps://node1/gitlab\u003c/a\u003e _(Gitlab CE)_. admin username: `root`; Password: `Password01`\n  * Gitlab takes a few minutes to become available so please be a little patient :)\n  * You can also view service logs, `docker service logs -f ci_gitlab`, to check when gitlab becomes available\n\n#### Cloning repositories with HTTPS\nThis is assuming you already have imported some repositories into gitlab or have created your own repos:\n* ```git config --global http.sslVerify false``` _(Turns off Git SSL Verification for a non trusted server certificate. Otherwise, you may receive 'SSL certificate problem: self signed certificate' error)_\n* ```git clone https://root:Password01@node1/gitlab/[GROUP_NAME]/[REPOSITORY_NAME].git```\n\n#### Cloning repositories with SSH\nThis is assuming you have already uploaded your ssh public key to gitlab:\n* ```git clone ssh://git@node1:10022/[GROUP_NAME]/[REPOSITORY_NAME].git```\n\n\n#### Clean-up\nOn the swarm master node, run the following commands:\n* `docker stack rm ci` to remove the stack\n* `exit` to exit the vagrant box\n* `vagrant destroy --force` to destroy the VMs\n\n# Deploy CI stack on \"Docker for AWS\"\nIt is assumed you have followed [Docker for AWS](https://docs.docker.com/docker-for-aws/) documentation to create a new VPC. Follow these commands in an ssh client to log in to your master node _(I'm using gitbash on Windows 10 Pro)_.\n\n**Please** note you can not ssh directly into worker nodes. You have to use a manager node as a jump box\n- `eval $(ssh-agent) OR exec ssh-agent bash`\n- `ssh-add -k ~/.ssh/personal.pem` _(You wiill have to use your own key)_\n- `ssh-add -L`\n- `ssh -A docker@\u003cManager Public IP\u003e`\n- `cat /etc/*-release`\n- `docker node ls`\n\nClone this repo and change directory by following these commands:\n- `alias git='docker run -it --rm --name git -v $PWD:/git -w /git alpine/git'` _(This alias is only required if git is not already installed on your machine. This alias will allow you to clone the repo using a git container)_\n- `git version`\n- `git clone https://github.com/shazChaudhry/docker-swarm-mode.git`\n- `sudo chown -R $USER:$USER docker-swarm-mode`\n- `cd docker-swarm-mode`\n\nStart the Portainer by running:\n- `docker stack deploy -c docker-compose.portainer.yml portainer`\n\nNotes:\n\u003e In a Docker swarm mode, only a single Compose file is accepted. If your configuration is split between multiple Compose files, e.g. a base configuration and environment-specific overrides, you can combine these by passing them to docker-compose config with the -f option and redirecting the merged output into a new file.\n\n\u003e These instructions are only needed if files to be merged have been edited / updated for any reason. For your info, a default \"docker-stack.yml\" has already been generated and is ready to be used\n- `alias docker-compose='docker run --interactive --tty --rm --name docker-compose --volume $PWD:/compose --workdir /compose docker/compose:1.23.1'`\n- `docker-compose version`\n- `docker-compose -f docker-compose.yml -f docker-compose.AWS.cloudstor.yml config \u003e docker-stack.yml`\n\nNote:\n\u003e If you did have to generate a new `docker-stack.yml` file above then ensure you follow the instructions below before launching the platform stack\n- Open the generated \"docker-stack.yml\" file, address WARNINGS _(top two lines)_ and then delete WARNING lines\n  - Ensure that the source path for settings.xml file mounted into jenkins' container is correct i.e. $PWD is not missing\n  - Ensure that the source path for secret files is correct i.e. $PWD is not missing\n\n Run the combined stack. Please note that secrets are defined in ./secrets/jenkins and ./certs directories in this repo:\n - `export DefaultDNSTarget=\u003c\u003cfind this value on your CloudFormation page in AWS console\u003e\u003e`\n - `docker stack deploy --compose-file docker-stack.yml ci` You may be interested in knowing that this stack defines a volume plugin called [Cloudstor](https://docs.docker.com/docker-for-aws/persistent-data-volumes/). Docker containers can use a volume created with Cloudstor _(available across entire cluster)_ to mount a persistent data volume\n - `docker stack services ci`\n\n Note:\n \u003e If Jenkins in this stack is unable to run sibling containers, set appropriate permission: `sudo setfacl -m u:1000:rw /var/run/docker.sock` on all nodes. If this command fails then a workround is - `sudo chmod 666 /var/run/docker.sock` on all nodes\n\n#### Service URLs\n* http://[DefaultDNSTarget]:9000 _(Portainer)_\n* https://[DefaultDNSTarget]/jenkins _(Jenkins)_. admin username: `admin`; Password: `admin`\n* https://[DefaultDNSTarget]/sonar\u003e _(SonarQube)_. admin username: `admin`; Password: `admin`\n  * Install / uninstall plugins as required by going to \"Administration -\u003e Marketplace\" section\n* https://[DefaultDNSTarget]/nexus _(Nexus)_. admin username: `admin`; Password: `admin123`\n  * Follow these instructions to setup docker registries; at least one should be of type proxy and should point to docker hub: https://help.sonatype.com/repomanager3/private-registry-for-docker/proxy-repository-for-docker\n  * Reserve _\"8082\"_ for docker group repo\n  * Reserve _\"5000\"_ docker hosted repo\n* https://[DefaultDNSTarget]/gitlab _(Gitlab CE)_. admin username: `root`; Password: `Password01`\n  * Gitlab takes a few minutes to become available so please be a little patient :)\n\n#### Configuring your clinet for interacting with docker repos in Nexus\n* You will need to look up docker daemon documentation for your system. This was tested on ubuntu/xenial64.\n* Create a file `/etc/docker/daemon.json` and copy the following content in that file\n  ```\n  {\n  \"insecure-registries\": [\n    \"[DefaultDNSTarget]:443\",\n    \"[DefaultDNSTarget]:5000\"\n  ],\n  \"disable-legacy-registry\": true\n  }\n  ```\n* You have to restart the daemon after setting this `sudo systemctl restart docker`\n* `docker login -u admin -p admin123 [DefaultDNSTarget]:443` This should be successful and then you should be able to pull images from docker group repo i.e. `docker image pull [DefaultDNSTarget]:443/hello-world:latest`\n* `docker login -u admin -p admin123 [DefaultDNSTarget]:5000` This should be successful and then you should be able to push images to your nexus hosted repo i.e. `docker image push [DefaultDNSTarget]:5000/hello-world:latest`\n\n#### Cloning repositories with SSH\nThis is assuming your ssh public key has already been uploaded to gitlab:\n* ```git clone ssh://git@[DefaultDNSTarget]:10022/[GROUP_NAME]/[REPOSITORY_NAME].git```\n\n#### Cloning repositories with HTTPS\nThis is assuming you already have imported some repositories into gitlab or have created your own repos:\n* ```git config --global http.sslVerify false``` _(Turns off Git SSL Verification for a non trusted server certificate. Otherwise, you may receive 'SSL certificate problem: self signed certificate' error)_\n* ```git clone https://root:Password01@[DefaultDNSTarget]/gitlab/[GROUP_NAME]/[REPOSITORY_NAME].git``` _(username \u0026 password will be required if repositories are private)_\n\n#### Clean up\n1. `docker stack rm ci`\n2. `swarm-exec docker system prune --volumes -af`\n3. Before deleting a Docker4AWS stack through CloudFormation, you should remove all relocatable Cloudstor volumes using docker volume rm from within the stack. EBS volumes corresponding to relocatable Cloudstor volumes are not automatically deleted as part of the CloudFormation stack deletion\n\n# Setup [Jenkins declarative](https://jenkins.io/doc/book/pipeline/syntax) pipeline\nSteps to setting up this pipeline should be identical regardless of the deployment methods shown above; Local or AWS\n- In the GitLab instance, import [spring-petclinic](https://github.com/shazChaudhry/spring-petclinic) project from GitHub: https://github.com/shazChaudhry/spring-petclinic ![alt text](pics/spring-petclinic.PNG \"Spring Petclinic\")\n- In the Jenkins instance, create a new Blue Ocean pipeline project:\n  - Where do you store your code? = `Git` _(Keep an eye on the [support for Gitlab](https://jenkins.io/projects/blueocean/roadmap/) in Blue Ocean. At the time of writing this, the plugin was still in planned phase)_\n  - Repository URL = `ssh://git@gitlab:22/java/spring-petclinic.git` ![alt text](pics/create-pipeline.PNG \"Create Pipeline\")\n- When informed to register the public SSH key with your Git server to continue, copy the generated key\n- In the GitLab servicer, go to \"Admin area -\u003e Deploy Keys\". And then paste the key ![alt text](pics/admin-deploy-keys.PNG \"Admin Deloy Keys\")\n- Once \"Deloy Key\" has been created, head over to the \"spring-petclinic\" project in GitLab\n- Go to \"Settings -\u003e Repository\" and then expand the Deploy Keys section. Ensure that you enable the deploy key at the bottom of the page ![alt text](pics/project-deploy-keys.PNG \"Enable Project Deloy Keys\")\n- Now, in Jenkins, click on the \"Create pipeline\". A job in jenkins will be created which then will:\n  - Build maven packages using Nexus maven repos\n  - Run static code analysis and publish report on SonarQube instance\n  - Inform GitLab of the build status\n\nAfter a successful build, a pipeline should look like this: ![alt text](pics/Jenkins-pipeline.PNG \"Jenkins Blue Ocean pipeline\")\n\nAfter a successful build, maven build artifacts should be deployed in maven snapshot repository: ![alt text](pics/maven-snapshot-repo.PNG \"Maven Snapshot Repository\")\n\nAfter a successful build, SonarQube reports should be available: ![alt text](pics/sonarqube.PNG \"SonarQube Report\")\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshazchaudhry%2Fdocker-swarm-mode","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshazchaudhry%2Fdocker-swarm-mode","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshazchaudhry%2Fdocker-swarm-mode/lists"}