{"id":21974420,"url":"https://github.com/abdes/submodule-docker-dev-workflow","last_synced_at":"2025-03-22T23:40:37.330Z","repository":{"id":120155460,"uuid":"63486570","full_name":"abdes/submodule-docker-dev-workflow","owner":"abdes","description":"Documentation and example multi-module application that uses git submodules and docker to simplify the development workflow (init, change, test, release).","archived":false,"fork":false,"pushed_at":"2017-05-22T12:42:46.000Z","size":24,"stargazers_count":46,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-28T03:19:21.605Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/abdes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-07-16T14:27:57.000Z","updated_at":"2025-01-10T14:31:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"1a30de16-8a58-4480-a7bc-dd6f8251490a","html_url":"https://github.com/abdes/submodule-docker-dev-workflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdes%2Fsubmodule-docker-dev-workflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdes%2Fsubmodule-docker-dev-workflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdes%2Fsubmodule-docker-dev-workflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/abdes%2Fsubmodule-docker-dev-workflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/abdes","download_url":"https://codeload.github.com/abdes/submodule-docker-dev-workflow/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245036127,"owners_count":20550662,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-29T15:45:15.144Z","updated_at":"2025-03-22T23:40:37.307Z","avatar_url":"https://github.com/abdes.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Development workflow using _git submodules_ and _docker_\n\nThis setup aims at simplifying the development workflow and increasing its\nefficiency no matter whether the developer is (remote, together with the dev\nteam), and the experience she/he has with the system(familiar with the full\nsystem or just contributing to a specific part).\n\n## Table of Content\n\u003c!-- TOC depthFrom:2 depthTo:6 withLinks:1 updateOnSave:1 orderedList:0 --\u003e\n\n- [Table of Content](#table-of-content)\n- [Credits](#credits)\n- [Prerequisites](#prerequisites)\n\t- [Git configuration](#git-configuration)\n- [Background and Assumptions](#background-and-assumptions)\n- [Creating the git repo for the development environment](#creating-the-git-repo-for-the-development-environment)\n- [Adding submodules](#adding-submodules)\n- [Removing a submodule](#removing-a-submodule)\n\t- [Temporarily removing a submodule](#temporarily-removing-a-submodule)\n\t- [Permanently removing a submodule](#permanently-removing-a-submodule)\n- [Working with container and submodules](#working-with-container-and-submodules)\n\t- [Setting up the development environment as a developer](#setting-up-the-development-environment-as-a-developer)\n\t- [Working independently on a module](#working-independently-on-a-module)\n\t- [Getting an update from the submodule’s remote](#getting-an-update-from-the-submodules-remote)\n\t- [Making updates to the container](#making-updates-to-the-container)\n\t- [Grabbing container updates](#grabbing-container-updates)\n\t- [Updating a submodule in-place in the container](#updating-a-submodule-in-place-in-the-container)\n- [Dockerizing everything](#dockerizing-everything)\n\t- [Quick start](#quick-start)\n\t- [How it's done](#how-its-done)\n\t\t- [arangodb](#arangodb)\n\t\t- [server](#server)\n\t\t- [web](#web)\n\t\t- [nginx - reverse proxy and router](#nginx-reverse-proxy-and-router)\n- [Conclusion](#conclusion)\n\n\u003c!-- /TOC --\u003e\n\n## Credits\n\n* The original idea by Amine Mouafik [Efficient development workflow using Git submodules and Docker Compose](https://www.airpair.com/docker/posts/efficiant-development-workfow-using-git-submodules-and-docker-compose).\n* The excellent [git documentation](https://git-scm.com/doc)\n* [Mastering Git submodules](https://medium.com/@porteneuve/mastering-git-submodules-34c65e940407#.s51gcqy2p)\n  by Christophe Porteneuve\n\n## Prerequisites\n\n```shell\n$ git -v\n$ docker -v\n$ docker-compose -v\n```\nDetailed instructions for installing and configuring the above tools are\navailable on the internet:\n* git: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git\n* docker: https://docs.docker.com/engine/installation/\n* docker-compose: https://docs.docker.com/compose/install/\n\nDocker containers are an important part of this development stack and its\nassociated workflow. Docker tools must be properly installed and configured\non the devloper workstation.\n\n\u003e On Mac OS X, it is not recommended to use the Docker Toolbox due to several\n\u003e issues with volumes through the virtualbox VM. The newer Docker for Mac,\n\u003e which comes with the lightweight [xhyve](https://github.com/mist64/xhyve/)\n\u003e virtualization layer, should be used instead.\n\nGit is the chosen SCM for this development environment/workflow and git\nsubmodules are an important part of it. The same principles and methodology\ncan be used with other SCM tools as long as they can provide a similar\nmechanism than git submodules to integrate source for multiple repositories\ninto the file tree of a master project.\n\n\u003e Git submodules are tricky to use and may seem a bit daunting at first but\n\u003e they have been chosen for this workflow over other alternatives for the\n\u003e following reasons:\n\u003e * built into git with no need for any extra tools\n\u003e * flexibility to work with each module as a separate repository or to\n\u003e   work from within the enclosing project.\n\u003e\n\u003e Alternative to git submodules include\n[git subtree](https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt)\nand [git-subrepo _(3rd party tool)_](https://github.com/ingydotnet/git-subrepo).\n\n### Git configuration\nTo make the work with git submodules more reliable, it is strongly recommended\nto have the following git configuration setup:\n```shell\n$ git config --global diff.submodule log\n```\nEquivalent to `git diff --submodules=log`.\n\n```shell\ngit config --global status.submoduleSummary true\n```\nExtend the status information to add the commits in the included submodules.\n\n```shell\ngit config --global alias.spush 'push --recurse-submodules=on-demand'\n```\nAn alias that can be used when pushing commits to the remote repos to make sure\nthat all submodules commits are pushed when pushing the main project.\n\u003e Failure to do so will create issues when developers need to update their\n\u003e environment with missing commits from the submodules and errors when trying\n\u003e to update them.\n\n## Background and Assumptions\n\nThe example application we will be working on is composed of the following modules:\n* arango: the backend database, using [ArangoDB](https://www.arangodb.com)\n* server: server example application, using [node.js](https://nodejs.org/en/), for the\n  implementation of the REST API\n* web: the public web site for the example application, served through\n  [nginx](https://www.nginx.com)\n\nThe domain name 'example.local' will be used locally for the development environment\nand two host entries need to be configured in **/etc/hosts** for the _server_ and\n_web_ modules as following:\n\n```\n127.0.0.1 api.example.lo\n127.0.0.1 www.example.lo\n```\nTo test that the host names are properly setup, simply use ping as following:\n```shell\n$ ping api.example.lo\nPING api.example.lo (127.0.0.1): 56 data bytes\n64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.044 ms\n\n--- api.example.lo ping statistics ---\n1 packets transmitted, 1 packets received, 0.0% packet loss\nround-trip min/avg/max/stddev = 0.044/0.044/0.044/0.000 ms\n```\n```shell\n$ ping www.example.lo\nPING www.example.lo (127.0.0.1): 56 data bytes\n64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.045 ms\n\n--- www.example.lo ping statistics ---\n1 packets transmitted, 1 packets received, 0.0% packet loss\nround-trip min/avg/max/stddev = 0.045/0.045/0.045/0.000 ms\n$\n```\n\nWe assume that git is running on the localhost as a server, and that the\nfollowing repositories already exist:\n* /var/git/example/arango.git, for the arango module\n* /var/git/example/server.git, for the server module\n* /var/git/example/web.git, for the web module\n\n## Creating the git repo for the development environment\n\n\u003e These steps are only required when setting up the development workflow fo the\n\u003e first time. Once it's done, developers being onboarded on this workflow only\n\u003e need to start with 'Setting up the development environment'\n\nOn the git server (localhost), login as the git user and create a new bare\nrepository with the name 'development-local.git'\n```shell\n$ git init --bare /var/git/example/development-local.git\nInitialized empty Git repository in /var/git/example/development-local.git/\n```\nWe will work on that repo to finish its setup from a regular user on her local\nenvironment.\n\nWe'll clone the development-local git repo, add the various application modules\nto it, populate it with the docker files we need to configure and manage the\nlifecycle of the different docker containers used for the testing.\n\n```shell\n$ git clone git@localhost:/var/git/example/development-local.git development-local\nCloning into 'development-local'...\nwarning: You appear to have cloned an empty repository.\nChecking connectivity... done.\n$ cd development-local\n$ touch README.md\n$ git add README.md\n$ git commit -m \"Empty README.md\"\n$ git push\n```\nWe have just added an empty README.md file to the development-local repo and\npushed the commit to the remote. The development repo should be populated with\nat least the basic workflow documentation. You can use this README.md file and\nadapt as necessary for your own needs.\n\n## Adding submodules\n\nLet's add the submodules...\n```shell\n$ git submodule add git@localhost:/var/git/example/arango.git\n$ git submodule add git@localhost:/var/git/example/server.git\n$ git submodule add git@localhost:/var/git/example/web.git\n```\nThis will have created a .gitmodules in the development-local repository so\nthat future developers will be able to fetch the submodules while fetching the\ndevelopment repo.\n\nBy default, submodules will add the subproject into a directory named the same\nas the repository, in this case “DbConnector”. You can add a different path at\nthe end of the command if you want it to go elsewhere.\n\nFinally we can commit the changes and push to the remote (either using the\npreviously defined alias `git spush` or simply `git push` as we did not make\nany changes to the submodules after adding them).\n\n```shell\n$ git commit -m \"Added submodules\"\n$ git spush\n```\n\n## Removing a submodule\nThere are two situations where you’d want to “remove” a submodule:\n* You just want to clear the working directory (perhaps before archiving the\n  container’s WD) but want to retain the possibility of restoring it later (so\n  it has to remain in .gitmodules and .git/modules);\n* You wish to definitively remove it from the current branch.\n\nLet’s see each case in turn.\n\n### Temporarily removing a submodule\nThe first situation is easily handled by git submodule deinit.\n```shell\n$ git submodule deinit web\nCleared directory 'web'\nSubmodule 'web' (git@localhost:/var/git/example/web.git) unregistered for path 'web'\n```\nThis has no impact on the container status whatsoever. The submodule is not\nlocally known anymore (it’s gone from .git/config), so its absence from the\nworking directory goes unnoticed. We still have the vendor/plugins/demo directory\nbut it’s empty; we could strip it with no consequence.\n\nThe submodule must not have any local modifications when you do this, otherwise\nyou’d need to --force the call.\n\nAny later subcommand of git submodule will blissfully ignore this submodule until\nyou init it again, as the submodule won’t even be in local config. Such commands\ninclude update, foreach and sync.\n\nOn the other hand, the submodule remains defined in .gitmodules: an init followed\nby an update (or a single update --init) will restore it as new:\n\n```shell\n$ git submodule update --init web\nSubmodule 'web' (git@localhost:/var/git/example/web.git) registered for path 'web'\nSubmodule path 'web': checked out 'ab2e7566ad972378dd51e98dff74250a4ff58b28'\n```\n\n### Permanently removing a submodule\nThis means you want to get rid of the submodule for good: a regular git rm will\ndo, just like for any other part of the working directory. This will only work\nif your submodule uses a gitfile (a .git which is a file, not a directory),\nwhich is the case starting with Git 1.7.8.\n\nIn addition to stripping the submodule from the working directory, the command\nwill update the .gitmodules file so it does not reference the submodule anymore.\n\nFor a comprehensive cleanup, it is recommended to do both in sequence, first the\n`deinit` then the `rm` so as not to end up with the module still in the local\nconfig.\n\n```shell\n$ git submodule deinit web\n$ git rm web\n$ git commit -m \"Removed web module\"\n$ git push\n```\nRegardless of the approach, the submodule’s repo remains present in\n.git/modules/web. If you need to add the module back again, you need to kill\nthat before.\n\n## Working with container and submodules\n\n### Setting up the development environment as a developer\n\nTo get quickly setup as a developer, you simply need to do the following\ninstructions (after ensuring the prerequisites and additional configuration for\ngit is done):\n\n```shell\n$ git clone --recursive git@localhost:/var/git/example/development-local.git development-local\n```\nThis will have the effect of cloning the development-local repo, initializing\n(`git submodule init`) and updating (`git submodule update`) the enclosed\nsubmodules, all in one go.\n\n```\ndevelopment-local\n├── README.md\n├── arango\n│   └── README.md\n├── server\n│   └── README.md\n└── web\n    └── README.md\n```\n\nSince Git 1.7.8, submodules use a simple .git file with a single gitdir: line\nmentioning a relative path to the actual repo folder, now located inside the\ncontainer’s .git/modules. This is mostly useful when the container has branches\nthat don’t have the submodule at all: this avoid having to scrap the submodule’s\nrepo when switching to such a container branch.\n```shell\n$ cd web\n$ ls -la\ntotal 8\ndrwxr-xr-x  4 abdessattar  staff  136 Jul 16 14:47 .\ndrwxr-xr-x  8 abdessattar  staff  272 Jul 16 14:47 ..\n-rw-r--r--  1 abdessattar  staff   84 Jul 16 14:47 .git\n-rw-r--r--  1 abdessattar  staff    0 Jul 16 14:47 README.md\n$ cat .git\ngitdir: /Users/abdessattar/Documents/example-dev/development-local/.git/modules/web\n```\n\nBe that as it may, the container and the submodule truly act as independent\nrepos: they each have their own history (log), status, diff, etc. Therefore be\nmindful of your current directory when reading your prompt or typing commands:\ndepending on whether you’re inside the submodule or outside of it, the context\nand impact of your commands differ drastically!\n\n\u003e The container and the submodule truly act as independent repos.\n\nFinally, the submodule commit referenced by the container is stored using its\nSHA1, not a volatile reference (such as a branch name). Because of this, a\nsubmodule does not automatically upgrade which is a blessing in disguise when\nit comes to reliability, maintenance and QA.\n\nBecause of this, most of the time a submodule is in detached head state inside\nits containers, as it’s updated by checking out a SHA1 (regardless of whether\nthat commit is the branch tip at that time).\n```shell\n$ cd web\n$ git status\nHEAD detached at ab2e756\nnothing to commit, working directory clean\n```\n\n### Working independently on a module\n\nIt is perfectly fine to work on a module independently of the development setup.\nWe will see later how we can grab the updated made to a submodule from its\nremote and integrate them into the development container.\n\nLet's assume a developer needs to add a node.js basic application scaffolding\ninto the server submodule. The updated will be done independently on the server\nrepo.\n\n```shell\n$ git clone git@localhost:/var/git/example/server.git server\n```\n\nAdd an `index.js` file:\n```js\n// index.js\nvar express = require('express');\nvar app = express();\n\napp.get('/hello', function (req, res) {\n  res.send('Hello World from node.js!');\n});\n\napp.listen(3000, function () {\n  console.log('Example app listening on port 3000!');\n});\n```\n\nAdd a `package.json` file:\n```json\n# package.json\n{\n  \"name\": \"example\",\n  \"version\": \"1.0.0\",\n  \"description\": \"Example application server\",\n  \"main\": \"index.js\",\n  \"scripts\": {\n    \"start\": \"node index.js\"\n  },\n  \"repository\": {\n    \"type\": \"git\",\n    \"url\": \"git@localhost:/var/git/example/server.git\"\n  },\n  \"dependencies\": {\n    \"express\": \"^4.14.0\"\n  }\n}\n```\n\nAdd `.gitignore` file:\n```\nnode_modules\n```\n\nCommit updates and push to remote:\n```shell\n$ git add -all\n$ git commit -m \"Added simple hello world node/express app\"\n$ git push\n\n-\u003e The rsulting history tree would like like this:\n\n(10 seconds ago)    * 03d50da  (HEAD -\u003e master, origin/master, origin/HEAD)\n                    |\n   (3 hours ago)    * ab2e756\n```\n\n### Getting an update from the submodule’s remote\n\nLet's assume now that a second developer is setting up a full development\nenvironment will all submodules.\n\n```shell\n$ git clone --recursive git@localhost:/var/git/example/development-local.git development-local\n\n$ cd development-local\n$ git status\nOn branch master\nYour branch is up-to-date with 'origin/master'.\nnothing to commit, working directory clean\n\n$ git submodule status\n3a16594c9272c2c521801bac93d3ac958deca336 arango (heads/master)\nab2e7566ad972378dd51e98dff74250a4ff58b28 server (ab2e756)\nab2e7566ad972378dd51e98dff74250a4ff58b28 web (heads/master)\n\n$ cd server\n$ git status\nHEAD detached at ab2e756\nnothing to commit, working directory clean\n```\n\nNotice that the server module is in a detached HEAD mode and no longer tracking\nthe remote master because it has been modified on the remote and the container\nonly includes the submodule through a specific commit SHA1 (ab2e756).\n\nSuppose we want to grab the latest commits made on the remote/master in our\nlocal submodule.\n\n\u003e On a side note, I would not recommend using pull for this kind of update.\n\u003e To properly get the updates in the working directory, this command requires\n\u003e that you’re on the proper active branch, which you usually aren’t (you’re on\n\u003e a detached head most of the time). You’d have to start with a checkout of\n\u003e that branch. But more importantly, the remote branch could very well have\n\u003e moved further ahead since the commit you want to set on, and a pull would\n\u003e inject commits you may not want in your local codebase.\n\nTherefore, I recommend splitting the process manually: first git fetch to get\nall new data from the remote in local cache, then log to verify what you have\nand checkout on the desired SHA1. In addition to finer-grained control, this\napproach has the added benefit of working regardless of your current state\n(active branch or detached head).\n\n```shell\n$ git fetch\n\n$ git log --oneline origin/master\n03d50da Added simple hello world node/express app\nab2e756 Empty README.md\n\n$ git checkout 03d50da\nPrevious HEAD position was ab2e756... Empty README.md\nHEAD is now at 03d50da... Added simple hello world node/express app\n\n$ ls -1\nREADME.md\nindex.js\npackage.json\n```\nAs a result of the above, the parent container is modified and we can see the\ndetailed status as below:\n```shell\n$ cd -\n$ git status\nOn branch master\nYour branch is up-to-date with 'origin/master'.\nChanges not staged for commit:\n  (use \"git add \u003cfile\u003e...\" to update what will be committed)\n  (use \"git checkout -- \u003cfile\u003e...\" to discard changes in working directory)\n\n\tmodified:   server (new commits)\n\nSubmodules changed but not updated:\n\n* server ab2e756...03d50da (1):\n  \u003e Added simple hello world node/express app\n\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n```\n\nWe now only need to perform the container commit that finalizes our submodule’s\nupdate.\n\n\u003e If you had to touch the container’s code to make it work with this update,\n\u003e commit it along, naturally. On the other hand, avoid mixing submodule-related\n\u003e changes and other stuff that would just pertain to the container code: by\n\u003e neatly separating the two, later migrations to other code-reuse approaches\n\u003e are made easier (also, as usual, atomic commits FTW).\n\n```shell\n$ git commit -am \"Moved server submodule to 03d50da\"\n$ git push\n```\n\n### Making updates to the container\n\nMaking changes and committing/pushing them within the container and not the\nsubmodules is starightforward. It is however recommended to keep such updates\nseparate from submodule version changes and updates for the sake of better\ncode management.\n\nLet's update the container with an enhanced REDME.md file that contains detailed\ninstructions and push the update to the remote.\n\n```shell\n$ vim README.md\n$ git status\nOn branch master\nYour branch is up-to-date with 'origin/master'.\nChanges not staged for commit:\n  (use \"git add \u003cfile\u003e...\" to update what will be committed)\n  (use \"git checkout -- \u003cfile\u003e...\" to discard changes in working directory)\n\n\tmodified:   README.md\n\nno changes added to commit (use \"git add\" and/or \"git commit -a\")\n\n$ git commit -am \"Updated README.md\"\n$ git push\n```\n\n### Grabbing container updates\n\nWhen pulling updates from the conatiner repo's remote, **Git auto-fetches, but\ndoes not auto-update**. The local cache is up-to-date with the submodule’s\nremote, but the submodule’s working directory stays with its former contents.\nAlthough this auto-fetching is limited to already-known submodules: any new\nones, not yet copied into local configuration, are not auto-fetched.\n\n\u003e If you don’t explicitly update the submodule’s working directory, your next\n\u003e container commit will regress the submodule. Is is therefore mandatory that\n\u003e you finalize the update.\n\n```shell\n$ git pull\n$ git submodule sync --recursive\n$ git submodule update --init --recursive\n```\n\n### Updating a submodule in-place in the container\n\nThis scenario should be limited to the cases where a submodule cannot be\ncompiled or tested outside the container. Otherwise it is preferred to modify\nsubmodules within their independent repos and pull the updates in the container\nafterwards.\n\nLet's update the web submodule to add a simple index.html page.\n\nFirst, we need to make sure we are starting from a clean start, which means the\ntip of a branch. Let's assume we can add on top of the current submodule master\nbranch without breaking the rest of the container and its submodules (if the\nrisk of breaking the container happens often, it probably means that embedding\nthis code as a submodule was not a good design architectural choice, and it maybe\nmore appropriate to just embed its code inline in the container or somewhere else).\n\n```shell\n$ cd web\n$ git checkout master\n$ git pull --rebase\n```\n\nWe can now edit the code, test it etc. then perform the **two commits and the\ntwo necessary pushes** to get the updates to the remote so other developers\ncan see them them and grab them.\n```shell\n$ git add index.html\n$ git status -s\nA  index.html\n```\n\nCommit the submodule updates:\n```shell\n$ git commit -am \"Added simple index.html\"\n```\n\nCommit the container to move the submodule to the new version:\n```shell\n$ cd ..\n$ git commit -am \"Moving submodule web to 7ff5ca7\"\n```\n\nFinally, push the container and the submodule either in one go with\n`git push --recurse-submodules=on-demand` or by doing two `push` operations,\none in the submodule and one in the container.\n```shell\n$ git push --recurse-submodules=on-demand\n```\n\n## Dockerizing everything\n\n### Quick start\n\nAll modules are dockerized. To get started with the testing, clone the\ndevelopment repo as described in '[Setting up the development environment as a developer](#setting-up-the-development-environment-as-a-developer)'\nand do this from the development repo root:\n\n```shell\n$ cd server \u0026\u0026 npm install \u0026\u0026 npm run build \u0026\u0026 cd ..\n$ cd arango \u0026\u0026 npm install \u0026\u0026 cd ..\n$ docker-compose -p example up\n```\n\nTo tear down the docker containers, simply do this from the development\nrepo root:\n```shell\n$ docker-compose -p example down\n```\n\n\u003e It is important to use `docker-compose -p \u003cproject-name\u003e` to make sure\n\u003e docker-compose does not prepend the current directory name to all containers\n\u003e it creates. We want those container names to be predictable as they are used\n\u003e in the tools within the submodules.\n\n### How it's done\n\nDocker containers enable us to automate the deployment of applications or\napplication modules inside lightweight containers. No more spending hours or\ndays making sure all dependencies with correct versions are downloaded and\ninstalled. No more problems due to environment pollution on the test systems.\nSimply pull the right container, deploy it and run it.\n\nWith this docker-compose comes to play as a simple but straightforward\ntool to define and run multi-container applications, providing basic\norchestration capabilities.\n\nThere are certainly many more sophisticated tools and platforms, such as Apache\nMesos, Kubernetes, etc... to manage complex environments and orchestrate large\nclusters of multi-container applications. But we are here for the main purpose\nof efficiently setting up a development environment and supporting our\nmulti-module development workflow with the minimal upfront requirements.\n\nAs we will see, docker and docker-compose will allow us to build everything we\nneed and get supported our workflow with simply a docker-compose descriptor\nfile. That in itself is amazing!\n\nLet's recall the overall application architecture:\n* We mainly provide a set of REST APIs served out of our node.js/express\n  application server, located in the `server` module.\n* Our `server` modules relies on a backend database, implemented using ArangoDB.\n  The choice of ArangoDB is motivated by its versatility to support document\n  storage and graph databases. Any other alternate database that satisfies the\n  need may be used, and a docker container would most likely be available of\n  docker hub. The scripts and optional Foxx microservices files for ArangoDB are\n  hosted in our `arango` module.\n* We also provide a nice web site for our app, served by nginx our of the `web`\n  module. The web site being fully static or using dynamic pages would not\n  impact at all our deployment architecture. We simply would need to augment\n  the container with the required additional capabilities.\n* Finally, to protect our application modules and enable us to leverage all the\n  goodies that can come with `nginx`, we front all our external interfaces with\n  a reverse proxy. There is no special module in our application repo dor that.\n  The whole thing can be implemented within the nginx container definition.\n  Should we need to add some artificats in the future to support enhancements\n  to our nginx reverse proxy, such as certificates and keys for TLS, we simply\n  need to create a new application module and add it to our development repo.\n\nThe required containers and their resources are summarized in the table below:\n\n| Container | Main function                                  | Baseline Docker Container                                                                              | Volumes                                                                                         | Ports     |\n|-----------|------------------------------------------------|--------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|-----------|\n| arangodb  | backend document and graph database            | Official container image from docker hub arangodb/arangodb                                             | data: docker volume apps: optional Foxx microservices source code, embedded in arango submodule | 8529:8529 |\n| server    | REST API and business logic application server | Official stable release of node.js from docker hub node:argon                                          | app: mapped from the server submodule                                                           | 3000:3000 |\n| web       | public web site for the application            | Alpine nginx container image from docker hub evild/alpine-nginx                                        | site: mapped from the web submodule config: mapped from the web submodule                       | 8000:80   |\n| nginx     | reverse proxy                                  | The amazing auto-configured reverse proxy container based on nginx from docker hub jwilder/nginx-proxy | internal mapping needed for the docker.sock, not related to our application modules             | 80:80     |\n\n\u003e NOTE: docker-compose.yml uses version 2 configuration format to get access to\n\u003e the volume declarations and to get a cleaner overall descriptor.\n\n#### arangodb\n\nA local volume created and managed by docker is used for the database storage,\nwhile we use our arango submodule to provide ArangoDB with the application\nsource code for Foxx microservices.\n\n```yml\narangodb:\n  image: arangodb/arangodb\n  environment:\n    - ARANGO_NO_AUTH=1\n  ports:\n    - \"8529:8529\"\n  volumes:\n    - arangodb-data:/var/lib/arangodb3/\n    - arangodb-apps:/var/lib/arangodb3-apps/\n    - ./arango/foxx:/var/lib/example-apps/:ro\n    - ./arango/scripts/:/var/lib/arangodb3-scripts/:ro\n```\n\n#### server\n\nWe need to link this container to the `arangodb` container for our application\nserver to be able to communicate with the database via the exposed ports.\n\n```yml\nserver:\n  image: node:argon\n  volumes:\n    - ./server/:/server/\n  working_dir: /server/\n  command: npm start\n  environment:\n    - VIRTUAL_HOST=api.example.lo\n    - VIRTUAL_PORT=3000\n  ports:\n    - \"3000:3000\"\n  links:\n    - arangodb\n```\n\n#### web\n\nWe don't want to override the entire nginx configuration filesystem in the\ncontainer, so we only mount volumes for the `nginx.conf` file and for the\n`conf.d/` sub-folder.\n\n```yml\nweb:\n  image: evild/alpine-nginx\n  volumes:\n    - ./web/public/:/usr/share/nginx/html/:ro\n    - ./web/nginx/nginx.conf:/etc/nginx/nginx.conf:ro\n    - ./web/nginx/conf.d/:/etc/nginx/conf.d/:ro\n  environment:\n    - VIRTUAL_HOST=www.example.lo\n    - VIRTUAL_PORT=80\n  ports:\n    - \"8000:80\"\n```\n\n#### nginx - reverse proxy and router\n\nJ. Wilder's nginx reverse proxy container provides an easy and straightforward\nmethod to automatically and dynamically configure all our containers for\nreverse proxying by nginx. To mark a container for configuration inside nginx,\nyou simply add the environment variables `VIRTUAL_HOST` and `VIRTUAL_PORT` to\nits container definition.\n\nFor our application, we have the following needs:\n* Everything coming to `app.example.lo` should be served by the node.js/express\n  inside the `server` container.\n* Everything coming to `www.example.lo` should be served by the nginx web server\n  inside the `web` container.\n\n\u003e The virtual hosts are locally defined in the machine's /etc/hosts file for\n\u003e the purpose of our development workflow. In production they can be replaced\n\u003e with the real hosts/domain.\n\nJ. Wilder's automated proxy container can do a lot more and can be extensiely\ncustomized to support complex configurations of nginx, TLS, multiple virtual\nhosts, etc... For reference, the full documentation is available at\nhttps://github.com/jwilder/nginx-proxy.\n\n```yml\nnginx:\n  image: jwilder/nginx-proxy\n  volumes:\n    - /var/run/docker.sock:/tmp/docker.sock\n  ports:\n    - \"80:80\"\n```\n\n## Conclusion\n\nWith a fully working developer workflow, supported by git and docker, we can\nget a full developer team or a single developer started in a matter of minutes\nrather than days. Moreover, with the flexibility that is offered by docker\nand the power of git submodules, we can easily adapt the application sturctures\nto add more modules.\n\nMicroservices architectures, coupled with containers and container composition\nare powerful software architecture tools to build powerful systems while\nsignificantly reducing the complexity overhead on the development workflows.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabdes%2Fsubmodule-docker-dev-workflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabdes%2Fsubmodule-docker-dev-workflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabdes%2Fsubmodule-docker-dev-workflow/lists"}