{"id":14384645,"url":"https://github.com/topheman/docker-experiments","last_synced_at":"2025-05-12T23:31:27.008Z","repository":{"id":145502965,"uuid":"144940632","full_name":"topheman/docker-experiments","owner":"topheman","description":"Discover docker with a simple use case in development, production (local kubernetes) and CI","archived":false,"fork":false,"pushed_at":"2018-11-05T11:17:20.000Z","size":1598,"stargazers_count":97,"open_issues_count":2,"forks_count":11,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-08-29T18:35:20.918Z","etag":null,"topics":["docker","docker-compose","kubernetes"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/topheman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-08-16T05:26:22.000Z","updated_at":"2024-03-09T14:00:56.000Z","dependencies_parsed_at":"2024-01-14T20:17:20.127Z","dependency_job_id":"ef72d305-de5e-4269-93f7-ce72d3eeee4f","html_url":"https://github.com/topheman/docker-experiments","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/topheman%2Fdocker-experiments","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/topheman%2Fdocker-experiments/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/topheman%2Fdocker-experiments/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/topheman%2Fdocker-experiments/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/topheman","download_url":"https://codeload.github.com/topheman/docker-experiments/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225158640,"owners_count":17430031,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","kubernetes"],"created_at":"2024-08-28T18:01:32.571Z","updated_at":"2024-11-18T10:24:42.803Z","avatar_url":"https://github.com/topheman.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# docker-experiments\n\n[![CircleCI](https://circleci.com/gh/topheman/docker-experiments/tree/master.svg?style=svg)](https://circleci.com/gh/topheman/docker-experiments)\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"./logo.png\" width=530 /\u003e\u003c/p\u003e\n\nThis started as a simple use case to discover `docker` and `docker-compose` 🐳 :\n\n* A [front](front) made with create-react-app, running in a nodejs container for development\n* A very simple [api](api) made in go (the challenge is also not to have everything in JavaScript)\n\nI also setup **deployments on a local kubernetes** ☸️ and tests are running on [CircleCI](https://circleci.com/gh/topheman/docker-experiments) on each push.\n\n## TL;DR\n\nYou are a true developer? You don't RTFM? After all, this is why we have docker ... not to bother with all the boring setup/install steps ... 😉\n\n```shell\ngit clone https://github.com/topheman/docker-experiments.git\ncd docker-experiments\ndocker-compose up -d\n```\n\nYou are good to go with a development server running at [http://localhost:3000](http://localhost:3000), with the front in react, the api in go and everything hot reloading. 👏\n\nTry to take a few minutes to read the doc bellow ... 😇\n\n## Summary\n\n* [Prerequisites](#prerequisites)\n* [Setup](#setup)\n* [Development 🛠](#development)\n* [Tests 🌡](#tests)\n* [Production - docker-compose 🐳](#production---docker-compose)\n* [Deployment - kubernetes ☸️](#deployment---kubernetes)\n* [Notes 📋](#notes)\n  * [Docker Multi-stage builds](#docker-multi-stage-builds)\n  * [Docker networks / Kubernetes services](#docker-networks--kubernetes-services)\n  * [Restart on failure](#restart-on-failure)\n  * [Commands](#commands)\n    * [Docker commands](#docker-commands)\n    * [Kubernetes commands](#kubernetes-commands)\n* [FAQ](#faq)\n  * [CircleCI](#circleci)\n    * [How to use latest version of docker-compose / docker-engine](#how-to-use-latest-version-of-docker-compose--docker-engine)\n    * [Docker vs Machine executors](#docker-vs-machine-executors)\n  * [Why does /api fallbacks to index.html in production](#why-does-api-fallbacks-to-indexhtml-in-production)\n* [What's next?](#whats-next)\n* [Resources](#resources)\n* [Author](#author)\n\n## Prerequisites\n\nYou need to have installed:\n\n* docker / docker-compose\n* npm / node (optional)\n* local kubernetes server and client (only if you want to play with kubernetes deployment - more about that on the [deployment section](#deployment---kubernetes))\n\n## Setup\n\n```shell\ngit clone https://github.com/topheman/docker-experiments.git\n```\n\nA [Makefile](Makefile) is available that automates all the commands that are described bellow. For each section, you'll find the related commands next to the 🖊 emoji.\n\nJust run `make help` to see the whole list.\n\n## Development\n\n### Launch development\n\n```shell\ndocker-compose up -d\n```\n\nThis will create (if not already done) and launch a whole development stack, based on [docker-compose.yml](docker-compose.yml), [docker-compose.override.yml](docker-compose.override.yml), [api/Dockerfile](api/Dockerfile) and [front/Dockerfile](front/Dockerfile) - following images:\n\n* `topheman/docker-experiments_front_development`: for react development (based on nodejs image)\n* `topheman/docker-experiments_api_development`: for golang in development mode (using [fresh](https://github.com/pilu/fresh) to build and restart the go webserver when you change the sources)\n  * The `services.api.command` entry in [docker-compose.override.yml](docker-compose.override.yml) will override the default `RUN` command and start a dev server (instead of running the binary compiled in the container at build time)\n\nGo to http://localhost:3000/ to access the frontend, you're good to go, the api is accessible at http://localhost:5000/.\n\n🖊 `make dev-start`, `make dev-start-d`, `make dev-stop`, `make dev-ps`, `make dev-logs`, `make dev-logs-front`, `make dev-logs-api`\n\n## Tests\n\n### Launch tests\n\n```shell\ndocker-compose run --rm -e CI=true front npm run -s test \u0026\u0026 docker-compose run --rm api go test -run ''\n```\n\n🖊 `make test`, `make test-front`, `make test-api`\n\n## Production - docker-compose\n\nThis section is about **testing the production images with docker-compose** 🐳 (check the [deployment section](#deployment---kubernetes) to deploy with kubernetes locally).\n\nMake sure you have built the frontend with `docker-compose run --rm front npm run build`, then:\n\n```shell\ndocker-compose -f ./docker-compose.yml -f ./docker-compose.prod.yml up --build\n```\n\nNote: make sure to use the `--build` flag so that it will rebuild the images if anything changed (in the source code or whatever), thanks to [docker images layers](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/), only changes will be rebuilt, based on cache (not the whole image).\n\nThis will create (if not already done) and launch a whole production stack:\n\n* No nodejs image (it should not be shipped to production, the development image is only used to launch the container that creates the build artefacts with create-react-app).\n* `topheman/docker-experiments_api_production`: for the golang server (with the app compiled) - containing only the binary of the golang app (that way the image)\n* `topheman/docker-experiments_nginx`: which will:\n  * serve the frontend (copied from `/front/build`)\n  * proxy `/api` requests to `http://api:5000` (the docker subnet exposed by the golang api container)\n\nAccess [http://localhost](http://localhost) and you're good to go.\n\n🖊 `make prod-start`, `make prod-start-d`, `make prod-start-no-rebuild`, `make prod-start-d-no-rebuild`, `make prod-stop`, `make prod-ps`, `make prod-logs`, `make prod-logs-front`, `make prod-logs-api`\n\n## Deployment - kubernetes\n\nThis section is about **deploying the app locally with kubernetes** ☸️ (not tested with a cloud provider). To stay simple, there aren't any TLS termination management (only port 80 exposed).\n\nLocal kubernetes server and client:\n\n* If you have the latest docker for Mac, both are [shipping with kubernetes](https://blog.docker.com/2018/01/docker-mac-kubernetes/)\n* Otherwise, you can use [minikube](https://github.com/kubernetes/minikube) for the server and install [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/) (kubernetes client)\n\nThe files descripting the deployments are stored in the [deployments](deployments) folder. You will find two files, each containing the deployment and the service.\n\n### Deploy with kubernetes\n\n1) If you haven't built the frontend, run `docker-compose run --rm front npm run build`\n\n2) Build the production images:\n\n```shell\ndocker build ./api -t topheman/docker-experiments_api_production:1.0.1\ndocker build . -f Dockerfile.prod -t topheman/docker-experiments_nginx:1.0.1\n```\n\nNote: They are tagged `1.0.1`, same version number as in the deployments files (want to put an other version number ? Don't forget to update the deployment files). For the moment, I'm not using [Helm](http://helm.readthedocs.io/en/latest/generate-and-template/) that let's you do string interpolation on yml files.\n\n3) Create your pods and services\n\nMake sure nothing is up on port `80`, then:\n\n```shell\nkubectl create -f ./deployments/api.yml -f ./deployments/front.yml\n```\n\nYou're good to go, check out [http://localhost](http://localhost)\n\nTo stop and delete the pods/services you created:\n\n```shell\nkubectl delete -f ./deployments/api.yml -f ./deployments/front.yml\n```\n\nThey won't stop right away, you can list them and see their status with:\n\n```shell\nkubectl get pods,services\n```\n\n[More commands](#kubernetes-commands)\n\n🖊 `make kube-start`, `make kube-start-no-rebuild`, `make kube-stop`, `make kube-ps`\n\n## Notes\n\n### Docker Multi-stage builds\n\nThanks to [docker multi-stage builds](https://docs.docker.com/develop/develop-images/multistage-build/), the golang application is built in a docker golang:alpine image (which contains all the tooling for golang such as compiler/libs ...) and produces a small image with only a binary in an alpine image (small Linux distrib).\n\nThe targets for multi-stage build are specified in the `docker*.yml` config files.\n\nThe [api/Dockerfile](api/Dockerfile) will create such a production image by default.\n\nYou can tell the difference of weight:\n\n```\ndocker images\ntopheman/docker-experiments_api_production      latest  01f1b575fae6  About a minute ago  11.5MB\ntopheman/docker-experiments_api_development     latest  fff1ef3ec29e  8 minutes ago       426MB\ntopheman/docker-experiments_front_development   latest  4ed3aea602ef  22 hours ago        225MB\n```\n\n### Docker networks / Kubernetes services\n\nIn development, the api server in golang is available at [http://localhost:5000](http://localhost:5000) and proxied onto [http://localhost:3000/api](http://localhost:3000/api) (the same port as the front, thanks to create-react-app [proxy](front/README.md#proxy)).\n\nIn **production** mode, we only want the golang server to be available via `/api` (we don't want to expose it on it's own port).\n\nTo make it work:\n\n* the docker-compose golang api service is named `api` - [see docker-compose.yml](docker-compose.yml).\n* the kubernetes services exposing the api is also named `api` - [see deployments/api.yml](deployments/api.yml)\n\nThat way, the nginx conf can work with both docker-compose AND kubernetes, proxying `http://api` - [see nginx/site.conf](nginx/site.conf).\n\n### Restart on failure\n\nIf your app exits with a failure code (greater than 0) inside the container, you'll want it to restart (like you would do with pm2 and node apps).\n\nWith [docker-compose/production](#production---docker-compose), the, directive `restart: on-failure` in the [docker-compose.yml](docker-compose.yml) file will ensure that. You'll be able to check it by clicking on the \"exit 1 the api server\" button, which will exit the golang api. You'll see that the uptime is back counting from 0 seconds.\n\nWith [kubernetes/deployment](#deployment---kubernetes), I setup 2 replicas of the api server, so when you retrieve the infos, the hostname might change according of the api pod you're balance on.\n\nExiting one pod won't break the app, it will fallback on the remaining replica. If you exit the two pods, you'll get an error retrieving infos, until one of the pod is back up by kubernetes (check their status with `kubectl get pods`).\n\n### Commands\n\n#### Docker commands\n\n* `docker-compose run --rm front npm run test`: launch a front container in *development* mode and run tests\n* `docker-compose -f ./docker-compose.yml run --rm api \u003ccmd\u003e`: launch an api container in *production* mode and run `\u003ccmd\u003e`\n* `docker-compose down`: stop and remove containers, networks, volumes, and images created by `docker-compose up`\n\nDon't want to use `docker-compose` (everything bellow is already specified in the `docker*.yml` files - only dropping to remember the syntax for the futur) ?\n\n* `docker build ./api -t topheman/docker-experiments_api_production:1.0.1`: build the `api` and tag it as `topheman/docker-experiments_api_production:1.0.1` based on [api/Dockerfile](api/Dockerfile)\n* `docker run -d -p 5000:5000 topheman/docker-experiments_api_production:1.0.1`: runs the `topheman/docker-experiments_api_production:1.0.1` image previously created in daemon mode and exposes the ports\n* `docker build ./front -t topheman/docker-experiments_front_development:1.0.1`: build the `front` and tag it as `topheman/docker-experiments_front_development:1.0.1` based on [front/Dockerfile](front/Dockerfile)\n* `docker run --rm -p 3000:3000 -v $(pwd)/front:/usr/front -v front-deps:/usr/front/node_modules topheman/docker-experiments_front_development:1.0.1`:\n  * runs the `topheman/docker-experiments_front_development:1.0.1` image previously created in attach mode\n  * exposes the port 3000\n  * creates (if not exists) and bind the volumes\n  * the container will be removed once you kill the process (`--rm`)\n* `docker rmi $(docker images -q --filter=\"dangling=true\")`: remove dangling images (layers that have no more relationships to any tagged image. Tagged as \u003cnone\u003e, they no longer serve a purpose and consume disk space)\n\n#### Kubernetes commands\n\n[kubectl Cheat Sheet](https://kubernetes.io/docs/reference/kubectl/cheatsheet/)\n\n* `kubectl create -f ./deployments/api.yml -f ./deployments/front.yml`: creates the resources specified in the declaration files\n* `kubectl delete -f ./deployments/api.yml -f ./deployments/front.yml`: deletes resources specified in the declaration files\n* `kubectl scale --replicas=3 deployment/docker-experiments-api-deployment`: scales up the api through 3 pods\n\n## FAQ\n\n### CircleCI\n\n#### How to use latest version of docker-compose / docker-engine\n\nI had the following error on my [first build](https://circleci.com/gh/topheman/docker-experiments/2):\n\n\u003e ERROR: Version in \"./docker-compose.yml\" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version (\"2.0\", \"2.1\", \"3.0\", \"3.1\", \"3.2\") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.\n\u003e\n\u003e For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/\n\nThe reason was because I'm using **docker-compose file format v3.4**, which doesn't seem to be supported by the version of docker-engine used on the default setup of CircleCI - [see compatibility matrix](https://docs.docker.com/compose/compose-file/#compose-and-docker-compatibility-matrix).\n\nWith CircleCI, in **machine executor mode**, you can change/customize the image your VM will be running (by default: `circleci/classic:latest`) - see the [list of images available](https://circleci.com/docs/2.0/configuration-reference/#machine). I simply changed the image to use:\n\n```diff\nversion: 2\njobs:\n  build:\n-    machine: true\n+    machine:\n+      image: circleci/classic:201808-01\n```\n\nCheckout [.circleci/config.yml](.circleci/config.yml)\n\nNote: Why use docker-compose file format v3.4 ? To take advantage of the `target` attribute.\n\n#### Docker vs Machine executors\n\n\u003e You can not build Docker within Docker.\n\nTo build/push docker images, you have two solutions on CircleCI:\n\n* Use the [machine executor mode](https://circleci.com/docs/2.0/executor-types/#using-machine): your jobs will be run in a dedicated, ephemeral Virtual Machine (VM) - so, you can directly run docker inside\n* Use the [setup_remote_docker](https://circleci.com/docs/2.0/building-docker-images/#overview) key: a remote environment will be created, and your current primary container will be configured to use it. Then, any docker-related commands you use will be safely executed in this new environment\n\n### Why does /api fallbacks to index.html in production\n\n#### Service Worker\n\ncreate-react-app ships with a service worker by default which implementation is based on [sw-precache-webpack-plugin](https://github.com/goldhand/sw-precache-webpack-plugin) (a Webpack plugin that generates a service worker using [sw-precache](https://github.com/GoogleChromeLabs/sw-precache) that will cache webpack's bundles' emitted assets).\n\nIt means that a `service-worker.js` file will be created at build time, listing your public static assets that the service worker will cache using a [cache first](https://developers.google.com/web/fundamentals/instant-and-offline/offline-cookbook/#cache-falling-back-to-network) strategy (on a request for an asset, will first hit the service worker cache and serve it, then call the network and update the cache - this makes the app fast and offline-first).\n\nFrom the [create-react-app doc](front/README.cra.md#serving-apps-with-client-side-routing):\n\n\u003e On a production build, and in a browser that supports [service workers](https://developers.google.com/web/fundamentals/getting-started/primers/service-workers),\nthe service worker will automatically handle all navigation requests, like for\n`/todos/42` or `/api`, by serving the cached copy of your `index.html`. This\nservice worker navigation routing can be configured or disabled by\n[`ejecting`](front/README.cra.md#npm-run-eject) and then modifying the\n[`navigateFallback`](https://github.com/GoogleChrome/sw-precache#navigatefallback-string)\nand [`navigateFallbackWhitelist`](https://github.com/GoogleChrome/sw-precache#navigatefallbackwhitelist-arrayregexp)\noptions of the `SWPreachePlugin` [configuration](https://github.com/facebook/create-react-app/blob/master/packages/react-scripts/config/webpack.config.prod.js).\n\n## What's next?\n\nThe next thing that will be comming are:\n\n* [x] setup CI\n* [x] using [nginx](https://www.nginx.com/) as a reverse-proxy to:\n  * [x] serve the golang api which is in its own container on `/api`\n  * [x] make a build of the front and serve it at the root\n* [x] use [kubernetes](https://kubernetes.io/) to automate deployment\n  * [x] start by playing in local with [minikube](https://github.com/kubernetes/minikube)\n  * [ ] add [Helm](http://helm.readthedocs.io/en/latest/generate-and-template/) to do string interpolation in yaml description files ?\n* [ ] linting / formatting + pre-commit hooks\n  * [ ] add linting / formatting support (eslint/prettier) with advanced config for the JavaScript part\n  * [ ] back it up with pre-commit hooks (husky ?)\n  * The challenge being:\n    * any npm dependency is currently installed on a volume mounted inside the front container (not accessible by host)\n    * how to elegantly have lint/formatting task also running on host (for vscode plugins for example but also npm tasks like), without relying on global modules (this would be cheating 😉 + we should not assume anything about the computer our project is cloned on)\n    * how to elegantly share pre-commit hooks ? (using husky would mean an `npm install` at the root of the project)\n\n*This is still in progress*.\n\n## Resources\n\n* Docker\n  * 📺 💯 [Better understand containers by coding one from scratch by Liz Rice](https://twitter.com/topheman/status/1014936620309647361)\n  * 📺 [Create a Development Environment with Docker Compose by Mark Ranallo](https://www.youtube.com/watch?v=Soh2k8lCXCA)\n  * 📺 [Rapid Development With Docker Compose](https://www.youtube.com/watch?v=o6SScget37w)\n  * [Golang and Docker for development and production](https://medium.com/statuscode/golang-docker-for-development-and-production-ce3ad4e69673) - use [pilu/fresh](https://github.com/pilu/fresh) to rebuild on changes in development\n  * [Create the smallest and secured golang docker image based on scratch](https://medium.com/@chemidy/create-the-smallest-and-secured-golang-docker-image-based-on-scratch-4752223b7324)\n    * [docker-compose with multi-stage build target (official doc)](https://docs.docker.com/compose/compose-file/#target)\n* Kubernetes\n  * 📺 [Learn Kubernetes by CoderJourney](https://www.youtube.com/playlist?list=PLbG4OyfwIxjFE5Ban_n2JdGad4EDWmisR)\n    * [Source code](https://github.com/coderjourney/meal_plan)\n  * 📺 [Run Kubernetes Locally Using Minikube by CoderJourney](https://coderjourney.com/run-kubernetes-locally-using-minikube/)\n  * [Setup bash completion for `kubectl`](https://twitter.com/topheman/status/1022939077602156546)\n\nMore bookmarks from my research:\n\n* Docker\n  * [Generic Docker Makefile](https://github.com/mvanholsteijn/docker-makefile)\n  * [Awesome-docker - A curated list of Docker resources and projects](https://awesome-docker.netlify.com/)\n* Kubernetes\n  * [Kubernetes \u0026 Traefik 101— When Simplicity Matters](https://dev.to/geraldcroes/kubernetes--traefik-101-when-simplicity-matters-6k6)\n  * [Tutorial : Getting Started with Kubernetes with Docker on Mac](https://rominirani.com/tutorial-getting-started-with-kubernetes-with-docker-on-mac-7f58467203fd)\n  * [kubernetes/dashboard](https://github.com/kubernetes/dashboard)\n  * [Kubernetes Ingress](https://medium.com/@cashisclay/kubernetes-ingress-82aa960f658e)\n    * [Setting up Nginx Ingress on Kubernetes](https://hackernoon.com/setting-up-nginx-ingress-on-kubernetes-2b733d8d2f45)\n    * [Advanced kubernetes ingress](https://koudingspawn.de/advanced-ingress/)\n    * [Kubernetes NodePort vs LoadBalancer vs Ingress? When should I use what?](https://medium.com/google-cloud/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0)\n  * [Kubectl apply vs kubectl create?](https://stackoverflow.com/questions/47369351/kubectl-apply-vs-kubectl-create)\n  * [awesome-kubernetes - A curated list for awesome kubernetes sources](https://ramitsurana.github.io/awesome-kubernetes/)\n* [Tutoriel Linux : Makefile](https://youtu.be/2VV9FAQWHdw)\n\n## Author\n\n[Christophe Rosset](https://github.com/topheman)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftopheman%2Fdocker-experiments","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftopheman%2Fdocker-experiments","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftopheman%2Fdocker-experiments/lists"}