{"id":15551713,"url":"https://github.com/bahmutov/double-docker","last_synced_at":"2025-07-18T03:01:54.318Z","repository":{"id":57215568,"uuid":"58669728","full_name":"bahmutov/double-docker","owner":"bahmutov","description":"Building NPM docker image in stages for faster NPM installs","archived":false,"fork":false,"pushed_at":"2016-05-26T17:22:23.000Z","size":44,"stargazers_count":6,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-01T13:48:58.904Z","etag":null,"topics":["development","docker","fast","npm"],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bahmutov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-05-12T19:12:45.000Z","updated_at":"2023-09-08T17:10:16.000Z","dependencies_parsed_at":"2022-08-26T13:31:36.123Z","dependency_job_id":null,"html_url":"https://github.com/bahmutov/double-docker","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bahmutov%2Fdouble-docker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bahmutov%2Fdouble-docker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bahmutov%2Fdouble-docker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bahmutov%2Fdouble-docker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bahmutov","download_url":"https://codeload.github.com/bahmutov/double-docker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252579978,"owners_count":21771247,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["development","docker","fast","npm"],"created_at":"2024-10-02T14:06:37.384Z","updated_at":"2025-05-05T21:33:34.858Z","avatar_url":"https://github.com/bahmutov.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# double-docker\n\n[![NPM][double-docker-icon] ][double-docker-url]\n\n[![Build status][double-docker-ci-image] ][double-docker-ci-url]\n[![semantic-release][semantic-image] ][semantic-url]\n\n\u003e NPM installs goes into the base docker image (based on SHA of package.json)\n\u003e Tests and builds are now both *repeatable and super fast*\n\n```\nlocal command                  Docker image\n\nnode \u003cversion\u003e          ----\u003e  alpine/mhart-\u003cversion\u003e\n                                     |\npackage.json \u003cSHASUM\u003e                V\nnpm install             ----\u003e  dd-deps-\u003cname\u003e-\u003cversion\u003e:\u003cSHASUM\u003e      clean, slow, rare\n                                     |\nsrc/                                 V\nnpm test                ----\u003e  dd-child-\u003cname\u003e-\u003cversion\u003e              FAST, often\n```\n\n## The problem\n\nThe CI should build your project in isolation, thus it should run `npm install` from an empty\nfolder. This takes some time. If we could separate the install from testing or building, we could\nrun the tests really quickly.\n\nWe also want to follow the same approach when working locally. For example \n[autochecker](https://github.com/victorbjelkholm/autochecker) tests your Node project\nin the separate Docker containers, yet does not run the `npm install` from the *clean folder*\non each build, instead relying on previous contents cache.\n\n## Solution\n\n1. Grab the SHA of `npm-shrinkwrap.json` or `package.json` file\n2. Build `\u003cproject-name-node-version-SHA\u003e` docker image with `npm install` inside\n   This docker image has the NPM dependencies installed and can be reused\n3. Build new docker image based on the previous image with the source code from the\n   current folder. The docker image runs `npm test` command.\n\nThat's it. You can work locally like this, or put the commands on CI and the builds become\nvery very fast.\n\nDon't forget to put `.dockerignore` file in the root of your project with `/node_modules`\nline to prevent copying the large folder for nothing.\n\n## Install and use\n\nOnce you have Docker on your local machine and CI, just add `double-docker` as your dev\ndependency\n\n```sh\nnpm i -D double-docker\n```\n\nCreate `.dockerignore` file in the root folder and at least ignore `node_modules` folder\n\n```sh\necho \"node_modules/\"\u003e.dockerignore\n```\n\nThen set the following script commands to call double-docker's aliases\n\n```json\n{\n  \"scripts\": {\n    \"build\": \"your current build command\",\n    \"test\": \"your current test command\",\n    \"dd-build\": \"dd-build\",\n    \"dd-test\": \"dd-test\",\n    \"dd-get\": \"dd-get dist\"\n  }\n}\n```\n\n`dd-test` will run the container and will call `npm test` with your code inside. \n`dd-build` will run `npm run build`.\n\nIf you want to test in a different version of Node just run\n\n```sh\nnpm run dd-test -- 4\n```\n\nOr add it to the `package.json` script command\n\n```json\n{\n  \"scripts\": {\n    \"dd-build\": \"dd-build 4\",\n    \"dd-test\": \"dd-test 4\"\n  }\n}\n```\n\nThe `dd-get` command copies the built folder / file from the finished Docker container\nto local file system. This is the way to grab the results of the isolated build.\n\n`dd-get \u003cfolder name\u003e \u003cnode version\u003e`\n\n## Additional scripts\n\nTo clear all containers and docker images run `dd-rm` command which runs the\n[utils/rm-docker-images.sh](utils/rm-docker-images.sh)\n\n## Custom Docker templates\n\nIf you have file named `DockerNpmDepsTemplate` in the current folder, it will be used\nto create the Docker base image. A good example is \n[test2/DockerNpmDepsTemplate](test2/DockerNpmDepsTemplate) file.\n\nIf you have file named `DockerTestTemplate` it will be used during the `dd-test` step.\nA good example is [test3/DockerTestTemplate](test3/DockerTestTemplate).\n\n**Important:** when using custom Docker template file, do not start with the `FROM` line.\nThe `FROM \u003cimage\u003e` line will be included automatically by the `double-docker`.\n\nWhen using the scripts, always assume the work directory is `WORKDIR /usr/src/app`\n\n## Rebuilding base image\n\nBy default, the base image is rebuilt if either `npm-shrinkwrap.json` or `package.json`\nSHA changes. You can provide list of filenames to use instead. For example, if you use\ncustom Docker template file, you probably want to rebuild if that file changes too.\n\n```json\n{\n  \"scripts\": {\n    \"dd-build\": \"dd-build 4 package.json DockerNpmDepsTemplate\",\n    \"dd-test\": \"dd-test 4 package.json DockerNpmDepsTemplate\"\n  }\n}\n```\n\nMake sure to add the list of filenames to both the build and the test script commands (if you\nhave two of them).\n\n## Examples\n\nSee test subfolders for examples. These examples use the commands via script file names,\nand not via NPM script aliases.\n\n* [test](test) - simple project with defaults (uses NPM script aliases)\n* [test2](test2) - custom NPM dependencies Docker file template example\n* [test3](test3) - custom test step Docker file template example\n\n## The timing\n\nInstalling NPM dependencies takes a long time. For example here, with\nexpress, babel and babel-cli\n\n```sh\n$ node -v\nv6.0.0\n$ npm -v\n3.8.6\n$ time npm install\nreal  0m19.302s\nuser  0m10.153s\nsys   0m2.957s\n```\n\nBuild locally once (which includes testing right now)\n\n```sh\n$ cd test\n$ time ./install-and-test.sh\n... package.json file has SHA 808de4d5ca619670e34b26d6e93bcaa7a4e2da81\n... builds double-docker-npm-deps:808de4d5ca619670e34b26d6e93bcaa7a4e2da81\n... with NPM dependencies\n... builds double-docker:0.12 on top of double-docker-npm-deps:808de4d ...\n... runs npm test in double-docker:0.12\nreal  0m28.460s\nuser  0m0.770s\nsys 0m0.346s\n```\n\nSo we lost 10 seconds because of building two Docker images the first time.\nBut any run after the first one is very fast\n\n```sh\n$ cd test\n$ time ./install-and-test.sh\n... first docker image with deps is already there, no changes\n... building second docker image by copying files\n... runs npm test in double-docker:0.12\nreal  0m1.264s\nuser  0m0.108s\nsys   0m0.053s\n```\n\nBut now clone the same repo somewhere else on the local machine. Without running\n`npm install` run tests.\n\n```sh\n$ git clone \u003crepo\u003e double-docker\n$ cd test-npm-layers/test\n$ time ./install-and-test.sh\n... pulls docker deps from local Docker registry\n... runs tests\nreal  0m0.941s\nuser  0m0.130s\nsys 0m0.073s\n```\n\nNice!\n\nWe could probably make it even faster by mounting the current working folder into the\ncontainer. But having an actual *built* container allows us one more trick - the tests\ncan be executed on *a separate machine*. Thus you can keep working locally and when running\nthe script, the built image could be pulled and executed from any other CI machine.\n\n## Removing extra containers and images\n\nCreated docker containers and images are prefixed with `dd-`.\nTo remove them, run the script [rm-docker-images.sh](rm-docker-images.sh)\n\n### Small print\n\nAuthor: Gleb Bahmutov \u0026copy; 2016\n\n* [@bahmutov](https://twitter.com/bahmutov)\n* [glebbahmutov.com](http://glebbahmutov.com)\n* [blog](http://glebbahmutov.com/blog/)\n\nLicense: MIT - do anything with the code, but don't blame me if it does not work.\n\nSpread the word: tweet, star on github, etc.\n\nSupport: if you find any problems with this module, email / tweet /\n[open issue](https://github.com/bahmutov/double-docker/issues) on Github\n\n## MIT License\n\nCopyright (c) 2016 Gleb Bahmutov\n\nPermission is hereby granted, free of charge, to any person\nobtaining a copy of this software and associated documentation\nfiles (the \"Software\"), to deal in the Software without\nrestriction, including without limitation the rights to use,\ncopy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the\nSoftware is furnished to do so, subject to the following\nconditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\nOF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\nHOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\nWHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\nFROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\nOTHER DEALINGS IN THE SOFTWARE.\n\n[double-docker-icon]: https://nodei.co/npm/double-docker.png?downloads=true\n[double-docker-url]: https://npmjs.org/package/double-docker\n[double-docker-ci-image]: https://travis-ci.org/bahmutov/double-docker.png?branch=master\n[double-docker-ci-url]: https://travis-ci.org/bahmutov/double-docker\n[semantic-image]: https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg\n[semantic-url]: https://github.com/semantic-release/semantic-release\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbahmutov%2Fdouble-docker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbahmutov%2Fdouble-docker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbahmutov%2Fdouble-docker/lists"}