{"id":25239256,"url":"https://github.com/seatgeek/docker-build-cacher","last_synced_at":"2025-10-26T14:30:42.312Z","repository":{"id":62435847,"uuid":"96434642","full_name":"seatgeek/docker-build-cacher","owner":"seatgeek","description":"Builds a service with docker and caches the intermediate stages","archived":false,"fork":false,"pushed_at":"2019-05-01T15:49:47.000Z","size":97,"stargazers_count":52,"open_issues_count":2,"forks_count":3,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-01-10T19:45:13.449Z","etag":null,"topics":["cli","docker","dockerfile"],"latest_commit_sha":null,"homepage":null,"language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seatgeek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-07-06T13:42:23.000Z","updated_at":"2024-12-21T07:22:54.000Z","dependencies_parsed_at":"2022-11-01T20:46:06.622Z","dependency_job_id":null,"html_url":"https://github.com/seatgeek/docker-build-cacher","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seatgeek%2Fdocker-build-cacher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seatgeek%2Fdocker-build-cacher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seatgeek%2Fdocker-build-cacher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seatgeek%2Fdocker-build-cacher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seatgeek","download_url":"https://codeload.github.com/seatgeek/docker-build-cacher/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238213343,"owners_count":19434960,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","docker","dockerfile"],"created_at":"2025-02-11T18:14:48.045Z","updated_at":"2025-10-26T14:30:41.973Z","avatar_url":"https://github.com/seatgeek.png","language":"Haskell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Docker Build Cacher\n\nThis tool is intended to speedup multi-stage Dockerfile build times by caching the results of each of the\nstages separately.\n\n## Why?\n\n[Multi-stage docker file](https://docs.docker.com/engine/userguide/eng-image/multistage-build/) builds are great,\nbut they still miss a key feature: It is not possible to carry from one build to another the statically generated\ncache files once the source file in your project change. Here's an example that illustrates the issue:\n\nImagine you create a generic Dockerfile for building node projects\n\n```Dockerfile\nFROM nodejs\n\nRUN apt-get install nodejs yarn\n\nWORKDIR /app\n\n# Whenever this image is used execute these triggers\nONBUILD ADD package.json yarn.lock .\nONBUILD RUN yarn\nONBUILD RUN yarn run dist\n```\n\nAnd then you call\n\n```bash\ndocker build -t nodejs-build .\n```\n\nSo now you can use the `nodejs-build` image in other builds, like this:\n\n```Dockerfile\n# Automatically build yarn dependencies\nFROM nodejs-build as nodedeps\n\n# Build the final container image\nFROM scratch\n\n# Copy the generated app.js from yarn run dist\nCOPY --from=nodedeps /app/app.js .\n...\n```\n\nSo far so good, we have build a pretty lean docker image that discards all the `node_modules`\nfolder and only keeps the final artifact. For example a bundled reactjs application.\n\nIt's also very fast to build! Since each of the steps in the Dockerfile are cached, as long as\nnone of the files changed.\n\nBut that's also where the problem is: Whenever `package.json` or `yarn.lock` files change, docker\nwill trash all the files in `node_modules` and all the cached yarn packages and will start from\nscratch downloading, linking and building every single dependency.\n\nThat's far from ideal. What if we could do a change in the process so that changes to those files\ndo not bust the yarn cache? It turns out that we can!\n\n## Enter docker-build-cacher\n\nThis utility overcomes the problem by providing a way to build the docker file and then cache the\nintermediate stages. On subsequent builds, it will make sure that the static cache files generated\nduring previous builds will also be present.\n\nThe effect it has should be obvious: your builds will be consistently fast, at the cost of more disk space.\n\n## Installation\n\nThere are binaries provided for `linux-x86_64` and MacOS, check\n[the releases page](https://github.com/seatgeek/docker-build-cacher/releases) for downloads.\n\n## How It Works\n\nThis works by parsing the Dockerfile and extracting the `COPY` or `ADD` instructions nested inside `ONBUILD` for each of\nthe stages found in the file.\n\nIt will compare the source files present in such `COPY` or `ADD` instructions to check for changes. If it can detect changes,\nit rewrites your Dockerfile on the fly so that the `FROM` directives in each of the stages use the locally cached images instead\nof the original base image.\n\nThe effect this `FROM` swap has, is that disk state for the image is preserved between builds.\n\n## Usage\n\n`docker-build-cacher` requires the following environment variables to be present in order to correctly build\nyour Dockerfile:\n\n* `APP_NAME`: The name for application you are trying to build. Usually this is just the folder name you are in.\n* `GIT_BRANCH`: The name of the git branch you are building. Used to \"namespace\" cache results\n* `DOCKER_TAG`: It will `docker build -t $DOCKER_TAG .` at some point. Let it know the image tag you want at the end.\n\nThis utility has two modes, `Build` and `Cache`. Both modes should be invoked for the cache to work:\n\n```bash\n# APP_NAME ispassed as argument in the build process, you can use it as an env var in your Dockerfile\nexport APP_NAME=fancyapp\n\n# GIT_BRANCH is used as part of the named for the resulting cached image\nexport GIT_BRANCH=master\n\n# DOCKER_TAG corresponds to the -t argument in docker build, that will be the resulting image name\nexport DOCKER_TAG=fancyapp:latest\n\ndocker-build-cacher build # This will build the docker file\ndocker-build-cacher cache # This will cache each of the stage results separately\n```\n\nAdditionally, `docker-build-cacher` accepts the `DOCKERFILE` env variable in case the file is not present in the\ncurrent directory:\n\n```bash\nDOCKERFILE=buildfiles/Dockerfile docker-build-cacher build\n```\n\nAt the end of the process you can call `docker images` and see that it has created `fancyapp:latest`, and if you are using\nmulti-stage builds, it should have created an image tag for each of the stages in your Dockerfile\n\n### Fallback Cache Keys\n\n\nAs mentioned before the `GIT_BRANCH` env variable is used as part of the name for the generated cached image, this means that\nthe generated cache is scope to that name. This is done so you can keep different caches where you can experiment with widly\ndifferent requirements and libraries in the dockerfile.\n\nThis has the unfortunate side effect that building other branches will require building the cache from scratch. In order to solve this\nyou can use the `FALLBACK_BRANCH` environment variable like this:\n\n```bash\nexport APP_NAME=fancyapp\nexport GIT_BRANCH=my-feature\nexport FALLBACK_BRANCH=master\nexport DOCKER_TAG=fancyapp:latest\n\ndocker-build-cacher build\ndocker-build-cacher cache\n```\n\nThe above will make the cached image for the `my-feature` branch to be based on the one from the `master` branch.\n\n### Caching Intermediate Images\n\nIn some circumstances, you may want to execute additional instructions after\nincluding the base builder image. For instance, building an executable or\nbundle using all the dependencies already downloaded:\n\n```Dockerfile\n# Automatically build haskell stack dependencies\nFROM haskell-stack as builder\n\nCOPY . .\nRUN stack install\n\n# Build the final container image\nFROM scratch\n\nCOPY --from=builder /root/.local/bin/my-app\n```\n\nThis very typical example has a shortcoming now, each time we do `COPY . .` we\nare also invalidating the compiling artifacts created in `stack install`, that\nis, we are losing the benefits of incremental compilation.\n\nIf you want to keep incremental compilation, or any files generated in between\nthe builder image and the final `FROM`, you can label the intermediate image so\nthat `docker-build-cacher` will include that into the cached artifacts:\n\n```Dockerfile\n# Automatically build haskell stack dependencies\nFROM haskell-stack as builder\n\n# Instructs the cacher to also copy the files generated in this stage\nLABEL cache_instructions=cache\n\nCOPY . .\nRUN stack install\n\n# Build the final container image\nFROM scratch\n\nCOPY --from=builder /root/.local/bin/my-app\n```\n\n**Warning:**\n\nThe files copied in `COPY . .` will also be cached! This not only increases the\ncache size, but also has a potentially dangerous inconvenient:\n\nAny files you delete from one build to the other will be restored again by the\ncacher. For example, if you delete one file in your source tree because you\ndon't use it anymore or you did a refactoring, it will pop up again in the build!\n\nThis may be a problem for compilers or build tools that scan all the files in\nthe folder, like the Go compiler. If you are certain that keeping old files\naround is not a problem, then it is safe to use this feature. The Haskell\ncompiler, for instance, does not care at all about extra cruft in the folder.\n\n## Passing extra arguments to docker build\n\nIt is possible to pass extra arguments and flags to the `docker build` step by providing the environment variable `DOCKER_BUILD_OPTIONS` as\nshown below:\n\n\n```bash\nDOCKER_BUILD_OPTIONS=\"--build-arg foo=bar --quiet\" docker-build-cacher build\n```\n\n## Building from source\n\nDependencies:\n\n- [Haskell stack](https://docs.haskellstack.org/en/stable/README/#how-to-install)\n\nInstall the `stack` tool from the link above. Then `cd` to the root folder of this repo and execute:\n\n```sh\nstack setup\nstack install\n```\n\nIf it is the first time, it will take *a lot* of time. Don't worry, it's only once you need to pay this price.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseatgeek%2Fdocker-build-cacher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseatgeek%2Fdocker-build-cacher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseatgeek%2Fdocker-build-cacher/lists"}