{"id":13903258,"url":"https://github.com/enspirit/makefile-for-monorepos","last_synced_at":"2026-03-04T00:08:03.455Z","repository":{"id":42627081,"uuid":"392370032","full_name":"enspirit/makefile-for-monorepos","owner":"enspirit","description":"A flexible and extensible makefile for monorepos with  docker/docker-compose.","archived":false,"fork":false,"pushed_at":"2023-10-13T13:22:53.000Z","size":85,"stargazers_count":165,"open_issues_count":3,"forks_count":7,"subscribers_count":9,"default_branch":"master","last_synced_at":"2024-12-09T08:36:35.697Z","etag":null,"topics":["continuous-delivery","continuous-integration","docker","makefile"],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/enspirit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-08-03T15:51:51.000Z","updated_at":"2024-11-30T13:02:41.000Z","dependencies_parsed_at":"2023-10-13T15:22:30.569Z","dependency_job_id":null,"html_url":"https://github.com/enspirit/makefile-for-monorepos","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enspirit%2Fmakefile-for-monorepos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enspirit%2Fmakefile-for-monorepos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enspirit%2Fmakefile-for-monorepos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/enspirit%2Fmakefile-for-monorepos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/enspirit","download_url":"https://codeload.github.com/enspirit/makefile-for-monorepos/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230520394,"owners_count":18238948,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["continuous-delivery","continuous-integration","docker","makefile"],"created_at":"2024-08-06T22:01:57.392Z","updated_at":"2026-03-04T00:08:03.421Z","avatar_url":"https://github.com/enspirit.png","language":"Makefile","funding_links":[],"categories":["docker","Makefile"],"sub_categories":[],"readme":"# `make up` and go\n\nThis project provides a reusable Makefile for architectures with multiple software components that use docker extensively and are organized as monorepositories.\n\nEverything else apart from the Makefile is just us showcasing its usage and capabilities.\n\n:warning: This Makefile requires make \u003e= 3.82 :warning:\n\n## Quick showcase video\n\n[![Quick Showcase Video](https://img.youtube.com/vi/dvBKAQKuk2s/0.jpg)](http://www.youtube.com/watch?v=dvBKAQKuk2s)\n\n## Why?\n\nAt Enspirit we have embraced [docker](https://www.docker.com/), [docker-compose](https://docs.docker.com/compose/) and [monorepositories](https://en.wikipedia.org/wiki/Monorepo). Our team is usually rotating among multiple software projects.\n\nWe want to have reproducible builds and tooling allowing us to work the same way on all these different projects. We also want only one command to be needed for building then starting a software after a fresh git clone, even for newcomers.\n\nThis Makefile is an open-source consolidation of years of work to achieve that goal. Why don't you see for yourself:\n\n```bash\ngit clone git@github.com:enspirit/makefile-for-monorepos.git\ncd makefile-for-monorepos\n\nmake up\n```\n\n_n.b. You need *docker*, *docker-compose* and *make* (~\u003e 3.81) installed locally._\n\n## Features\n\n* it manages builds of all components' docker images\n* it allows managing inter-component dependencies\n* it only rebuilds images whose source code or dependencies have changed\n* it manages lifecycles of the various components (start, stop, restart, ...)\n* it pushes and pulls images from repositories\n* it provides extension points for standard rules (e.g. unit testing, cleaning, ...)\n* it also allows the extension of the Makefile with ad-hoc rules per component\n\n## Installation\n\nCopy our Makefile from this repo and place it in your own monorepository project. Your source code organization needs to follow the [conventions below](#conventions).\n\nWe all love those one-liner installation methods, so here you go:\n\n_n.b. make sure you are inside the folder of your monorepo project._\n\nOption 1, with wget:\n```bash\nwget https://raw.githubusercontent.com/enspirit/makefile-for-monorepos/1.0.3/Makefile\n```\n\nOption 2, with curl:\n```bash\ncurl https://raw.githubusercontent.com/enspirit/makefile-for-monorepos/1.0.3/Makefile -o Makefile\n```\n\n## Conventions\n\nBy following some conventions our Makefile adds some magic. Let's consider the file\nstructure of this very repository:\n\n```\nmonorepo\n├── base                       # A component used as a base for others (dependency)\n│   └── Dockerfile             # ... with its Dockerfile\n├── api                        # An api component\n│   ├── Dockerfile             # ... with its Dockerfile extending the base image\n│   ├── env                    #\n│   │   └── devel.env          # Default env vars for the devel environment\n│   ├── makefile.mk            # Extensions and ad-hoc rules for the component\n│   └── ...                    #\n├── frontend                   # Another component\n│   └── Dockerfile             #\n├── .env                       # Main environment variables (e.g. COMPOSE_FILE)\n├── docker-compose.base.yml    #\n├── docker-compose.devel.yml   # Orchestration with docker-compose files\n├── docker-compose.testing.yml #\n├── Makefile                   # Our reusable Makefile\n└── config.mk                  # Specific configuration and global ad-hoc rules\n```\n\nThe Makefile provides tooling organized in three layers:\n\n1. Builds: a folder at level one will be considered a *component* of the architecture as soon as it includes a Dockerfile. It is the case above for *base*, *api* and *frontend*. For all of them you magically get all the [component image rules](#per-component-image-rules) and [component test rules](#per-component-test-rules).\n\n2. Lifecycle: All services defined in the docker-compose files [currently enabled by the COMPOSE_FILE variable](https://docs.docker.com/compose/reference/envvars/#compose_file) automatically get the [component lifecycle rules](#per-component-lifecycle-rules).\n\n3. Extension: It provides 'magic' but it is still based on *make* so you can [configure or override](#configure-it-optional) things and even [extend](#extend-it) them globally or on a component basis using `config.mk` and `makefile.mk` files. This allows you to extend rules without changing the original Makefile so that you can get [bugfixes and improvements](#how-to-update)\n\nThe three layers are independent of each other. In particular the Builds layer only requires docker, so the build tooling works even if you use another orchestrator than docker-compose.\n\n## Configuration\n\nThe first time you run one of the Makefile's rules, it will create a config.mk where you can configure the name of your project (it defaults to your project's folder name). That name will be used as a prefix for all the subsequent images built.\n\nLet's take this repository as an example. [We've used \"monorepo\" as a project name](config.mk#L1) and we have 4 components: *api*, *base*, *frontend* and *tests*. The images built will be tagged monorepo/api:latest, monorepo/base:latest, etc...\n\nYou can override other things in your `config.mk` if our defaults are not to your taste. It's as simple as adding a line specifying which variable you want to override and providing its new value.\n\nFor instance to use your own private docker registry:\n```make\nPROJECT := monorepo\nDOCKER_REGISTRY := my.private.registry\n```\n\nHere is the list of variables you can override:\n\n```\n# Specify which docker tag is built (defaults to 'latest')\nDOCKER_TAG :=\n\n# Which command is used to build docker images (defaults to 'docker build')\nDOCKER_BUILD :=\n\n# Which command is used for docker-compose (defaults to 'docker-compose')\nDOCKER_COMPOSE :=\n\n# Docker build extra options for all builds (optional)\nDOCKER_BUILD_ARGS :=\n\n# Which command is used to scan docker images (defaults to 'docker scan')\nDOCKER_SCAN :=\n\n# Docker scan extra options (optional)\nDOCKER_SCAN_ARGS :=\n\n# Should docker scan fail on errors (true/false, defaults to 'true')\n# When running the `make images.scan` rule with this setting to true\n# the scan will stop at the first image with vulnerabilities\nDOCKER_SCAN_FAIL_ON_ERR :=\n```\n\nIt is important to note that these variables can also be overriden by exporting environment variables, e.g. `DOCKER_TAG=test make images`.\n\n## Extensions\n\nYou can add global rules in your `config.mk` and ad-hoc component rules in `makefile.mk` files.\n\nAs soon as you create one of them it will be included automatically. You can see an example of that in [tests/makefile.mk](tests/makefile.mk) where we created a `tests.run` rule that runs the unit tests for this project.\n\n*N.B. We recommend keeping all custom rules prefixed with the name of the component.*\n\n## Reference of available make rules\n\n### General image rules\n\n* `make clean`: removes the sentinel files (see [Sentinel files](#sentinel-files))\n* `make images`: builds all the docker images for the repo's components\n* `make images.push`: pushes all images to the docker registry (after building them if necessary)\n* `make images.pull`: pulls all images from the docker registry\n* `make images.scan`: scan all images for vulnerabilities\n\n### General lifecycle rules\n\n* `make up`: build images for components that have changed then force-starts the docker-compose project\n* `make down`: stops the docker-compose project\n* `make start`: starts the docker-compose project\n* `make restart`: restarts the docker-compose project\n* `make ps`: alias for docker-compose ps\n\n### General test rules\n\n* `make tests`: Runs all tests (equivalent to `tests.unit` then `tests.integration`)\n* `make tests.unit`: Runs all unit tests on all components\n* `make tests.integration`: Runs all integration tests on all components\n\n### Per-component image rules\n\nFor every docker component in your repo, you can run:\n\n* `make {component}.clean`: removes the component's sentinel files (see [Sentinel files](#sentinel-files))\n* `make {component}.image`: builds the component's docker image\n* `make {component}.image.pull`: pulls the component image from the docker registry\n* `make {component}.image.push`: pushes the image to the registry, :warning: it also rebuilds the component if any files or dependencies have changed\n* `make {component}.image.scan`: scans the component image for vulnerabilities\n\n### Per-component lifecycle rules\n\n* `make {component}.on`: starts the component\n* `make {component}.off`: stops the component\n* `make {component}.up`: forces the recreation of the container, :warning: it also rebuilds the component's image if any files or dependencies have changed\n* `make {component}.down`: stops the component\n* `make {component}.restart`: restarts the component (equivalent of `off` then `on`)\n* `make {component}.logs`: tails the logs of the component\n* `make {component}.bash`: gets a bash on the component\n\n`make {component}.up` and `make {component}.on` behave slightly differently: the former will first rebuild the image when needed (if files or dependencies have changed) while the latter simply starts the component using the last known image.\n\n`make {component}.down` and `make {component}.off` behave exactly the same way, they stop the component. It is just for  consistency: if we can start things with both `up` and `on` one would expect to have both `down` and `off`.\n\n### Per-component test rules\n\n* `make {component}.tests`: Runs all tests for the component (equivalent to `{component}.tests.unit` then `{component}.tests.integration`)\n* `make {component}.tests.unit`: Runs all the component's unit tests\n* `make {component}.tests.integration`: Runs all the component's integration tests\n\nThese three rules are placeholders. Their recipes must be implemented in the components' `makefile.mk`.\n\nFor example consider the [api/makefile.mk](api/makefile.mk#L4) file from this repository:\n```make\napi.tests.unit::\n\t@docker run monorepo/api npm run test:unit\n\napi.tests.integration::\n\t@docker run monorepo/api npm run test:integration\n```\n\n## Advanced use cases\n\n### Inter-component dependencies\n\nSometimes components depend on each other. There is an example of this in this repository: the *api* component depends on the *base* component [since its Dockerfile uses the base component as a base image](api/Dockerfile).\n\nThis can be expressed in the component's `makefile.mk` like we do in [api/makefile.mk](api/makefile.mk) by defining a variable `{component}_DEPS` that lists such dependencies.\n\nIn our example it means that *make* will know that the *base* component has to be built __before__ *api*. Not only that, but also any rebuild of *base* should retrigger a build of *api*.\n\n### Having a different shell per component\n\nThe `make {component}.bash` rule assumes that your component's image has `bash` installed. Not all of them do (e.g: [alpine](https://hub.docker.com/_/alpine), so the rule may not always work out of the box.\n\nYou can override the shell that is run by creating a `{component}_SHELL` override in the component's `makefile.mk`. [See our example](api/makefile.mk#L2).\n\n## Under the hood\n\n### Sentinel files\n\nFirst a little quote from [make's user manual](https://www.gnu.org/software/make/manual/html_node/Rules.html):\n\n\u003e A *rule* appears in the makefile and says when and how to remake certain files, called the rule's *targets* (most often only one per rule). It lists the other files that are the *prerequisites* of the target, and the *recipe* to use to create or update the target.\n\nNow let's look at a makefile such as:\n\n```make\ncomponent.image: component/Dockerfile\n  docker build -t component component/\n```\n\n`component.image` is the rule, `component/Dockerfile` is a prerequisite and the recipe is the second line with `docker build ...`. But what about the *target*?\n\nWell, since `docker build` does not produce any file we don't really have a real target here, it is what is called a [phony target](https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html#Phony-Targets).\n\nAnd the 'problem' with a phony target is that its recipe will be executed every time the rule is called. That is: the example makefile would re-run docker build every single time.\n\n\u003e What's the problem? docker build has a cache mechanism.\n\nYes and with [buildkit](https://docs.docker.com/develop/develop-images/build_enhancements/) it's getting even better.\n\nBut on monorepos with a lot of components we can save precious seconds by not re-running 'docker build' for components that haven't changed. That also allows us to rely on `make up` to decide which components need rebuildind before starting the docker-compose project.\n\n\u003e Ok, now what about those sentinels?\n\nSentinels are files that we can use to tell make when a rebuild is needed.\nLet's look at an example:\n\n```make\ncomponent.image: component/.built\n\ncomponent/.built: component/Dockerfile\n  docker build -t component component/\n  touch component/.built\n```\n\nHere as you can see we have now two rules: `component.image` and `component/.built`.\n\n`component.image` has one prerequisite: `component/.built`.\n`component/.built` also has one prerequisite: `component/Dockerfile`.\n\nYou can also see that the first rule does not have a recipe and that our second rule ['touches'](https://man7.org/linux/man-pages/man1/touch.1.html) the `component/.built` after building the docker image.\n\n\u003e Now what does this do?\n\nWell let's look again at some of make's user manual:\n\n\u003e A normal prerequisite makes two statements: first, it imposes an order in which recipes will be invoked: the recipes for all prerequisites of a target will be completed before the recipe for the target is run. Second, it imposes a dependency relationship: if any prerequisite is newer than the target, then the target is considered out-of-date and must be rebuilt.\n\nIn other terms this means that since the recipe that builds our docker image finishes by touching the sentinel file, make will know:\n* if the file is there, it doesn't have to build the image again\n* if our sentinel file is older than our Dockerfile (the prerequisite for our sentinel), then we do need a rebuild.\n\nOur Makefile is using this in the following ways:\n\n* `make {component}.image` uses a prerequisite: `.build/{component}/Dockerfile.built` (a sentinel file)\n* For every component we generate a `.build/{component}/Dockerfile.built` rule.\n* That rule produces the sentinel file after building the image, like in the example above.\n* When listing [inter component dependencies](#inter-component-dependencies) we generate additional prerequisites that use the dependencies' sentinels as prerequisites. This way we can ensure that our dependent images rebuild when their dependencies change.\n* `make {component}.push` is another example of such usage. It lists `.build/{component}/Dockerfile.pushed` as a prerequisite. The rule for it, in turn, lists `.build/{component}/Dockerfile.built` as a prerequisite. This means that if you already pushed your image and that none of its files (nor dependencies) have changed: there is no need to push it again.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fenspirit%2Fmakefile-for-monorepos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fenspirit%2Fmakefile-for-monorepos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fenspirit%2Fmakefile-for-monorepos/lists"}