{"id":17328189,"url":"https://github.com/charypar/monobuild","last_synced_at":"2025-03-22T19:33:12.196Z","repository":{"id":32274243,"uuid":"131838835","full_name":"charypar/monobuild","owner":"charypar","description":"A build orchestration tool for Continuous Integration in a monorepo.","archived":false,"fork":false,"pushed_at":"2022-06-17T01:35:02.000Z","size":157,"stargazers_count":27,"open_issues_count":7,"forks_count":5,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-10-16T14:22:44.817Z","etag":null,"topics":["build-tool","dependency-graph","monorepo"],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/charypar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-05-02T11:05:33.000Z","updated_at":"2023-11-05T10:25:05.000Z","dependencies_parsed_at":"2022-09-12T03:41:22.526Z","dependency_job_id":null,"html_url":"https://github.com/charypar/monobuild","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charypar%2Fmonobuild","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charypar%2Fmonobuild/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charypar%2Fmonobuild/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charypar%2Fmonobuild/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/charypar","download_url":"https://codeload.github.com/charypar/monobuild/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221832484,"owners_count":16888259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["build-tool","dependency-graph","monorepo"],"created_at":"2024-10-15T14:22:50.874Z","updated_at":"2024-10-28T13:35:06.837Z","avatar_url":"https://github.com/charypar.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Monobuild\n\nA build orchestration tool for Continuous Integration in a monorepo.\n\nNOTE: this is Readme driven development. Not everything described in this readme\nis fully implemented.\n\n## About\n\nMonobuild is a simple tool that understands a graph of dependencies in\na monorepo codebase (where separate components live side by side in folders)\nand based on it, it can decide what should be built, given a set of changes.\n\nFor help, run\n\n```sh\n$ monobuild help\n```\n\n### Versioning\n\nMonobuild uses [Immutable Versioning](https://imver.github.io) as its versioning\nscheme.\n\n## Usage\n\nMonobuild constructs the dependency graph from dependency manifests. By\ndefault, manifests are files named `Dependencies`, which contain a simple\nline-by-line list of dependencies for the component in the directory of the\nfile.\n\n### Declare dependencies\n\nAn example manifest in `app1/Dependencies` might look like this\n\n```\n# Content\n!data/content\n!shared/images\n\n# Libs\ncommon-lib\nlibs/date-time\n```\n\nMonobuild will ignore any empty lines and lines starting with `#`. Every other\nline is considered a dependency and is a path relative to current working\ndirectory (typically repository root). Monobuild will expect a dependency\nmanifest (possibly empty) to be present at that path.\n\nLines starting with `!` are strong dependencies all other dependencies are\nconsidered weak. The difference is in the way the dependency graph is translated\nto a build schedule.\n\nOne of the benefits of a monorepo, is components and services can be built from\ncode, including their dependencies. Changing a weak dependency of a component\nmeans a change to the component, which therefore needs to be rebuilt, but the\nbuilds can be run in parallel. Output or result of the dependency does not\naffect the build of this component.\n\nA strong dependency has to successfully build first, in order for the build of\nthe component to be possible. If the dependency build fails, the component\nbuild does not even start.\n\nTypically, services are built from source, including their libraries, so the\ndependencies on libraries are weak (we still want to run the library build to\nrun tests and get a result though). Deploying orchestrations of services\ntypically has a strong dependency on the service builds (as they produce\nartifacts, e.g. docker images, needed by the deployment).\n\n### Visualise dependency graph and build schedule\n\nTo better understand the dependency graphs and build schedules, Monobuild can\nprint them.\n\n```sh\n$ monobuild print\n```\n\nwill print the build schedule, which will ignore weak dependencies\n\n```sh\n$ cd test/fixtures/manifests-test\n$ monobuild print\napp4:\nlibs/lib1:\nlibs/lib2:\nlibs/lib3:\nstack1: app1, app2, app3\napp1:\napp2:\napp3:\n```\n\nYou can also print the dependency structure with one component per line. For example\n\n```sh\n$ cd test/fixtures/manifests-test\n$ monobuild print --dependencies\napp4:\nlibs/lib1: libs/lib3\nlibs/lib2: libs/lib3\nlibs/lib3:\nstack1: app1, app2, app3\napp1: libs/lib1, libs/lib2\napp2: libs/lib2, libs/lib3\napp3: libs/lib3\n```\n\nMonobuild can also print the entire dependency structure including whether the\ndependencies are weak or strong. This is useful when\n[working without a local repo](#working-without-a-local-repository)\n\n```\n$ monobuild print --full\napp4:\nlibs/lib1: libs/lib3\nlibs/lib2: libs/lib3\nlibs/lib3:\nstack1: !app1, !app2, !app3\napp1: libs/lib1, libs/lib2\napp2: libs/lib2, libs/lib3\napp3: libs/lib3\n```\n\n#### Graphical output\n\nPrint also supports graphical output using GraphViz\n\n```\n$ cd test/fixtures/manifests-test\n$ monobuild print --dot\n```\n\nto produce a PDF, you can pipe the output into the `dot` tool:\n\n```\n$ cd test/fixtures/manifests-test\n$ monobuild print --dependencies --dot | dot -Tpdf -o dependencies.pdf\ndigraph dependencies {\n  \"app1\" -\u003e \"libs/lib1\"\n  \"app1\" -\u003e \"libs/lib2\"\n  \"app2\" -\u003e \"libs/lib2\"\n  \"app2\" -\u003e \"libs/lib3\"\n  \"app3\" -\u003e \"libs/lib3\"\n  \"libs/lib1\" -\u003e \"libs/lib3\"\n  \"libs/lib2\" -\u003e \"libs/lib3\"\n  \"stack1\" -\u003e \"app1\"\n  \"stack1\" -\u003e \"app2\"\n  \"stack1\" -\u003e \"app3\"\n}\n```\n\n### Change detection\n\nIf the current directory is a git repository, monobuild can decide which\ncomponents changed (using git).\n\n```sh\n$ monobuild diff\napp2\napp2\nlib3\n```\n\nMonobuild assumes use of [Mainline Development](https://gitversion.readthedocs.io/en/latest/reference/mainline-development/)\nand changes are detected in two modes:\n\n1.  for a feature branch, the change detection is equivalent to\n\n    ```sh\n    $ git diff --no-commit-id --name-only -r $(git merge-base master HEAD)\n    ```\n\n    in other words, list all the changes that happened since the current branch\n    was cut from `master`.\n\n    This is the default mode and the base branch is `master` by default.\n    You can override this with\n\n    ```sh\n    $ monobuild diff --base-branch develop\n    ```\n\n2.  for a `master` branch (or other main branch) the change detection is equivalent\n    to\n\n    ```sh\n    $ git diff --no-commit-id --name-only -r HEAD^1\n    ```\n\n    To work in the main-branch mode, use the `--main-branch` flag\n\n    ```sh\n    $ monobuild diff --main-branch\n    ```\n\nThe main difference between the above `git diff`s and `monobuild diff` is the\ndependency graph awareness.\n\nMonobuild will start with the list from `git diff`, filter it down to known\ncomponents, and then extend it with all components that depend on any of the\ncomponents in the initial list, including transitive dependencies.\n\nFor the resulting \"to do\" list, `diff` will then build a build schedule using the\nstrong dependencies.\n\nYou can print the relevant part of the dependency graph (rather than\nthe build schedule) with `--dependencies`\n\n```\n$ monobuild diff --dependencies\n```\n\nBoth modes also support DOT output with `--dot`. You can also print\nthe entire graph with the affected components with `--dot-highlight`.\n\n#### Rebuilding strong dependencies\n\nThe assumption behind strong dependencies is that their outcome is required\nfor the dependent builds to proceed. In most cases, this means that if no\nchanges affected a component, the build does not need to run, because the outcome\n(e.g. a build artifact) already exists from a previous run of the build (when\nthat component was affected).\n\nIn certain situations, it could be useful to run the build again, to ensure its\noutput is present. This will result in wasted work, but ensures builds won't\nfail because, for example, an artifact cache was lost. The wasted work can\nalso largely be prevented by making builds idempotent.\n\nMonobuild supports this with an `--rebuild-strong` option on `diff`, which will\ninclude strong dependencies of all components affected by the change.\n\n### Override the manifest matching\n\nIf you want to use a different filename for the manifest files, you can do so\nusing the global `--manifests` flag.\n\n### Filters\n\n#### Scope\n\nYou can scope the results of both `diff` and `print` to a given component\nand its dependencies using the `--scope` flag\n\n#### Top-level components\n\nSometimes it's useful to know the \"entrypoints\" into your dependency graph -\nthe components that nothing depends on (typically services or applications).\nYou can list only those with a `--top-level` flag on both `diff` and `print`.\n\n### Creating a Makefile\n\n**not implemented**\n\nMonobuild can also generate a `Makefile`, that can be used by individual\ncomponent builds to build their dependencies.\n\nYou can generate the makefile with\n\n```sh\n$ monobuild makefile\n```\n\nThe resulting Makefile consists of targets like this:\n\n```make\ndirectory/component1: [dependency1] [dependency2] [dependency3]\n  @cd directory/component1 \u0026\u0026 make build\n```\n\nThis assumes each component has a minimal `Makefile` which looks like this:\n\n```make\n# directory/component1/Makefile\n\ndefault:\n  @cd ../.. \u0026\u0026 make directory/component1\n\nbuild:\n  # steps to make the component available as a dependency of others\n  # this could be empty\n```\n\nYou can also override the build command (`make build` by default) with the\n`--build-command` flag.\n\n## Working without a local repository\n\nMonobuild can work without a local clone and checkout of your repository. All of\nthe use can be supported with GitHub API calls.\n\nThis is an **optimisation** and may not be necessary on relatively small monorepos or\nif your CI caches the repo and only pulls changes since last build.\n\n### Load dependency graph from a file\n\nIt is possible to keep the dependencies in a central dependency graph file, which\nis useful for big repos, when you want to work without a full local checkout (e.g. on CI).\n\nMonobuild supports this by allowing a map file to be specified with the `-f` flag.\nFor example:\n\n```sh\n$ curl -O https://raw.githubusercontent.com/myorg/myrepo/blob/master/dependencies.monobuild\n$ mononobuild print -f ./dependencies.monobuild --dependencies\n```\n\nThe file can be created with the `print` command itself, which will collect the\nmanifests and produce the full map.\n\n```sh\n$ monobuild print --full\napp4:\nlibs/lib1: libs/lib3\nlibs/lib2: libs/lib3\nlibs/lib3:\nstack1: !app1, !app2, !app3\napp1: libs/lib1, libs/lib2\napp2: libs/lib2, libs/lib3\napp3: libs/lib3\n```\n\nYou can update the dependency map as follows (with a local checkout):\n\n```sh\n$ monobuild print --full \u003e dependencies.monobuild\n```\n\n### Read changed files from standard input\n\nSimilarly, the changed files for `diff` can be supplied externally, from\nstandard input, e.g.\n\n```\n$ git diff --no-commit-id --name-only -r $(git merge-base master HEAD) | monobuild diff -\n```\n\n**note the hyphen at the end**, indicating the diff should be taken from stdin,\ninstead of running the git command.\n\n### A full CI example\n\nA full example may look as follows:\n\n```\n# Fetch the dependency map\n$ curl -O https://raw.githubusercontent.com/myorg/myrepo/blob/master/dependencies.monobuild\n\n# Get a list of changed files remotely from Github API\n# (note the followig only works up to 300 files)\n$ curl -s -o changed-files https://api.github.com/repos/charypar/monobuild/pulls/21/files | jq -r .[].filename\n\n# Get a build schedule\n$ cat changed-files | monobuild diff -f dependencies.monobuild -\n```\n\nThis is complicated enough that I might wrap it in a small service which can\nrespond to a GitHub webhook, get the right information off of GitHub and then\ntrigger the right builds from a set of webhooks.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharypar%2Fmonobuild","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcharypar%2Fmonobuild","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharypar%2Fmonobuild/lists"}