{"id":20773977,"url":"https://github.com/indoorvivants/sbt-commandmatrix","last_synced_at":"2025-04-30T15:09:47.116Z","repository":{"id":46678294,"uuid":"359869318","full_name":"indoorvivants/sbt-commandmatrix","owner":"indoorvivants","description":"Define commands for projectmatrix subsets to parallelise your CI builds","archived":false,"fork":false,"pushed_at":"2024-10-28T16:35:08.000Z","size":57,"stargazers_count":6,"open_issues_count":9,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-28T17:36:44.531Z","etag":null,"topics":["jvm","matrix","sbt-projectmatrix","scala"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/indoorvivants.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-20T15:43:37.000Z","updated_at":"2024-10-28T16:35:12.000Z","dependencies_parsed_at":"2024-10-28T16:37:43.185Z","dependency_job_id":null,"html_url":"https://github.com/indoorvivants/sbt-commandmatrix","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indoorvivants%2Fsbt-commandmatrix","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indoorvivants%2Fsbt-commandmatrix/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indoorvivants%2Fsbt-commandmatrix/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/indoorvivants%2Fsbt-commandmatrix/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/indoorvivants","download_url":"https://codeload.github.com/indoorvivants/sbt-commandmatrix/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225036882,"owners_count":17410941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jvm","matrix","sbt-projectmatrix","scala"],"created_at":"2024-11-17T12:27:53.649Z","updated_at":"2024-11-17T12:27:54.236Z","avatar_url":"https://github.com/indoorvivants.png","language":"Scala","readme":"# sbt-commandmatrix\n\n- [sbt-commandmatrix](#sbt-commandmatrix)\n    - [Installation](#installation)\n    - [Status (September 2021)](#status-september-2021)\n  - [Define commands to run matrix subsets](#define-commands-to-run-matrix-subsets)\n    - [Motivation](#motivation)\n    - [Proposal and Usage](#proposal-and-usage)\n    - [Aggregate commands and stubbing](#aggregate-commands-and-stubbing)\n  - [Whole matrix definition](#whole-matrix-definition)\n    - [Motivation](#motivation-1)\n    - [Proposition and Usage](#proposition-and-usage)\n\nThis plugin extends [sbt-projectmatrix](https://github.com/sbt/sbt-projectmatrix) plugin with two additional features:\n\n1. [Define SBT commands to run subsets of the\n   matrix](#define-commands-to-run-matrix-subsets), i.e. turning single\n   `test` command into `test-2_13-jvm`, `test-2_13-js`, etc., depending\n   on what dimensions your matrix has\n\n2. [A helper method to define the matrix](#whole-matrix-definition), configuring and controlling the holes\n\n### Installation\n\n[![sbt-commandmatrix Scala version support](https://index.scala-lang.org/indoorvivants/sbt-commandmatrix/sbt-commandmatrix/latest-by-scala-version.svg?targetType=Sbt)](https://index.scala-lang.org/indoorvivants/sbt-commandmatrix/sbt-commandmatrix)\n\nThis plugin only works with sbt-projectmatrix, so in your `project/plugins.sbt` you should have:\n\n```scala\naddSbtPlugin(\"com.eed3si9n\" % \"sbt-projectmatrix\" % \"0.8.0\")\naddSbtPlugin(\"com.indoorvivants\" % \"sbt-commandmatrix\" % \"\u003cVERSION\u003e\")\n```\n\nWhere the version can be picked up from the badge above.\n\n### Status (September 2021)\n\nActive, super experimental, any usage will be punished by weird errors and unpredictable behaviour.\n\n## Define commands to run matrix subsets\n\n### Motivation\n\nIt's not uncommon to encounter projects that publish for:\n\n1. multiple platforms, such as Scala Native, Scala.js and JVM\n2. multiple Scala versions, e.g. 2.13, 2.12, 3.0.x\n3. multiple custom axis - i.e. different versions of some dependency\n\nSubjectively, `sbt-projectmatrix` solves this problem really, really well. Projects such as [Weaver Test](https://github.com/disneystreaming/weaver-test/blob/master/build.sbt) and [Sttp](https://github.com/softwaremill/sttp/blob/master/build.sbt) manage huge project matrices with all of the variations listed above.\n\nIssues start when it comes to CI: because projectmatrix generates a project per set of axis values, you can't use `++2.13.5` style of cross building.\n\nThis limits the ability to parallelise jobs on CI, where matrix builds became the norm a while ago - CIs such as Travis and Github Actions support it.\n\n### Proposal and Usage\n\nA more involved demonstration of techniques below is in the example:\n\n* [build.sbt](https://github.com/indoorvivants/sbt-commandmatrix/blob/master/example/build.sbt)\n\n* [Github Actions\n   workflow](https://github.com/indoorvivants/sbt-commandmatrix/blob/master/.github/workflows/example.yml)\n\n* [Sample matrix build](https://github.com/indoorvivants/sbt-commandmatrix/actions/runs/822941619)\n\n---\n\nThis plugin generates SBT [commands](https://www.scala-sbt.org/1.x/docs/Commands.html#Commands) based on some minimal information provided by the user.\n\nLet's consider an example:\n\n```scala\nimport commandmatrix._\n\nlazy val scala212 = \"2.12.13\"\nlazy val scala213 = \"2.13.5\"\nlazy val scala3   = \"3.0.0-RC3\"\n\nlazy val core = projectMatrix\n  .in(file(\"core\"))\n  .jvmPlatform(Seq(scala3, scala213, scala212))\n  .jsPlatform(Seq(scala3, scala212))\n```\n\nHere our `core` projectmatrix defines several projects (2 for JVM, 1 for JavaScript). \n\nOur goal is to run the `test` command on subsets of this matrix:\n\n```scala\ninThisBuild(\n  Seq(\n    commands ++= CrossCommand.single(\n      \"test\",\n      matrices = Seq(core),\n      dimensions = Seq(\n        Dimension.scala(\"2.12\"), // \"2.12\" is the default one\n        Dimension.platform()\n      )\n    )\n)\n```\n\nWhich will generate the following commands in the build:\n\n* `test-2_12-jvm`\n* `test-2_13-jvm`\n* `test-3.0.0-RC3-jvm`\n* `test-2_12-js`\n\nAnd run the `test` command in appropriate projects.\n\n### Aggregate commands and stubbing\n\nWe often want to run a list of commands, but some of those commands\nmight not be available for particular platform/scala version combination.\n\nCommon examples I've seen:\n\n1. Running versions of scalafmt/scalafix that don't support Scala 3.\n2. Running `undeclaredCompileDependencies` tasks from sbt-explicit-dependencies\nplugin on Scala.js project produces false positives and breaks the build.\n\n\nIn this case, you can use `CrossCommand.composite` like this:\n\n```scala\n    commands ++= CrossCommand.composite(\n      \"codeQuality\",\n      Seq(\n        \"scalafmtCheckAll\",\n        \"unusedCompileDependenciesTest\",\n        \"undeclaredCompileDependenciesTest\"\n      ),\n      matrices = Seq(core),\n      dimensions = Seq(\n        Dimension.scala(\"2.12\", fullFor3 = true),\n        Dimension.platform()\n      ),\n      filter = axes =\u003e // 1\n        CrossCommand.filter.notScala3(axes) \u0026\u0026\n          CrossCommand.filter.onlyJvm(axes),\n      stubMissing = true // 2\n    )\n  )\n```\n\n* [1]: only consider JVM projects not on Scala 3\n\n* [2]: for all the filtered out project, produce an empty command\n\nBecause of stubbing, running `codeQuality-3.0.0-RC3-jvm` will succeed without\ndoing\nanything (same for `codeQuality-2_12-js`, filtered out because it's non-JVM).\n\nHaving this allows us to define a signifcantly simpler Github Actions workflow:\n\n```yaml\nname: Example\non:\n  push:\n    branches: [\"master\"]\n  pull_request:\n    branches: [\"*\"]\n\njobs:\n  example:\n    name: Example ${{matrix.scalaVersion}} (${{matrix.scalaPlatform}})\n    strategy:\n      fail-fast: false\n      matrix:\n        os: [ubuntu-latest]\n        java: [adopt@1.8]\n        scalaVersion: [\"2_12\", \"2_13\", \"3_0_0-RC3\"]\n        scalaPlatform: [\"jvm\", \"js\"]\n    runs-on: ${{ matrix.os }}\n    env:\n      BUILD_KEY: ${{matrix.scalaVersion}}-${{matrix.scalaPlatform}}\n    steps:\n      - name: Checkout current branch\n        uses: actions/checkout@v2\n\n      - name: Setup Java and Scala\n        uses: olafurpg/setup-scala@v10\n        with:\n          java-version: ${{ matrix.java }}\n\n      - name: Run code quality\n        run: |\n          sbt codeQuality-$BUILD_KEY\n```\n\n## Whole matrix definition\n\n### Motivation\n\nSimilar to the matrix commands, we want to deal with the fact that not all \"cells\" in our matrix have the same configuration. For example:\n\n1. If we don't have platform-specific code, then we don't want to run Scalafmt on platforms other than JVM (same for Scala versions)\n  \n2. Some plugins don't support Scala 3 at all, and as such it's better to just disable them, which is not a settingm but rather a modification of the `Project` in SBT (for example, Scalafix is still experimental with Scala 3)\n\n3. Some combinations of virtual axes just shouldn't exist, for example Scala Native is not yet available for Scala 3, and we want to avoid creating projects for this combination.\n\nLet's consider an example.\n\nSay we have a project that is built for \n\n1. Scala versions `2.12, 2.13, 3.0.2`\n2. Platforms `jvm, js, native`\n3. Some library (which is our main dependency), versions `ver1, ver2`\n   1. An example of this may be Cats Effect, where series 2.x and 3.x are not binary compatible, so libraries that depend on them either have to choose, or cross-build\n\nIn this relatively simple setup (and it is simple, by Scala ecosystem standards :D) we now have the following combinations:\n\n```scala\n  (\"2.12\", \"jvm\", \"ver1\"),\n  (\"2.12\", \"jvm\", \"ver2\"),\n  (\"2.12\", \"js\", \"ver1\"),\n  (\"2.12\", \"js\", \"ver2\"),\n  (\"2.12\", \"native\", \"ver1\"),\n  (\"2.12\", \"native\", \"ver2\"),\n  (\"2.13\", \"jvm\", \"ver1\"),\n  (\"2.13\", \"jvm\", \"ver2\"),\n  (\"2.13\", \"js\", \"ver1\"),\n  (\"2.13\", \"js\", \"ver2\"),\n  (\"2.13\", \"native\", \"ver1\"),\n  (\"2.13\", \"native\", \"ver2\"),\n  (\"3.0.2\", \"jvm\", \"ver1\"),\n  (\"3.0.2\", \"jvm\", \"ver2\"),\n  (\"3.0.2\", \"js\", \"ver1\"),\n  (\"3.0.2\", \"js\", \"ver2\"),\n  (\"3.0.2\", \"native\", \"ver1\"),\n  (\"3.0.2\", \"native\", \"ver2\")\n```\n\nThe goal of this project is to make generating those combinations as simple as possible, and **then** poking holes in the matrix, for example removing the whole `Scala 3 + Scala Native` subset, because it just doesn't exist.\n\n### Proposition and Usage\n\nTo gain access to this functionality, just import everything from the `commandmatrix.extra` package in your `build.sbt`:\n\n```scala\nimport commandmatrix.extra._\n```\n\nAnd that's it. What we propose, is that first we define the entire matrix (the matrix is dense, i.e. most cells are present), and then we refine it, by conditionally removing/keeping/configuring rows in it.\n\nLet's say, that for example above, we want to do the following things:\n\n1. **Completely remove** the `Scala 3 + Scala Native` combination\n2. **Disable** Scalafix plugin for all Scala 3 projects\n3. **Not publish** any of the Scala 2 projects on Scala.js\n\nFirst, let's define the special axis for our imaginary library dependency.\n\n**project/LibraryVersionAxis.scala**\n\n```scala\nimport sbt.VirtualAxis\n\nsealed abstract class LibraryAxis(\n    val idSuffix: String,\n    val directorySuffix: String\n) extends VirtualAxis.WeakAxis\n\nobject LibraryAxis {\n  case object V1 extends LibraryAxis(\"-V1\", \"-v1\")\n  case object V2 extends LibraryAxis(\"-V2\", \"-v2\")\n}\n```\n\nThis will allow us to identify projects generated for distinct versions of this imaginary dependency.\n\nNow, in our **build.sbt** we can first define the whole matrix, and then conditionally define omissions/changes to desired cells:\n\n**build.sbt**\n```scala\nlazy val core = \n  projectMatrix\n    .in(file(\".\"))\n    .someVariations(\n      List(scala212, scala213, scala3),\n      List(VirtualAxis.jvm, VirtualAxis.js, VirtualAxis.native),\n      List(LibraryAxis.V1, LibraryAxis.V2)\n    )(\n      // 1: Completely remove the `Scala 3 + Scala Native` combination\n      MatrixAction((scalaV, axes) =\u003e\n        scalaV.isScala3 \u0026\u0026 axes.contains(VirtualAxis.native)\n      ).Skip,\n      // 2: Disable Scalafix plugin for all Scala 3 projects\n      MatrixAction\n        .ForScala(_.isScala3)\n        .Configure(_.disablePlugins(ScalafixPlugin)),\n      // 3: Not publish any of the Scala 2 projects on Scala.js *\n      MatrixAction((scalaV, axes) =\u003e\n        scalaV.isScala2 \u0026\u0026 axes.contains(VirtualAxis.js)\n      ).Settings(\n        Seq(\n          publish / skip := true,\n          publishLocal / skip := true\n        )\n      )\n    )\n```\n\n**(See the version of this snippet in the [test file](core/src/sbt-test/commandMatrix/extra/build.sbt) which is verified on CI and is guaranteed to be correct)**\n\nIf you want to generate the full matrix without any holes, you can use the `allVariations` method:\n\n```scala\nlazy val core = \n  projectMatrix\n    .in(file(\".\"))\n    .allVariations(\n      List(scala212, scala213, scala3),\n      List(VirtualAxis.jvm, VirtualAxis.js, VirtualAxis.native),\n      List(LibraryAxis.V1, LibraryAxis.V2)\n    )\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findoorvivants%2Fsbt-commandmatrix","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Findoorvivants%2Fsbt-commandmatrix","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Findoorvivants%2Fsbt-commandmatrix/lists"}