{"id":20663615,"url":"https://github.com/networkupstools/jenkins-dynamatrix","last_synced_at":"2025-04-19T15:55:30.293Z","repository":{"id":44707828,"uuid":"371539655","full_name":"networkupstools/jenkins-dynamatrix","owner":"networkupstools","description":"A Jenkins Shared Library to do a sort of matrix build based on available swarm agent labels","archived":false,"fork":false,"pushed_at":"2025-03-16T00:50:46.000Z","size":1061,"stargazers_count":2,"open_issues_count":10,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-29T09:41:56.763Z","etag":null,"topics":["jenkins","jsl","library","multiplatform","nut"],"latest_commit_sha":null,"homepage":"","language":"Groovy","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/networkupstools.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-28T00:45:19.000Z","updated_at":"2025-03-26T12:55:27.000Z","dependencies_parsed_at":"2023-01-23T08:31:09.056Z","dependency_job_id":"7f3f563a-7464-46c8-8842-72a5152cfe31","html_url":"https://github.com/networkupstools/jenkins-dynamatrix","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networkupstools%2Fjenkins-dynamatrix","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networkupstools%2Fjenkins-dynamatrix/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networkupstools%2Fjenkins-dynamatrix/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/networkupstools%2Fjenkins-dynamatrix/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/networkupstools","download_url":"https://codeload.github.com/networkupstools/jenkins-dynamatrix/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249731219,"owners_count":21317341,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jenkins","jsl","library","multiplatform","nut"],"created_at":"2024-11-16T19:18:50.614Z","updated_at":"2025-04-19T15:55:30.273Z","avatar_url":"https://github.com/networkupstools.png","language":"Groovy","funding_links":[],"categories":[],"sub_categories":[],"readme":"Dynamatrix\n==========\n\nGeneral overview\n~~~~~~~~~~~~~~~~\n\nA Jenkins Pipeline Shared Library allows to re-use code blocks in\ndifferent pipelines, to use Java class approach to data and logic\nimplementation, as well as to store global static state data for\nthe duration of a pipeline script execution.\n\nIts source repo has a predefined structure, and has to be registered\non the Jenkins controller to be trusted, and the pipeline scripts\ncan then include it like this:\n\n----\n@Library ('org.nut.dynamatrix') _\n----\n\nWith https://github.com/jenkinsci/pipeline-groovy-lib-plugin/pull/19\nfeature used in the build of the `@Library` implementation running on\nthe Jenkins server, it is also possible to request \"same branch of the\nlibrary as the branch of pipeline script\" with fallback to configured\ndefault branch, to avoid superficial differences between pipeline code\nbranches, like this:\n\n----\n@Library ('org.nut.dynamatrix@${BRANCH_NAME}') _\n----\n\nKeep in mind that this is not a Groovy variable, just intentionally\nlooks like one for simpler code maintenance. The `@Library` annotation\nis processed before compilation of pipeline script to Groovy and Java.\n\n[NOTE]\n.Obsoleted Note (valid for unmodified Jenkins servers):\n======\nThe `@Library` method only allows non-dynamic references to the library\n(a different approach is possible for run-time resolution of branches\nwith `library` step whose limitations make it not usable for us).\n\nFor this reason, dev/test version of the code of the library from\ndev/test versions of its consumers had to be named explicitly, like this:\n\n----\n@Library ('org.nut.dynamatrix@fightwarn') _\n----\n\n======\n\nWhat and why\n~~~~~~~~~~~~\n\nThis project is intended as a relatively generic solution for Jenkins\npipelines to build and test the codebase in a variety of environments\n(such as operating systems, compiler brands/versions or programming\nlanguage revisions) and setups (such as the sets of warnings marked\nto be fatal vs. ones that are still tolerated) based dynamically on\nwhat capabilities are supported by the workers of the build farm.\n\nThese workers may be not necessarily managed or provisioned by the\nsame administrator or entity as manages the build farm, but can be\ne.g. a link:https://plugins.jenkins.io/swarm/[Swarm] of specially\nlabeled agents provided by contributors of a FOSS project -- which\nis especially of interest to such users in order to have their less\ncommon system or setup supported by the FOSS project they like and\nuse, or to contribute access to architectures that the project does\nnot own.\n\nFor projects that have a minimal baseline of scenarios that must pass,\nthis information can be conveyed by the configuration for the pipeline,\nso that dynamic workers matching those requirements can be spun up and\nlater deactivated or destroyed to conserve run-time resources of the\ncentralized build farm. This allows to sequentialize builds of varied\noperating environments hosted on systems with limited memory.\n\nThis library, or generally persistent CI farms, can also take advantage\nof https://github.com/jimklimov/git-refrepo-scripts and a (currently\ncustom) build of Git Client Plugin with the changes proposed in PR\nhttps://github.com/jenkinsci/git-client-plugin/pull/644\nto maintain reference repositories on the CI server, and so reduce\nthe traffic and time needed for future Git check-outs.\n\n\nMain idea\n~~~~~~~~~\n\nDynamatrix provides a Shared Pipeline library, so that the pipeline\ndefinitions in actual source code repositories can be kept compact\nand not need to update their Jenkinsfiles whenever this idea evolves.\n\nBuild agent definitions, whether static (e.g. SSH Agents) or dynamic\n(e.g. Swarm Client agents) can declare labels which can be matched in\nthe pipeline stages. The Dynamatrix is layered on top of structurally\nnamed labels to generate a list of possible build combinations as a\nboolean label expression, and so dynamically generate the payload\nstructure of each build.\n\nA sibling link:https://github.com/networkupstools/jenkins-swarm-nutci[jenkins-swarm-nutci]\nproject is used on build agents which can dial in to the NUT CI farm\nand provide their services and capabilities (conveyed by the labels\nwhich are part of their configuration), and includes some scripting\nfor common configuration approach as well as for automated start-up\non numerous platforms. It might be usable \"as is\" or easily adapted\nby other projects.\n\nSince the same CI farm and Swarm agent workers can be used for the\nbenefit of several FOSS projects, one of the label patterns allows\nto declare that this worker is suitable for (and contributes to)\nbuilds of a particular project.\n\nThis library is designed to provide a number of classes for larger\npieces of code as separate compiliation units (my first stint with\nstandard pipeline `matrix{}` failed due to resulting method binary\nexceeding 64K) and as separate callable routines. While it may be\nhelpful to directly use `prepareDynamatrix()` in some cases, many\npractical jobs would define a `dynacfgBase` with common dynamatrix\nproperties for a suitable node and a `dynacfgPipeline` with certain\nfields such as the closure(s) to run dozens of times, and pass that\nto a `dynamatrixPipeline(dynacfgBase, dynacfgPipeline)` call that\nimplements the pipeline in a reusable fashion.\n\nExample configuration\n~~~~~~~~~~~~~~~~~~~~~\n\nIn the illustrative example below, two workers are provisioned and\nbased on their installed tools, they declare the following agent\nlabels relevant to the dynamatrix:\n\n* \"testoi\":\n----\nCOMPILER=CLANG COMPILER=GCC\nCLANGVER=8 CLANGVER=9\nGCCVER=10 GCCVER=4.4.4 GCCVER=4.9 GCCVER=6 GCCVER=7\nOS=openindiana\nARCH=x86_64 ARCH=x86\nnut-builder zmq-builder\n----\n* \"testdeb\":\n----\nCOMPILER=GCC\nGCCVER=4.8 GCCVER=4.9 GCCVER=5 GCCVER=7\nOS=linux\nARCH=x86_64 ARCH=x86 ARCH=armv7l\nnut-builder zmq-builder linux-kernel-builder\n----\n\nA Jenkinsfile in a project using this library would look like:\n----\n@Library ('dynamatrix') _\n\n// Prepare parallel stages for the dynamatrix:\ndef parallelStages = prepareDynamatrix(\n    commonLabelExpr: 'nut-builder',\n    compilerType: 'C',\n    compilerLabel: 'COMPILER',\n    compilerTools: ['CC', 'CXX', 'CPP'],\n    dynamatrixAxesLabels: ['OS', '${COMPILER}VER', 'ARCH'],\n    dynamatrixAxesCommonEnv: [['LANG=C', 'TZ=UTC'], ['LANG=ru_RU']],\n    dynamatrixAxesCommonOpts: [\n        ['\"CFLAGS=-stdc=gnu99\" CXXFLAGS=\"-stdcxx=g++99\"', '\"CFLAGS=-stdc=c89\" CXXFLAGS=\"-stdcxx=c++89\"'],\n        ['-m32', '-m64'] ],\n    dynamatrixRequiredLabelCombos: [[~/OS=bsd/, ~/CLANG=12/]],\n    allowedFailure: [[~/OS=.*/, ~/GCCVER=4\\..*/], ['ARCH=armv7l'], [~/std.*=.*89/]],\n    excludeCombos: [[~/OS=openindiana/, ~/CLANGVER=9/, ~/ARCH=x86/]]\n) {\n    unstash 'preparedSource'\n    sh \"\"\" ./autogen.sh \"\"\"\n    sh \"\"\" ${dynamatrix.commonEnv} CC=${dynamatrix.COMPILER.CC.VER} CXX=${dynamatrix.COMPILER.CXX.VER} ./configure ${dynamatrix.commonOpts} \"\"\"\n    sh \"\"\" make -j 4 \"\"\"\n    sh \"\"\" make check \"\"\"\n}\n\npipeline {\n    // List some unique options, build parameters, and build scenarios\n    // like spell checking or docs generation that you do not need to\n    // run dozens of times using every agent.\n    agent { label 'persistent-worker' }\n    stages {\n        stage('Prepare source') {\n            steps {\n                // The persistent agent may be in better position to\n                // e.g. use a Git reference repository for faster\n                // checkouts, or just to have the internet access\n                // which CI farm workers may lack.\n                checkout scm\n                stash 'preparedSource'\n            }\n        }\n        stage('Spell Check') {\n            steps {\n                sh \"\"\" aspell ... \"\"\"\n            }\n        }\n        stage('Make docs') {\n            agent { label 'docs-builder' }\n            steps {\n                unstash 'preparedSource'\n                sh \"\"\" make pdf \"\"\"\n            }\n        }\n    }\n}\n\nparallel parallelStages\n----\n\nWith this configuration, the Dynamatrix should detect the running\nagents and know their capabilities, so it is in position to prepare\na series of builds covering every available OS and compiler version\nand CPU architecture.\n\nIt can optionally be filtered through constraints, such as that we\ndo not even want to try building a combination described by (matching)\nthe `skip` option, that we require to run some combination(s) even if\nan agent for that is not currently running so labels are not detected\n(things can hang in queue waiting for a worker, or can cause spinning\nup a build agent if it is configured but dormant), or that some certain\nbuild setups may fail (e.g. we wonder how they fare, but they are not\na required baseline and so not blockers for a merge) so their results\nwould not be impacting the overall job verdict.\n\nFor certain compiler toolkits (e.g. 'C' family) it would provide an\nautomatic preparation of variables for several same-versioned tools\n(e.g. C and C++ compilers).\n\nDeclaring additional configuration of build agents\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nDepending on their implementation and connectivity, build agents may\nhave different preferences, and this library allows to tune them with\ntheir node labels.\n\nOne such area is delivery of tested codebase to the agents: default\napproach which is to stash on master, and unstash on agents, should\nbe reliable (should reach agents that can not use the SCM platform\ndirectly, and should ensure all agents test the same revision even\nif it disappears from the SCM platform -- by e.g. force-push to a PR),\nbut at a cost of repetitive traffic and I/O to unstash same code time\nand again (often on same machine) during a matrix build.\n\nSetting `DYNAMATRIX_UNSTASH_PREFERENCE` to `scm-ws`, `scm` or `unstash`\nin the individual agent labels allows that system to start with either\nan SCM checkout augmented by a Git reference repository (persistent) in\nthe workspace and maintained during each run (this currently requires\na custom build of Jenkins Git Client plugin including the feature from\nhttps://github.com/jenkinsci/git-client-plugin/pull/644 unless/until\nit gets properly merged); or using a plain SCM checkout; or unstashing.\nThese methods fall-back from one to next in the order listed above.\n\nThe DynamatrixStash methods dealing with code checkout, stashing and\nunstashing, allow a concept of `stashName` used to identify archives\nas well as to track metadata for that codebase (so the same pipeline\ncan mix several repositories). Preferences for each repository can\nbe tailored, using e.g. `scm:githubProject` and `unstash:privateRepo`\nlabel values to use different delivery methods for the two stashNames.\n\nTo prevent several parallel jobs and build scenarios from corrupting\nthe reference repository maintained in the workspace, maintenance of\nthis location is protected by Lockable Resources plugin. Since agents\nrunning with independent storage should not wait on each other, this\nlock can be tuned by setting `DYNAMATRIX_REFREPO_WORKSPACE_LOCKNAME`\nlabel; note that agents that do indeed use same storage (shared over\nNFS, or using containers with same homedirs from their host) should\nset identical values in their common lock name.\n\nDirectory naming\n~~~~~~~~~~~~~~~~\n\nThis is a Jenkins Shared Library. As such, it has some required file\nsystem structure including:\n\n* `vars/` -- the \"groovy variables\" which are, at least in this context,\n  sources for single-use class instances and their methods that can\n  be called from each other or from the pipeline which uses the lib,\n  and as far as the pipeline is concerned, `call()` methods in these\n  groovy files are custom \"steps\" (named same as the file);\n\n* `src/` -- formal classes including ones that can be static, such as\n  to store some persistent configuration for the run.\n\nThere are further standard structure points that we do not currently\nuse, such as location and naming of documentation to accompany the\ndeclared steps so this can be displayed by Jenkins UI, and location\nfor resources such as shell scripts and arbitrary data used by JSL.\n\nApproach recommended in some of the articles linked below is that the\nlogic is mostly (ideally all) in `src/` classes, while the `vars/`\nsteps only wrap calls to that.\n\nPractice, especially during early development iterations, may be mixed.\n\nInspirational reading\n~~~~~~~~~~~~~~~~~~~~~\n\n* https://www.jenkins.io/blog/2020/10/21/a-sustainable-pattern-with-shared-library/ - it\n  provides a useful pattern allowing a default configuration for a\n  generic librarly build recipe implementation to be merged with\n  options desired for a particular pipeline's build, including an\n  OOP-style selection of build method based on files present in\n  the specific repo. This way whatever looks similar on some level\n  of abstraction is handled the same way, and whatever really differs\n  has the hooks and hacks for that individuality.\n\n* https://github.com/jenkins-infra/pipeline-library/blob/master/vars/buildPlugin.groovy - this\n  code orchestrating standard builds of Jenkins plugins manages a\n  similar matrix, optionally based on build parameters\n\n* https://bmuschko.com/blog/jenkins-shared-libraries/ - goes\n  into the much welcome and somewhat gritty detail about using\n  classes instead of \"vars\" used quickly as steps, which is what most\n  of the other articles focus on\n\n* https://www.linkedin.com/pulse/jenkins-shared-pipeline-libraries-custom-runtime-delgado-garrido - a\n  pattern for configs in component sources that can tune behavior of\n  otherwise standardized library pipelines and/or maintain a Singleton\n  with config (and other) data during the run\n\n* https://www.linkedin.com/pulse/jenkins-global-shared-pipeline-libraries-real-unit-delgado-garrido - and\n  another pattern for keeping real logic hidden in classes, frontended\n  by steps in \"vars\" folder\n\nGood explanatory articles with varied detail; many other texts seem\nto tell the same things differently while reasonably assuming a\nnon-beginner level from the reader. Some of those below may be a bit\ntoo long and chewing the basics delicately -- but sometimes that is\nreally a good thing:\n\n* https://www.lambdatest.com/blog/use-jenkins-shared-libraries-in-a-jenkins-pipeline/\n* https://tomd.xyz/jenkins-shared-library/\n* https://medium.com/@werne2j/jenkins-shared-libraries-part-1-5ba3d072536a\n* https://medium.com/@werne2j/how-to-build-your-own-jenkins-shared-library-9dc129db260c\n* https://medium.com/@werne2j/unit-testing-a-jenkins-shared-library-9bfb6b599748 - article\n  about testing with maven and Jenkins-Spock library\n* https://medium.com/@werne2j/collecting-code-coverage-for-a-jenkins-shared-library-c2d8f502732e\n* https://medium.com/disney-streaming/testing-jenkins-shared-libraries-4d4939406fa2 - article\n  about testing with gradle and Jenkins Pipeline Unit library\n* https://dev.to/kuperadrian/how-to-setup-a-unit-testable-jenkins-shared-pipeline-library-2e62 - article\n  about testing with gradle, mockito and IntelliJ IDEA integration and injectable contexts\n\nStandard reading library:\n\n* https://www.jenkins.io/doc/book/pipeline/shared-libraries/\n* https://www.jenkins.io/doc/pipeline/steps/workflow-cps-global-lib/\n* https://www.jenkins.io/blog/2019/12/02/matrix-building-with-scripted-pipeline/\n* https://www.jenkins.io/blog/2019/11/22/welcome-to-the-matrix/\n\nDevelopment info\n~~~~~~~~~~~~~~~~\n\nIntelliJ IDEA setup as the IDE for Jenkins-related contents, and creation\nof a Gradle project for easier maintenance and testing of Jenkins Shared\nPipeline Libraries followed these articles:\n\n* http://tdongsi.github.io/blog/2018/02/09/intellij-setup-for-jenkins-shared-library-development/\n** https://github.com/tdongsi/jenkins-config/blob/develop/docs/IDE.md\n* https://github.com/mkobit/jenkins-pipeline-shared-libraries-gradle-plugin\n* https://github.com/mkobit/jenkins-pipeline-shared-library-example\n* https://stackoverflow.com/questions/53363828/jenkins-shared-library-with-intellij\n\nRandom example self-tests:\n\n* https://github.com/jenkinsci/workflow-cps-plugin/blob/master/src/test/java/org/jenkinsci/plugins/workflow/cps/DSLTest.java\n\nUnit-tests of the JSL library\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nThis library is tested with the help of \"mkobit\" plugins referenced above.\nTests are located in the `tests/` directory and implement example pipelines\nwhich are executed in either a temporary Jenkins environment provided by a\n`JenkinsRule` environment (from `jenkins-test-harness` project) or by a more\ndedicated server behind a `RealJenkinsRule`. They can be executed by an IDE\n(e.g. press `F9` in IntelliJ IDEA) or by `./gradlew integrationTest`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetworkupstools%2Fjenkins-dynamatrix","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetworkupstools%2Fjenkins-dynamatrix","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetworkupstools%2Fjenkins-dynamatrix/lists"}