Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/jenkins-infra/pipeline-library

Collection of custom steps and variables for our Jenkins instance(s)
https://github.com/jenkins-infra/pipeline-library

jenkins jenkins-pipeline library

Last synced: 4 days ago
JSON representation

Collection of custom steps and variables for our Jenkins instance(s)

Awesome Lists containing this project

README 

        
= Pipeline Global Library for ci.jenkins.io



image:https://ci.jenkins.io/job/Infra/job/pipeline-library/job/master/badge/icon[link="https://ci.jenkins.io/job/Infra/job/pipeline-library/job/master/"]

image:https://img.shields.io/github/v/release/jenkins-infra/pipeline-library[link="https://github.com/jenkins-infra/pipeline-library/releases"]



This repository contains a series of steps and variables for use inside of the

Jenkins project's own link:https://ci.jenkins.io[Jenkins] instance(s).



Check link:https://github.com/jenkins-infra/documentation/blob/master/ci.adoc[this description] of available services.



== Useful steps:



=== buildPlugin



Applies the appropriate defaults for building a Maven-based plugin project on

Linux and Windows.



You are advised to be using a link:https://github.com/jenkinsci/plugin-pom/blob/master/README.md[2.x or newer parent POM].



.Jenkinsfile

[source,groovy]

----

buildPlugin(

  forkCount: '1C', // run this number of tests in parallel for faster feedback.  If the number terminates with a 'C', the value will be multiplied by the number of available CPU cores

  useContainerAgent: true, // Set to `false` if you need to use Docker for containerized tests

  configurations: [

    [platform: 'linux', jdk: 21],

    [platform: 'windows', jdk: 17],

])

----



==== Optional arguments



* `repo` (default: `null`  inherit from Multibranch) - custom Git repository to check out

* `useContainerAgent` (default: `false`) - uses a link:https://github.com/jenkins-infra/documentation/blob/main/ci.adoc#container-agents[Container agent] instead of a Virtual Machine: usually faster to start and generates less costs for the project

** Please note that the implementation of "containers" can be changed over time

* `forkCount` (default: null) - Execute tests with `forkCount` Java virtual machines. If no value is specified, a single JVM is used.  Common values include `0.45C`, `1`, `2`, and `1C`. If you terminate the value with a 'C', that value will be multiplied by the number of available CPU cores. This controls the number of JVM's used for the test code.  Additional JVM's will be launched if the test code starts agents or uses `RealJenkinsRule`.

* `failFast` (default: `true`) - instruct Maven tests to fail fast

* `platforms` (default: `['linux', 'windows']`) - Labels matching platforms to

  execute the steps against in parallel

* `jdkVersions` (default: `[8]`) - JDK version numbers, must match a version

  number jdk tool installed

* `jenkinsVersions`: (default: `[null]`) - a matrix of Jenkins baseline versions to build/test against in parallel (null means default,

  only available for Maven projects)

* `configurations`: An alternative way to specify `platforms`, `jdkVersions` and `jenkinsVersions` (that can not be combined

  with any of them).

* `useArtifactCachingProxy`: (default: `true`) - if set to `false`, artifacts will not use one of the artifact caching proxy depending on the agent provider (Azure, DigitalOcean or AWS), and will directly use repo.jenkins-ci.org instead (discouraged as less reliable and consuming bandwidth)



** Those options will run the build for all combinations of their values. While that is desirable in

  many cases, `configurations` permit to provide a specific combinations of label and java/jenkins versions to use

+

[source,groovy]

----

buildPlugin(/*...*/, configurations: [

  [ platform: "linux", jdk: "17", jenkins: null ],

  [ platform: "windows", jdk: "17", jenkins: null ],

  [ platform: "linux", jdk: "21", jenkins: "2.436" ]

])

----



* `tests`: (default: `null`) - a map of parameters to run tests during the build. The test results and the JaCoCo code

coverage results are recorded after the build with the corresponding Jenkins plugins.

** `skip` - If `true`, skip all the tests by setting the `-skipTests` profile.

  It will also skip FindBugs in modern Plugin POMs.

* `jacoco`: (default: `null`) - a map of parameters to change the default configuration of the `recordCoverage` step of the https://github.com/jenkinsci/code-coverage-api-plugin[Code Coverage Plugin]. This step is called after a plugin build to record the code coverage results of JaCoCo. See https://www.jenkins.io/doc/pipeline/steps/code-coverage-api/#recordcoverage-record-code-coverage-results[recordCoverage step documentation] for a list of available configuration parameters.

* `pit`: (default: `null`) - a map of parameters to change the default configuration of the `recordCoverage` step of the https://github.com/jenkinsci/code-coverage-api-plugin[Code Coverage Plugin]. See https://www.jenkins.io/doc/pipeline/steps/code-coverage-api/#recordcoverage-record-code-coverage-results[recordCoverage step documentation] for a list of available configuration parameters. Since running PIT is a time-consuming task, PIT is disabled by default. You need to enable it by setting the property `skip` to `false` as well.

* `spotbugs`, `checkstyle`, `pmd`, `cpd`: (default: `null`) - a map of parameters to archive SpotBugs, CheckStyle, PMD, or CPD warnings, respectively (only available for Maven projects).

These values can replace or amend the default configuration for the `recordIssues` step of the https://github.com/jenkinsci/warnings-ng-plugin[Warnings NG Plugin].

See https://github.com/jenkinsci/warnings-ng-plugin/blob/master/doc/Documentation.md#configuration[Warnings NG Plugin documentation]

for a list of available configuration parameters.

* `timeout`: (default: `60`) - the number of minutes for build timeout, cannot be bigger than 180, i.e. 3 hours.



NOTE: The `recordIssues` steps of the warnings plugin and the `recordCoverage` steps of the coverage plugin run on the first platform/jdkVersion,jenkinsVersion combination only.

So in the example below it will run for `linux`/`jdk11` but not on `jdk17`.



Usage:



.Jenkinsfile

[source,groovy]

----

buildPlugin(platforms: ['linux'],

        jdkVersions: [11, 17],

        jacoco: [sourceCodeRetention: 'MODIFIED'],

        pit: [skip: false],

        checkstyle: [qualityGates: [[threshold: 1, type: 'NEW', unstable: true]]],

        pmd: [trendChartType: 'TOOLS_ONLY', qualityGates: [[threshold: 1, type: 'NEW', unstable: true]]])

----



=== buildPluginWithGradle()



Builds a Jenkins plugin using Gradle.

The implementation follows the standard build/test/archive pattern.

The method targets compatibility with link:https://github.com/jenkinsci/gradle-jpi-plugin[Gradle JPI Plugin],

and it may not work for other use-cases.



==== Optional arguments



* `repo` (default: `null`  inherit from Multibranch) - custom Git repository to check out

* `failFast` (default: `true`) - instruct the build to fail fast when one of the configurations fail

* `platforms` (default: `['linux', 'windows']`) - Labels matching platforms to

  execute the steps against in parallel

* `jdkVersions` (default: `[8]`) - JDK version numbers, must match a version

  number jdk tool installed

* `configurations`: An alternative way to specify `platforms`, `jdkVersions` (that can not be combined

  with any of them)

** Those options will run the build for all combinations of their values. While that is desirable in

  many cases, `configurations` permit to provide a specific combinations of label and java/jenkins versions to use

+

[source,groovy]

----

buildPluginWithGradle(/*...*/, configurations: [

  [ platform: "linux", jdk: "8" ],

  [ platform: "windows", jdk: "8"],

])

----



* `tests`: (default: `null`) - a map of parameters to run tests during the build

** `skip` - If `true`, skip all the tests.

* `jacoco`: (default: `null`) - a map of parameters to change the default configuration of the `recordCoverage` step of the https://github.com/jenkinsci/code-coverage-api-plugin[Code Coverage Plugin]. This step is called after a plugin build to record the code coverage results of JaCoCo. See https://www.jenkins.io/doc/pipeline/steps/code-coverage-api/#recordcoverage-record-code-coverage-results[recordCoverage step documentation] for a list of available configuration parameters.

* `spotbugs`, `checkstyle`: (default: `null`) - a map of parameters to archive SpotBugs or CheckStyle warnings, respectively.

These values can replace or amend the default configuration for the `recordIssues` step of the https://github.com/jenkinsci/warnings-ng-plugin[Warnings NG Plugin].

See https://github.com/jenkinsci/warnings-ng-plugin/blob/master/doc/Documentation.md#configuration[Warnings NG Plugin documentation]

for a list of available configuration parameters.

* `timeout`: (default: `60`) - the number of minutes for build timeout, cannot be bigger than 180, i.e. 3 hours.

* `noIncrementals`: (default: `false`) - de-activates incremental version publication.



==== Limitations



Not all features of `buildPlugin()` for Maven are supported in the gradle flow.

Examples of not supported features:



* Configuring `jenkinsVersion` for the build flow (as standalone arguments or as `configurations`)

* Usage of link:https://azure.microsoft.com/en-us/services/container-instances/[Azure Container Instances] as agents (only Maven agents are configured)



=== infra.isTrusted()



Determine whether the Pipeline is executing in an internal "trusted" Jenkins

environment



.Jenkinsfile

[source,groovy]

----

if (infra.isTrusted()) {

    /* perform some trusted action like a deployment */

}

----



=== infra.ensureInNode(nodeLabels, body)



Ensures that the given code block is runs in a node with the specified labels



.Jenkinsfile

[source,groovy]

----

infra.ensureInNode('docker,java') {

    sh 'docker -v'

}

----



=== runBenchmarks



Runs JMH benchmarks and archives benchmark reports on `highmem` nodes.



Supported parameters:



`artifacts`::

(Optional) If `artifacts` is not null, invokes `archiveArtifacts` with the given string value.



==== Example



[source, groovy]

----

runBenchmarks('jmh-report.json')

----



=== buildDockerAndPublishImage(imageName, config)



Lints, Builds, then publishes a docker image.



Adds a bunch of build args you can use in your docker image:



* GIT_COMMIT_REV - The commit that triggered this build

* GIT_SCM_URL - Url to repo

* BUILD_DATE - Date that the image was built (now)



Supported parameters:



`imageName`::

Name of the docker image to build



`config`::

(Optional) map of extra flags



* agentLabels: String expression for the labels the agent must match

* automaticSemanticVersioning: Do not automagically increase semantic version by default

* includeImageNameInTag: Set to true for multiple semversioned images built in parallel, will include the image name in tag to avoid conflict

* dockerfile: override the default dockerfile of Dockerfile

* targetplatforms: defined the platforms to build as TARGET

* nextVersionCommand: Commmand line used to retrieve the next version (default 'jx-release-version')

* gitCredentials: override Credential ID for tagging and creating release

* imageDir: Relative path to the context directory for the Docker build

* registryNamespace: empty = autodiscover based on the current controller, but can override the smart default of jenkinsciinfra/ or jenkins4eval/

* unstash: Allow to unstash files if not empty

* dockerBakeFile: Allow to build from a bake file instead

* dockerBakeTarget: Allow to specify a docker bake target other than 'default'

* disablePublication: (Optional, default to false) Allow to disable tagging and publication of container image and GitHub release



==== Example

[source, groovy]

----

buildDockerAndPublishImage('plugins-site-api')

buildDockerAndPublishImage('inbound-agent-maven:jdk8-nanoserver', [

      dockerfile: 'maven/jdk8/Dockerfile.nanoserver',

      agentLabels: 'docker-windows-2019 && amd64',

      targetplatforms: 'windows/amd64',

      imageDir: 'maven/jdk8',

    ])

----



is also called from `parallelDockerUpdatecli` with `config` within `buildDockerConfig` like this :

[source, groovy]

----

parallelDockerUpdatecli([

  imageName: 'wiki',

  rebuildImageOnPeriodicJob: false,

  updatecliConfig: [containerMemory: '1G'],

  buildDockerConfig : [targetplatforms: 'linux/amd64,linux/arm64,linux/s390x']

])

----



== Contribute



=== Requirements



* (Open)JDK v8

* Maven 3.6.x



=== Testing a pull request



By adding `@Library('pipeline-library@pull//head') _` at the top of a Jenkinsfile from a repository built on one of the *.ci.jenkins.io instances, you can test your pipeline library pull request on ci.jenkins.io.



A repository is dedicated for these kind of tests: https://github.com/jenkinsci/jenkins-infra-test-plugin/