{"id":13463769,"url":"https://github.com/jenkinsci/kubernetes-plugin","last_synced_at":"2025-05-13T20:20:17.659Z","repository":{"id":27384661,"uuid":"30860673","full_name":"jenkinsci/kubernetes-plugin","owner":"jenkinsci","description":"Jenkins plugin to run dynamic agents in a Kubernetes/Docker environment","archived":false,"fork":false,"pushed_at":"2025-04-25T19:15:54.000Z","size":8167,"stargazers_count":2282,"open_issues_count":26,"forks_count":1258,"subscribers_count":62,"default_branch":"master","last_synced_at":"2025-04-28T11:54:41.807Z","etag":null,"topics":["agent","cloud","kubernetes"],"latest_commit_sha":null,"homepage":"https://plugins.jenkins.io/kubernetes/","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jenkinsci.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG-archive.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2015-02-16T09:02:22.000Z","updated_at":"2025-04-23T06:10:44.000Z","dependencies_parsed_at":"2023-12-21T15:34:51.871Z","dependency_job_id":"8f7572c8-ebc4-4c4c-a2df-956707f5d96f","html_url":"https://github.com/jenkinsci/kubernetes-plugin","commit_stats":{"total_commits":3042,"total_committers":240,"mean_commits":12.675,"dds":0.7084155161078238,"last_synced_commit":"c91e951eaebd823be6ad34478eba9fd50e6a631d"},"previous_names":[],"tags_count":293,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jenkinsci%2Fkubernetes-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jenkinsci%2Fkubernetes-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jenkinsci%2Fkubernetes-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jenkinsci%2Fkubernetes-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jenkinsci","download_url":"https://codeload.github.com/jenkinsci/kubernetes-plugin/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251311332,"owners_count":21569008,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","cloud","kubernetes"],"created_at":"2024-07-31T14:00:28.206Z","updated_at":"2025-04-28T11:54:57.887Z","avatar_url":"https://github.com/jenkinsci.png","language":"Java","funding_links":[],"categories":["Tools","Continuous Delivery","Java","cloud","Agent Integration \u0026 Deployment Tools"],"sub_categories":["[Jenkins](#jenkins)","MCP Server Framework"],"readme":"Kubernetes plugin for Jenkins\n=========================\n\n[![kubernetes version](https://img.shields.io/jenkins/plugin/v/kubernetes.svg?label=kubernetes)](https://plugins.jenkins.io/kubernetes)\n[![kubernetes installs](https://img.shields.io/jenkins/plugin/i/kubernetes.svg)](https://plugins.jenkins.io/kubernetes)\n[![kubernetes license](https://img.shields.io/github/license/jenkinsci/kubernetes-plugin)](https://github.com/jenkinsci/kubernetes-plugin/blob/master/LICENSE)\n[![Language grade: Java](https://img.shields.io/lgtm/grade/java/g/jenkinsci/kubernetes-plugin.svg?logo=lgtm\u0026logoWidth=18)](https://lgtm.com/projects/g/jenkinsci/kubernetes-plugin/context:java)\n\nJenkins plugin to run dynamic agents in a Kubernetes cluster.\n\nBased on the [Scaling Docker with Kubernetes](http://www.infoq.com/articles/scaling-docker-with-kubernetes) article,\nautomates the scaling of Jenkins agents running in Kubernetes.\n\nThe plugin creates a Kubernetes Pod for each agent started, and stops it after each build.\n\nAgents are launched as inbound agents, so it is expected that the container connects automatically to the Jenkins controller.\nFor that some environment variables are automatically injected:\n\n* `JENKINS_URL` : Jenkins web interface url\n* `JENKINS_SECRET` : the secret key for authentication\n* `JENKINS_AGENT_NAME` : the name of the Jenkins agent\n* `JENKINS_NAME` : the name of the Jenkins agent (Deprecated. Only here for backwards compatibility)\n\nTested with [`jenkins/inbound-agent`](https://hub.docker.com/r/jenkins/inbound-agent),\nsee the [Docker image source code](https://github.com/jenkinsci/docker-agent).\n\nIt is not required to run the Jenkins controller inside Kubernetes.\n\n# 📜 Table of Contents\n\n- [Generic setup](#generic-setup)\n- [Usage](#usage)\n- [Configuration reference](#configuration-reference)\n- [Inheritance](#inheritance)\n- [Declarative Pipeline](#declarative-pipeline)\n- [Misc.](#misc.)\n- [Running on OpenShift](#running-on-openshift)\n- [Features controlled using system properties](#features-controlled-using-system-properties)\n- [Windows support](#windows-support)\n- [Constraints](#constraints)\n- [Configuration on minikube](#configuration-on-minikube)\n- [Configuration on Google Container Engine](#configuration-on-google-container-engine)\n- [Troubleshooting 🔨](#troubleshooting)\n- [Building and Testing](#building-and-testing)\n- [Docker image](#docker-image)\n- [Running in Kubernetes](#running-in-kubernetes)\n- [Related Projects](#related-projects)\n\n# Generic Setup\n## Prerequisites\n* A running Kubernetes cluster 1.14 or later. For OpenShift users, this means OpenShift Container Platform 4.x.\n* A Jenkins instance installed\n* The Jenkins Kubernetes plugin installed\n* A ServiceAccount with sufficient privileges ([example](src/main/kubernetes/service-account.yml))\n\n## Configuration\n\nFill in the Kubernetes plugin configuration.\nIn order to do that, you will open the Jenkins UI and navigate to **Manage Jenkins -\u003e Manage Nodes and Clouds -\u003e Configure Clouds -\u003e Add a new cloud -\u003e Kubernetes** and enter the *Kubernetes URL* and *Jenkins URL* appropriately, unless Jenkins is running in Kubernetes in which case the defaults work.\n\nSupported credentials include:\n\n* Username/password\n* Secret File (kubeconfig file)\n* Secret text (Token-based authentication) (OpenShift)\n* Google Service Account from private key (GKE authentication)\n* X.509 Client Certificate\n\nIf you check **WebSocket** then agents will connect over HTTP(S) rather than the Jenkins service TCP port.\nThis is unnecessary when the Jenkins controller runs in the same Kubernetes cluster,\nbut can greatly simplify setup when agents are in an external cluster\nand the Jenkins controller is not directly accessible (for example, it is behind a reverse proxy or a ingress resource).\nSee [JEP-222](https://jenkins.io/jep/222) for more.\n\nTo test this connection is successful you can use the **Test Connection** button to ensure there is\nadequate communication from Jenkins to the Kubernetes cluster, as seen below\n\n![image](images/cloud-configuration.png)\n\n\n### Garbage collection (beta)\n\nIn some exceptional cases, agent pods can be left behind, with no declared Jenkins agent in the controller. They will try to reconnect over and over, until something deletes them.\n\nThe plugin provides a garbage collection mechanism to clean up these pods. As it has been introduced recently,\nand generates extra load on the Kubernetes API server, it is disabled by default.\n\nFeel free to enable it and provide feedback about this functionality.\n\n![image](images/garbage-collection.png)\n\n## Static pod templates\n\nIn addition to that, in the **Kubernetes Pod Template** section, we need to configure the image that will be used to \nspin up the agent pod. The Jenkins agent runs by default in a container named `jnlp` (historical name, kept for backward compatibility).\n\nYou may use it, use a different agent container name and/or provide your own container image.\n\nTo avoid drift between the Jenkins controller and your agent, we recommend *not* to include the agent JAR in your image, but instead use the *Inject Jenkins agent in agent container* checkbox.\n\nThe provided container image must have a JRE installed that is compatible with the Java version required by the Jenkins controller.\n\nIn the ‘Kubernetes Pod Template’ section you need to specify the following (the rest of the configuration is up to you):\nKubernetes Pod Template Name - can be any and will be shown as a prefix for unique generated agent’ names, which will \nbe run automatically during builds\nDocker image - the docker image name that will be used as a reference to spin up a new Jenkins agent, as seen below\n\n![image](images/pod-template-configuration.png)\n\n\u003e **Note:** If your Jenkins controller is outside the cluster and uses a self-signed HTTPS certificate,\n\u003e   you will need some [additional configuration](#using-websockets-with-a-jenkins-master-with-self-signed-https-certificate).\n\n### Restricting what jobs can use your configured cloud\n\nClouds can be configured to only allow certain jobs to use them.\n\nTo enable this, in your cloud's advanced configuration check the\n`Restrict pipeline support to authorized folders` box. For a job to then\nuse this cloud configuration you will need to add it in the jobs folder's configuration.\n\n# Usage\n## Overview\n\nThe Kubernetes plugin allocates Jenkins agents in Kubernetes pods.\n\nWithin these pods, there is always one special container that is running the Jenkins agent.\n\nIts name defaults to `jnlp`, but you can override the name with one of your choosing in pod template configuration.\n\nOther containers can run arbitrary processes of your choosing, and it is possible to run commands dynamically in any container in the agent pod using the `container` step (pipeline only). \n\n## Using a label\n\nPod templates defined using the user interface declare a label. When a freestyle job or a pipeline job using\n`node('some-label')` uses a label declared by a pod template, the Kubernetes Cloud allocates a new pod to run the\nJenkins agent.\n\nIt should be noted that the main reason to use the global pod template definition is to migrate a huge corpus of\nexisting projects (including freestyle) to run on Kubernetes without changing job definitions.\nNew users setting up new Kubernetes builds should use the `podTemplate` step as shown in the example snippets\n[here](examples).\n\n## Using the pipeline step\n\nThe `podTemplate` step defines an ephemeral pod template. It is created while the pipeline execution is within the\n`podTemplate` block. It is immediately deleted afterwards. Such pod templates are not intended to be shared with other\nbuilds or projects in the Jenkins instance.\n\nThe following idiom creates a pod template with a generated unique label (available as `POD_LABEL`) and runs commands inside it.\n\n```groovy\npodTemplate {\n    node(POD_LABEL) {\n        // pipeline steps...\n    }\n}\n```\n\nCommands will be executed by default in the designated agent container, where the Jenkins agent is running.\n\nThis will run in the agent container:\n```groovy\npodTemplate {\n    node(POD_LABEL) {\n        stage('Run shell') {\n            sh 'echo hello world'\n        }\n    }\n}\n```\n\nFind more examples in the [examples dir](examples).\n\nThe default agent image used can be customized by adding it to the template\n\n```groovy\ncontainerTemplate(name: 'jnlp', image: 'jenkins/inbound-agent', args: '${computer.jnlpmac} ${computer.name}'),\n```\n\nor with the yaml syntax. Pretty much any field from the [pod model](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/) can be specified through the yaml syntax.\n\n```yaml\napiVersion: v1\nkind: Pod\nspec:\n  containers:\n  - name: jnlp\n    image: 'jenkins/inbound-agent'\n    args: ['\\$(JENKINS_SECRET)', '\\$(JENKINS_NAME)']\n```\n\n### Multiple containers support\n\nMultiple containers can be defined for the agent pod, with shared resources, like mounts. Ports in each container can\nbe accessed as in any Kubernetes pod, by using `localhost`.\n\nOne container must run the Jenkins agent. If unspecified, a container named `jnlp` will be created with the inbound-agent image.\nThe Jenkins agent requires a JRE to run, so you can avoid the extra container by providing a name using the `agentContainer`.\nTo get the Jenkins agent injected, you will also need to set `agentInjection` to `true`, and leave the command and argument fields empty for this container.\nThe container specified by `agentContainer` will be the one where shell steps (or any other step running remote commands on the agent) will run on.\n\nTo execute commands in another container part of the pod (different from the one running the Jenkins agent), you can use the `container` step.\n\n**Note**\n---\nDue to implementation constraints, there can be issues when executing commands in different containers if they run using different uids.\nIt is recommended to use the same uid across the different containers part of the same pod to avoid any issue.\n---\n\n```groovy\npodTemplate(\n  agentContainer: 'maven',\n  agentInjection: true,\n  containers: [\n    containerTemplate(name: 'maven', image: 'maven:3.9.9-eclipse-temurin-17'),\n    containerTemplate(name: 'golang', image: 'golang:1.16.5', command: 'sleep', args: '99d')\n  ]) {\n\n    node(POD_LABEL) {\n        stage('Get a Maven project') {\n            git 'https://github.com/jenkinsci/kubernetes-plugin.git'\n            container('maven') {\n                stage('Build a Maven project') {\n                    sh 'mvn -B -ntp clean install'\n                }\n            }\n        }\n\n        stage('Get a Golang project') {\n            git url: 'https://github.com/hashicorp/terraform.git', branch: 'main'\n            container('golang') {\n                stage('Build a Go project') {\n                    sh '''\n                    mkdir -p /go/src/github.com/hashicorp\n                    ln -s `pwd` /go/src/github.com/hashicorp/terraform\n                    cd /go/src/github.com/hashicorp/terraform \u0026\u0026 make\n                    '''\n                }\n            }\n        }\n\n    }\n}\n```\n\nor\n\n```groovy\npodTemplate(\n  agentContainer: 'maven',\n  agentInjection: true,\n  yaml: '''\n    apiVersion: v1\n    kind: Pod\n    spec:\n      containers:\n      - name: maven\n        image: maven:3.9.9-eclipse-temurin-17\n      - name: golang\n        image: golang:1.16.5\n        command:\n        - sleep\n        args:\n        - 99d\n''') {\n  node(POD_LABEL) {\n    stage('Get a Maven project') {\n      git 'https://github.com/jenkinsci/kubernetes-plugin.git'\n      container('maven') {\n        stage('Build a Maven project') {\n          sh 'mvn -B -ntp clean install'\n        }\n      }\n    }\n\n    stage('Get a Golang project') {\n      git url: 'https://github.com/hashicorp/terraform-provider-google.git', branch: 'main'\n      container('golang') {\n        stage('Build a Go project') {\n          sh '''\n            mkdir -p /go/src/github.com/hashicorp\n            ln -s `pwd` /go/src/github.com/hashicorp/terraform\n            cd /go/src/github.com/hashicorp/terraform \u0026\u0026 make\n          '''\n        }\n      }\n    }\n  }\n}\n```\n\n#### `POD_CONTAINER` variable\n\nThe variable `POD_CONTAINER` contains the name of the container in the current context.\nIt is defined only within a `container` block.\n\n```groovy\npodTemplate(containers: […]) {\n  node(POD_LABEL) {\n    stage('Run shell') {\n      container('mycontainer') {\n        sh \"echo hello from $POD_CONTAINER\" // displays 'hello from mycontainer'\n      }\n    }\n  }\n}\n```\n\n### Retrying after infrastructure outages\n\nYou can use the `retry` step to automatically try the whole build stage again with a fresh pod in case of fatal infrastructure problems.\n(For example: cluster backup \u0026 restore; node pool used for agents drained and upgraded.)\n\n```groovy\npodTemplate(…) {\n  retry(count: 2, conditions: [kubernetesAgent(), nonresumable()]) {\n    node(POD_LABEL) {\n      sh 'your-build-process'\n    }\n  }\n}\n```\n\nwill rerun the whole `node` block (using the same pod definition) in case the first attempt fails for a qualifying reason traceable to loss of the pod\n(_not_ routine problems such as compilation errors or `OutOfMemoryError`).\n\nFor Declarative Pipeline, just add the `retries` option, as shown below.\n\n# Configuration reference\n## Pod template\n\nPod templates are used to create agents. They can be either configured via the user interface, or in a pipeline, using\nthe `podTemplate` step.\nEither way it provides access to the following fields:\n\n* **cloud** The name of the cloud as defined in Jenkins settings. Defaults to `kubernetes`\n* **name** The name of the pod. This is only used for inheritance.\n* **namespace** The namespace of the pod.\n* **label** The node label. This is how the pod template can be referred to when asking for an agent through the `node` step. In a pipeline, it is recommended to omit this field and rely on the generated label that can be referred to using the `POD_LABEL` variable defined within the `podTemplate` block.\n* **yaml** [yaml representation of the Pod](https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/), to allow setting any values not supported as fields\n* **yamlMergeStrategy** `merge()` or `override()`. Controls whether the yaml definition overrides or is merged with the yaml definition inherited from pod templates declared with `inheritFrom`. Defaults to `override()` (for backward compatibility reasons).\n* **containers** The container templates part of the pod *(see below for details)*.\n* **serviceAccount** The service account of the pod.\n* **nodeSelector** The node selector of the pod.\n* **nodeUsageMode** Either `NORMAL` or `EXCLUSIVE`, this controls whether Jenkins only schedules jobs with label expressions matching or use the node as much as possible.\n* **volumes** Volumes that are defined for the pod and are mounted by **ALL** containers.\n  * `configMapVolume` : a read only volume that is mounted from a ConfigMap.\n  * `dynamicPVC()` : a persistent volume claim managed dynamically. It is deleted at the same time as the pod.\n  * `emptyDirVolume` (default): an empty dir allocated on the host machine\n  * `hostPathVolume()` : a host path volume\n  * `nfsVolume()` : a nfs volume\n  * `persistentVolumeClaim()` : an existing persistent volume claim by name.\n  * `secretVolume` : a read only volume that is mounted from a Kubernetes secret.\n* **envVars** Environment variables that are applied to **ALL** containers.\n    * **envVar** An environment variable whose value is defined inline.\n    * **secretEnvVar** An environment variable whose value is derived from a Kubernetes secret.\n* **imagePullSecrets** List of pull secret names, to [pull images from a private Docker registry](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/).\n* **annotations** Annotations to apply to the pod.\n* **inheritFrom** List of one or more pod templates to inherit from *(more details below)*.\n* **slaveConnectTimeout** Timeout in seconds for an agent to be online *(more details below)*.\n* **podRetention** Controls the behavior of keeping agent pods. Can be 'never()', 'onFailure()', 'always()', or 'default()' - if empty will default to deleting the pod after `activeDeadlineSeconds` has passed.\n* **activeDeadlineSeconds** If `podRetention` is set to `never()` or `onFailure()`, the pod is deleted after this deadline is passed.\n* **idleMinutes** Allows the pod to remain active for reuse until the configured number of minutes has passed since the last step was executed on it.\n* **showRawYaml** Enable or disable the output of the raw pod manifest. Defaults to `true`\n* **runAsUser** The user ID to run all containers in the pod as.\n* **runAsGroup** The group ID to run all containers in the pod as. \n* **hostNetwork** Use the hosts network.\n* **workspaceVolume** The type of volume to use for the workspace.\n  * `dynamicPVC()` : a persistent volume claim managed dynamically. It is deleted at the same time as the pod.\n  * `emptyDirWorkspaceVolume` (default): an empty dir allocated on the host machine\n  * `hostPathWorkspaceVolume()` : a host path volume\n  * `nfsWorkspaceVolume()` : a nfs volume\n  * `persistentVolumeClaimWorkspaceVolume()` : an existing persistent volume claim by name.\n\n## Container template\n\nContainer templates are part of pod. They can be configured via the user interface or in a pipeline and allow you to set the following fields:\n\n* **name** The name of the container.\n* **image** The image of the container.\n* **envVars** Environment variables that are applied to the container **(supplementing and overriding env vars that are set on pod level)**.\n    * **envVar** An environment variable whose value is defined inline.\n    * **secretEnvVar** An environment variable whose value is derived from a Kubernetes secret.\n* **command** The command the container will execute. Will overwrite the Docker entrypoint. A typical value is `sleep`.\n* **args** The arguments passed to the command. A typical value is `99999999`.\n* **ttyEnabled** Flag to mark that tty should be enabled.\n* **livenessProbe** Parameters to be added to a exec liveness probe in the container (does not support httpGet liveness probes)\n* **ports** Expose ports on the container.\n* **alwaysPullImage** The container will pull the image upon starting.\n* **runAsUser** The user ID to run the container as.\n* **runAsGroup** The group ID to run the container as.\n\n#### Specifying a different default agent connection timeout\n\nBy default, the agent connection timeout is set to 1000 seconds. It can be customized using a system property. Please refer to the section below.\n\n#### Using yaml to define Pod Templates\n\nIn order to support any possible value in Kubernetes `Pod` object, we can pass a yaml snippet that will be used as a base\nfor the template. If any other properties are set outside the YAML, they will take precedence.\n\n```groovy\npodTemplate(yaml: '''\n    apiVersion: v1\n    kind: Pod\n    metadata:\n      labels: \n        some-label: some-label-value\n    spec:\n      containers:\n      - name: busybox\n        image: busybox\n        command:\n        - sleep\n        args:\n        - 99d\n    ''') {\n    node(POD_LABEL) {\n      container('busybox') {\n        echo POD_CONTAINER // displays 'busybox'\n        sh 'hostname'\n      }\n    }\n}\n```\n\nYou can use [`readFile`](https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#readfile-read-file-from-workspace) or [`readTrusted`](https://jenkins.io/doc/pipeline/steps/coding-webhook/#readtrusted-read-trusted-file-from-scm) steps to load the yaml from a file.\nAlso note that in declarative pipelines the `yamlFile` can be used (see this [example](examples/declarative_from_yaml_file)).\n\n##### Example\n\n`pod.yaml`\n```yaml\napiVersion: v1\nkind: Pod\nspec:\n  containers:\n  - name: maven\n    image: maven:3.9.9-eclipse-temurin-17\n    command:\n    - sleep\n    args:\n    - 99d\n  - name: golang\n    image: golang:1.16.5\n    command:\n    - sleep\n    args:\n    - 99d\n```\n\n`Jenkinsfile`\n```groovy\npodTemplate(yaml: readTrusted('pod.yaml')) {\n  node(POD_LABEL) {\n    // ...\n  }\n}\n```\n\n### Liveness Probe Usage\n```groovy\ncontainerTemplate(name: 'busybox', image: 'busybox', command: 'sleep', args: '99d',\n                  livenessProbe: containerLivenessProbe(execArgs: 'some --command', initialDelaySeconds: 30, timeoutSeconds: 1, failureThreshold: 3, periodSeconds: 10, successThreshold: 1)\n)\n```\nSee [Defining a liveness command](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#defining-a-liveness-command) for more details.\n\n# Inheritance\n\n## Overview\n\nA pod template may or may not inherit from an existing template.\n\nDepending on fields, the inheritance behaviour can vary. For simple values (strings), the child template will override the parent template. However for complex values (lists, maps), the child template will merge with the parent template.\n\n* **yaml** is merged according to the value of `yamlMergeStrategy` specified in the child pod template.\n* **Service account** and **Node selector** when are overridden completely substitute any possible value found on the 'parent'.\n* **Container templates** that are added to the podTemplate, that has a matching containerTemplate (a container template\nwith the same name) in the 'parent' template, will inherit the configuration of the parent containerTemplate.\nIf no matching container template is found, the template is added as is.\n* **Volume** inheritance works exactly as **Container templates**.\n* **Image Pull Secrets** are combined (all secrets defined both on 'parent' and 'current' template are used).\n\nIn the example below, we will inherit from a pod template we created previously, and will just override the version of\n`maven` so that it uses Java 21 instead:\n\n![image](images/mypod-1-general.png)\n![image](images/mypod-2-golang.png)\n![image](images/mypod-3-maven.png)\n\n```groovy\npodTemplate(inheritFrom: 'mypod', containers: [\n    containerTemplate(name: 'maven', image: 'maven:3.9.9-eclipse-temurin-21')\n  ]) {\n  node(POD_LABEL) {\n    …\n  }\n}\n```\n\nOr in declarative pipeline\n\n```groovy\npipeline {\n  agent {\n    kubernetes {\n      inheritFrom 'mypod'\n      yaml '''\n      spec:\n        containers:\n        - name: maven\n          image: maven:3.9.9-eclipse-temurin-21\n'''\n      …\n    }\n  }\n  stages {\n    …\n  }\n}\n```\n\nNote that we only need to specify the things that are different. So, `command` and `arguments` are not specified, as\nthey are inherited.\nAlso, the `golang` container will be added as defined in the 'parent' template.\n\n## Multiple Pod template inheritance\n\nField `inheritFrom` may refer a single pod template or multiple separated by space. In the later case each template will\nbe processed in the order they appear in the list *(later items overriding earlier ones)*.\nIn any case if the referenced template is not found it will be ignored.\n\n\n## Nesting Pod templates\n\nField `inheritFrom` provides an easy way to compose pod templates that have been pre-configured. In many cases it would\nbe useful to define and compose pod templates directly in the pipeline using groovy.\nThis is made possible via nesting. You can nest multiple pod templates together in order to compose a single one.\n\nThe example below composes two different pod templates in order to create one with maven and docker capabilities.\n\n```groovy\npodTemplate(containers: [containerTemplate(image: 'docker', name: 'docker', command: 'sleep', args: '99d')]) {\n    podTemplate(containers: [containerTemplate(image: 'maven', name: 'maven', command: 'sleep', args: '99d')]) {\n      node(POD_LABEL) { // gets a pod with both docker and maven\n        …\n      }\n    }\n}\n```\n\nThis feature is extra useful, pipeline library developers as it allows you to wrap pod templates into functions and let\nusers nest those functions according to their needs.\n\nFor example one could create functions for their podTemplates and import them for use.\nSay here's our file `src/com/foo/utils/PodTemplates.groovy`:\n```groovy\npackage com.foo.utils\n\npublic void dockerTemplate(body) {\n  podTemplate(\n        containers: [containerTemplate(name: 'docker', image: 'docker', command: 'sleep', args: '99d')],\n        volumes: [hostPathVolume(hostPath: '/var/run/docker.sock', mountPath: '/var/run/docker.sock')]) {\n    body.call()\n}\n}\n\npublic void mavenTemplate(body) {\n  podTemplate(\n        containers: [containerTemplate(name: 'maven', image: 'maven', command: 'sleep', args: '99d')],\n        volumes: [secretVolume(secretName: 'maven-settings', mountPath: '/root/.m2'),\n                  persistentVolumeClaim(claimName: 'maven-local-repo', mountPath: '/root/.m2repo')]) {\n    body.call()\n}\n}\n\nreturn this\n```\n\nThen consumers of the library could just express the need for a maven pod with docker capabilities by combining the two,\nhowever once again, you will need to express the specific container you wish to execute commands in.\nYou can **NOT** omit the `node` statement.\n\nNote that `POD_LABEL` will be the innermost generated label to get a node which has all the outer pods available on the\nnode, as shown in this example:\n\n```groovy\nimport com.foo.utils.PodTemplates\n\npodTemplates = new PodTemplates()\n\npodTemplates.dockerTemplate {\n  podTemplates.mavenTemplate {\n    node(POD_LABEL) {\n      container('docker') {\n        sh \"echo hello from $POD_CONTAINER\" // displays 'hello from docker'\n      }\n      container('maven') {\n        sh \"echo hello from $POD_CONTAINER\" // displays 'hello from maven'\n      }\n     }\n  }\n}\n```\n\nIn scripted pipelines, there are cases where this implicit inheritance via nested declaration is not wanted or another\nexplicit inheritance is preferred.\nIn this case, use `inheritFrom ''` to remove any inheritance, or `inheritFrom 'otherParent'` to override it.\n\n# Declarative Pipeline\n\nDeclarative agents can be defined from yaml\n\n```groovy\npipeline {\n  agent {\n    kubernetes {\n      yaml '''\n        apiVersion: v1\n        kind: Pod\n        metadata:\n          labels:\n            some-label: some-label-value\n        spec:\n          containers:\n          - name: maven\n            image: maven:3.9.9-eclipse-temurin-17\n            command:\n            - cat\n            tty: true\n          - name: busybox\n            image: busybox\n            command:\n            - cat\n            tty: true\n        '''\n      retries 2\n    }\n  }\n  stages {\n    stage('Run maven') {\n      steps {\n        container('maven') {\n          sh 'mvn -version'\n        }\n        container('busybox') {\n          sh '/bin/busybox'\n        }\n      }\n    }\n  }\n}\n```\n\nor using `yamlFile` to keep the pod template in a separate `KubernetesPod.yaml` file\n\n```groovy\npipeline {\n  agent {\n    kubernetes {\n      yamlFile 'KubernetesPod.yaml'\n      retries 2\n    }\n  }\n  stages {\n    …\n  }\n}\n```\n\nNote that it was previously possible to define `containerTemplate` but that has been deprecated in favor of the yaml format.\n\n```groovy\npipeline {\n  agent {\n    kubernetes {\n      //cloud 'kubernetes'\n      containerTemplate {\n        name 'maven'\n        image 'maven:3.9.9-eclipse-temurin-17'\n        command 'sleep'\n        args '99d'\n      }\n    }\n  }\n  stages {\n    …\n  }\n}\n```\n\nRun steps within a container by default. Steps will be nested within an implicit `container(name) {...}` block instead\nof being executed in the agent container.\n\n```groovy\npipeline {\n  agent {\n    kubernetes {\n      defaultContainer 'maven'\n      yamlFile 'KubernetesPod.yaml'\n    }\n  }\n\n  stages {\n    stage('Run maven') {\n      steps {\n        sh 'mvn -version'\n      }\n    }\n  }\n}\n```\n\nRun the Pipeline or individual stage within a custom workspace - not required unless explicitly stated.\n\n```groovy\npipeline {\n  agent {\n    kubernetes {\n      customWorkspace 'some/other/path'\n      defaultContainer 'maven'\n      yamlFile 'KubernetesPod.yaml'\n    }\n  }\n\n  stages {\n    stage('Run maven') {\n      steps {\n        sh 'mvn -version'\n        sh \"echo Workspace dir is ${pwd()}\"\n      }\n    }\n  }\n}\n```\n\n## Default inheritance\nUnlike scripted k8s template, declarative templates do not inherit from parent template.\nSince the agents declared at stage level can override a global agent, implicit inheritance was leading to confusion.\n\nYou need to explicitly declare the inheritance if necessary using the field `inheritFrom`.\n\nIn the following example, `nested-pod` will only contain the `maven` container.\n\n```groovy\npipeline {\n  agent {\n    kubernetes {\n      yaml '''\n        spec:\n        containers:\n        - name: golang\n            image: golang:1.16.5\n            command:\n            - sleep\n            args:\n            - 99d\n        '''\n    }\n  }\n  stages {\n    stage('Run maven') {\n        agent {\n            kubernetes {\n                yaml '''\n                    spec:\n                    containers:\n                    - name: maven\n                      image: maven:3.9.9-eclipse-temurin-17\n                      command:\n                      - sleep\n                      args:\n                      - 99d\n                    '''\n            }\n        }\n      steps {\n        …\n      }\n    }\n  }\n}\n\n```\n\n# Misc.\n\n## Accessing container logs from the pipeline\n\nIf you use the `containerTemplate` to run some service in the background\n(e.g. a database for your integration tests), you might want to access its log from the pipeline.\nThis can be done with the `containerLog` step, which prints the log of the\nrequested container to the build log.\n\n#### Required Parameters\n* **name** the name of the container to get logs from, as defined in `podTemplate`. Parameter name\ncan be omitted in simple usage:\n\n```groovy\ncontainerLog 'mongodb'\n```\n\n#### Optional Parameters\n* **returnLog** return the log instead of printing it to the build log (default: `false`)\n* **tailingLines** only return the last n lines of the log (optional)\n* **sinceSeconds** only return the last n seconds of the log (optional)\n* **limitBytes** limit output to n bytes (from the beginning of the log, not exact).\n\nAlso see the online help and [examples/containerLog.groovy](examples/containerLog.groovy).\n\n# Features controlled using system properties\n\nPlease read [Features controlled by system properties](https://www.jenkins.io/doc/book/managing/system-properties/) page to know how to set up system properties within Jenkins.\n\n* `KUBERNETES_JENKINS_URL` : Jenkins URL to be used by agents. This is meant to be used for OEM integration.\n* `io.jenkins.plugins.kubernetes.disableNoDelayProvisioning` (since 1.19.1) Whether to disable the no-delay provisioning strategy the plugin uses (defaults to `false`).\n* `io.jenkins.plugins.kubernetes.NoDelayProvisionerStrategy.disableCloudShuffle` Whether to disable the shuffling of clouds. When true clouds will be searched in order they are defined (defaults to `false`).\n* `jenkins.host.address` : (for unit tests) controls the host agents should use to contact Jenkins\n* `org.csanchez.jenkins.plugins.kubernetes.PodTemplate.connectionTimeout` : The time in seconds to wait before considering the pod scheduling has failed (defaults to `1000`)\n* `org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator.stdinBufferSize` : stdin buffer size in bytes for commands sent to Kubernetes exec api. A low value will cause slowness in commands executed. A higher value will consume more memory (defaults to `16*1024`)\n* `org.csanchez.jenkins.plugins.kubernetes.pipeline.ContainerExecDecorator.websocketConnectionTimeout` : Time to wait for the websocket used by `container` step to connect (defaults to `30`)\n\n# Running on OpenShift\n\n## Random UID problem\n\nOpenShift runs containers using a _random_ UID that is overriding what is specified in Docker images.\nFor this reason, you may end up with the following warning in your build\n\n```\n[WARNING] HOME is set to / in the agent container. You may encounter troubles when using tools or ssh client. This usually happens if the uid doesnt have any entry in /etc/passwd. Please add a user to your Dockerfile or set the HOME environment variable to a valid directory in the pod template definition.\n```\n\nAt the moment the jenkinsci agent image is not built for OpenShift and will issue this warning.\n\nThis issue can be circumvented in various ways:\n* build a docker image for OpenShift in order to behave when running using an arbitrary uid.\n* override HOME environment variable in the pod spec to use `/home/jenkins` and mount a volume to `/home/jenkins` to ensure the user running the container can write to it\n\nSee this [example](examples/openshift-home-yaml.groovy) configuration.\n\n## Running with OpenShift 3\n\nOpenShift 3 is based on an older version of Kubernetes, which is not anymore directly supported since Kubernetes plugin version 1.26.0.\n\nTo get agents working for Openshift 3, add this `Node Selector` to your Pod Templates:\n```\nbeta.kubernetes.io/os=linux\n```\n\n# Windows support\n\nYou can run pods on Windows if your cluster has Windows nodes.\nSee the [example](src/main/resources/org/csanchez/jenkins/plugins/kubernetes/pipeline/samples/windows.groovy).\n\n# Configuration on minikube\n\nCreate and start [minikube](https://github.com/kubernetes/minikube)\n\nThe client certificate needs to be converted to PKCS, will need a password\n\n    openssl pkcs12 -export -out ~/.minikube/minikube.pfx -inkey ~/.minikube/apiserver.key -in ~/.minikube/apiserver.crt -certfile ~/.minikube/ca.crt -passout pass:secret\n\nValidate that the certificates work\n\n    curl --cacert ~/.minikube/ca.crt --cert ~/.minikube/minikube.pfx:secret --cert-type P12 https://$(minikube ip):8443\n\nAdd a Jenkins credential of type certificate, upload it from `~/.minikube/minikube.pfx`, password `secret`\n\nFill *Kubernetes server certificate key* with the contents of `~/.minikube/ca.crt`\n\n\n# Configuration on Google Container Engine\n\nCreate a cluster\n\n    gcloud container clusters create jenkins --num-nodes 1 --machine-type g1-small\n\nand note the admin password and server certificate.\n\nOr use Google Developer Console to create a Container Engine cluster, then run\n\n    gcloud container clusters get-credentials jenkins\n    kubectl config view --raw\n\nthe last command will output kubernetes cluster configuration including API server URL, admin password and root certificate\n\n\n# Troubleshooting\n\nFirst watch if the Jenkins agent pods are started.\nMake sure you are in the correct cluster and namespace.\n\n    kubectl get -a pods --watch\n\nIf they are in a different state than `Running`, use `describe` to get the events\n\n    kubectl describe pods/my-jenkins-agent\n\nIf they are `Running`, use `logs` to get the log output\n\n    kubectl logs -f pods/my-jenkins-agent jnlp\n\nIf pods are not started or for any other error, check the logs on the controller side.\n\nFor more detail, configure a new [Jenkins log recorder](https://wiki.jenkins-ci.org/display/JENKINS/Logging) for\n`org.csanchez.jenkins.plugins.kubernetes` at `ALL` level.\n\nTo inspect the json messages sent back and forth to the Kubernetes API server you can configure\na new [Jenkins log recorder](https://wiki.jenkins-ci.org/display/JENKINS/Logging) for `okhttp3`\nat `DEBUG` level.\n\n## Deleting pods in bad state\n\n    kubectl get pods -o name --selector=jenkins=slave --all-namespaces  | xargs -I {} kubectl delete {}\n\n## Pipeline `sh` step hangs when multiple containers are used\nTo debug this you need to set `-Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true` system property\nand then restart the pipeline. Most likely in the console log you will see the following:\n```console\nsh: can't create /home/jenkins/agent/workspace/thejob@tmp/durable-e0b7cd27/jenkins-log.txt: Permission denied\nsh: can't create /home/jenkins/agent/workspace/thejob@tmp/durable-e0b7cd27/jenkins-result.txt.tmp: Permission denied\nmv: can't rename '/home/jenkins/agent/workspace/thejob@tmp/durable-e0b7cd27/jenkins-result.txt.tmp': No such file or directory\ntouch: /home/jenkins/agent/workspace/thejob@tmp/durable-e0b7cd27/jenkins-log.txt: Permission denied\ntouch: /home/jenkins/agent/workspace/thejob@tmp/durable-e0b7cd27/jenkins-log.txt: Permission denied\ntouch: /home/jenkins/agent/workspace/thejob@tmp/durable-e0b7cd27/jenkins-log.txt: Permission denied\n```\nUsually this happens when UID of the user in agent container differs from the one in another container(s). \nAll containers you use should have the same UID of the user, also this can be achieved by setting `securityContext`:\n```yaml\napiVersion: v1\nkind: Pod\nspec:\n  securityContext:\n    runAsUser: 1000 # default UID of jenkins user in agent image\n  containers:\n  - name: maven\n    image: maven:3.9.9-eclipse-temurin-17\n    command:\n    - cat\n    tty: true\n```\n\n## Using WebSockets with a Jenkins controller with self-signed HTTPS certificate\n\nUsing WebSockets is the easiest and recommended way to establish the connection between agents and a Jenkins controller running outside the cluster.\nHowever, if your Jenkins controller has HTTPS configured with self-signed certificate, you'll need to make sure the agent container trusts the CA.\nTo do that, you can extend the `jenkins/inbound-agent` image and add your certificate as follows:\n\n```Dockerfile\nFROM jenkins/inbound-agent\n\nUSER root\n\nADD cert.pem /tmp/cert.pem\n\nRUN keytool -noprompt -storepass changeit -cacerts \\\n  -import -file /tmp/cert.pem -alias jenkinsMaster \u0026\u0026 \\\n  rm -f /tmp/cert.pem\n\nUSER jenkins\n```\n\nThen, use it as the agent container for the pod template as usual. No command or args need to be specified.\n\n\u003e **Notes:**\n\u003e\n\u003e * Support for using WebSockets with JDK 11 was added in the Remoting v4.11, so make sure your base image is new enough. See [here](https://issues.jenkins.io/browse/JENKINS-61212) for more information.\n\u003e\n\u003e * When using the WebSocket mode, the `-disableHttpsCertValidation` on the `jenkins/inbound-agent` becomes unavailable, as well as `-cert`, and that's why you have to extend the docker image.\n\n## [WARNING] label option is deprecated\n\n```\n[WARNING] label option is deprecated. To use a static pod template, use the 'inheritFrom' option.\n```\n\nYou need to change from something like:\n\n```\nagent {\n\tkubernetes {\n\t\tlabel 'somelabel'\n\t}\n}\n```\n\nTo something like:\n\n```\nagent {\n\tkubernetes {\n\t\tinheritFrom 'somelabel'\n\t}\n}\n```\n\n\n# Building and Testing\n\nIntegration tests will use the currently configured context auto-detected from kube config file or service account.\n\n## Manual Testing\n\nRun `mvn clean install` and copy `target/kubernetes.hpi` to Jenkins plugins folder.\n\n## Running Kubernetes Integration Tests\n\nSet up your `$KUBECONFIG` however you like, for example\n\n```bash\nkind create cluster\n```\n\nthen\n\n```bash\nkubectl krew install tunnel # as needed; install Krew first\nkubectl tunnel expose jenkins 8000:8000 8001:8001 \u0026\nmvn test -Djenkins.host.address=jenkins.default -Dport=8000 -DslaveAgentPort=8001 -Dtest=KubernetesPipelineTest#runInPod\n```\n\nAlternately, you can run everything like in CI:\n\n```bash\nexport KIND_PRELOAD=true # optionally\n./kind.sh -Dtest=KubernetesPipelineTest#runInPod\n```\n\nYou can also run interactively after setting up the tunnel:\n\n```bash\nmvn hpi:run -Djenkins.host.address=jenkins.default -Dport=8000 -Djenkins.model.Jenkins.slaveAgentPort=8001\n```\n\n# Docker image\n\nDocker image for Jenkins, with plugin installed.\nBased on the [official image](https://hub.docker.com/r/jenkins/jenkins/).\n\n## Running the Docker image\n\n```bash\ndocker run --rm --name jenkins -p 8080:8080 -p 50000:50000 -v /var/jenkins_home csanchez/jenkins-kubernetes\n```\n\n\n# Running in Kubernetes\n\nThe example configuration will create a stateful set running Jenkins with persistent volume\nand using a service account to authenticate to Kubernetes API.\n\n## Running locally with minikube\n\nA local testing cluster with one node can be created with [minikube](https://github.com/kubernetes/minikube)\n\n```bash\nminikube start\n```\n\nYou may need to set the correct permissions for host mounted volumes\n\n```bash\nminikube ssh\nsudo chown 1000:1000 /tmp/hostpath-provisioner/pvc-*\n```\n\nThen create the Jenkins namespace, controller and Service with\n\n```bash\nkubectl create namespace kubernetes-plugin\nkubectl config set-context $(kubectl config current-context) --namespace=kubernetes-plugin\nkubectl create -f src/main/kubernetes/service-account.yml\nkubectl create -f src/main/kubernetes/jenkins.yml\n```\n\nGet the url to connect to with\n\n```bash\nminikube service jenkins --namespace kubernetes-plugin --url\n```\n\n## Running in Google Container Engine GKE\n\nAssuming you created a Kubernetes cluster named `jenkins` this is how to run both Jenkins and agents there.\n\nCreating all the elements and setting the default namespace\n\n```bash\nkubectl create namespace kubernetes-plugin\nkubectl config set-context $(kubectl config current-context) --namespace=kubernetes-plugin\nkubectl create -f src/main/kubernetes/service-account.yml\nkubectl create -f src/main/kubernetes/jenkins.yml\n```\n\nConnect to the ip of the network load balancer created by Kubernetes, port 80.\nGet the ip (in this case `104.197.19.100`) with `kubectl describe services/jenkins`\n(it may take a bit to populate)\n\n    $ kubectl describe services/jenkins\n    Name:           jenkins\n    Namespace:      default\n    Labels:         \u003cnone\u003e\n    Selector:       name=jenkins\n    Type:           LoadBalancer\n    IP:         10.175.244.232\n    LoadBalancer Ingress:   104.197.19.100\n    Port:           http    80/TCP\n    NodePort:       http    30080/TCP\n    Endpoints:      10.172.1.5:8080\n    Port:           agent   50000/TCP\n    NodePort:       agent   32081/TCP\n    Endpoints:      10.172.1.5:50000\n    Session Affinity:   None\n    No events.\n\nUntil Kubernetes 1.4 removes the SNATing of source ips, seems that CSRF (enabled by default in Jenkins 2)\nneeds to be configured to avoid `WARNING: No valid crumb was included in request` errors.\nThis can be done checking _Enable proxy compatibility_ under Manage Jenkins -\u003e Configure Global Security\n\nConfigure Jenkins, adding the `Kubernetes` cloud under configuration, setting\nKubernetes URL to the container engine cluster endpoint or simply `https://kubernetes.default.svc.cluster.local`.\nUnder credentials, click `Add` and select `Kubernetes Service Account`,\nor alternatively use the Kubernetes API username and password. Select 'Certificate' as credentials type if the\nkubernetes cluster is configured to use client certificates for authentication.\n\nUsing `Kubernetes Service Account` will cause the plugin to use the default token mounted inside the Jenkins pod. See [Configure Service Accounts for Pods](https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/) for more information.\n\n![image](credentials.png)\n\nYou may want to set `Jenkins URL` to the internal service IP, `http://10.175.244.232` in this case,\nto connect through the internal network.\n\nSet `Container Cap` to a reasonable number for tests, i.e. 3.\n\nAdd an image with\n\n* Docker image: `jenkins/inbound-agent`\n* Jenkins agent root directory: `/home/jenkins/agent`\n\n![image](configuration.png)\n\nNow it is ready to be used.\n\nTearing it down\n\n```bash\nkubectl delete namespace/kubernetes-plugin\n```\n\n\n## Customizing the deployment\n\n### Modify CPUs and memory request/limits (Kubernetes Resource API)\n\nModify file `./src/main/kubernetes/jenkins.yml` with desired limits\n\n```yaml\nresources:\n  limits:\n    cpu: 1\n    memory: 1Gi\n  requests:\n    cpu: 0.5\n    memory: 500Mi\n```\n\nNote: the JVM will use the memory `requests` as the heap limit (-Xmx)\n\n## Building\n\n```bash\ndocker build -t csanchez/jenkins-kubernetes .\n```\n \n# Related Projects\n\n* [Kubernetes Pipeline plugin](https://github.com/jenkinsci/kubernetes-pipeline-plugin): pipeline extension to provide native support for using Kubernetes pods, secrets and volumes to perform builds\n* [kubernetes-credentials](https://github.com/jenkinsci/kubernetes-credentials-plugin): Credentials provider that reads Kubernetes secrets\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjenkinsci%2Fkubernetes-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjenkinsci%2Fkubernetes-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjenkinsci%2Fkubernetes-plugin/lists"}