Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/funbox/pipeline-heavy-job-plugin
This plugin allows defining job weight for pipeline projects as HeavyJob Plugin does for freestyle projects.
https://github.com/funbox/pipeline-heavy-job-plugin
heavyjob jenkins pipeline
Last synced: 3 months ago
JSON representation
This plugin allows defining job weight for pipeline projects as HeavyJob Plugin does for freestyle projects.
- Host: GitHub
- URL: https://github.com/funbox/pipeline-heavy-job-plugin
- Owner: funbox
- License: mit
- Created: 2020-07-24T08:17:33.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2022-04-11T19:02:58.000Z (almost 3 years ago)
- Last Synced: 2024-04-29T17:09:43.928Z (10 months ago)
- Topics: heavyjob, jenkins, pipeline
- Language: Java
- Homepage:
- Size: 39.1 KB
- Stars: 5
- Watchers: 4
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Pipeline HeavyJob Plugin
This plugin allows defining job weights for pipeline projects like
[HeavyJob Plugin](https://github.com/jenkinsci/heavy-job-plugin) does for
freestyle projects.
[По-русски](./README.ru.md)
## Usage
Pipeline:
```groovy
nodeWithWeight(label: 'nodejs', weight: 2) {
echo 'Run heavy tests'
}
```
Pipeline Declarative Syntax:
```groovy
pipeline {
agent {
nodeWithWeight {
label 'nodejs'
weight 2
}
}
stages {
stage('Build') {
steps {
echo 'Run heavy build'
}
}
}
}
```
## Rationale
A typical big Jenkins setup consists of the main server with several worker nodes.
Each node has several slots. By default one build occupies one slot. Some builds
will wait in a queue if there are no more free slots available.
Each node has a limited amount of resources like CPU, RAM, or network bandwidth. Different
builds require different amounts of these resources but each build occupies only
one slot. That is why in some cases one node can be overloaded and another one
can be almost idle.
There are several ways to solve this problem.
Suppose we have two types of builds: light and heavy. A heavy build requires
twice as much resources as a light one. Suppose we have two worker nodes. Each
node has enough resources to run two parallel light builds or one heavy build.
We want to run as many builds as possible but do it in such a way that none of
the nodes will be overloaded.
### Separate worker for heavy builds
The idea is to configure Jenkins setup this way:
* worker 1 should have 2 slots and perform only light builds;
* worker 2 should have 1 slot and perform light and heavy builds.
Pros: workers will never be overloaded.
Cons: in some cases worker 2 will utilize only half of the available resources.
### lockable-resources
Configure 4 resources named “slot” using
[lockable-resources](https://www.jenkins.io/doc/pipeline/steps/lockable-resources/).
For a light build lock one “slot” resource:
```groovy
lock(label: 'slot', quantity: 1) {
echo 'Perform light build'
}
```
For a heavy build — lock 2 resources:
```groovy
lock(label: 'slot', quantity: 2) {
echo 'Perform heavy build'
}
```
Pros: workers will run as many builds as possible.
Cons: in some cases workers can be overloaded.
### Freestyle projects
It's possible to use freestyle projects with
[HeavyJob Plugin](https://github.com/jenkinsci/heavy-job-plugin) which solves
the problem completely.
Pros: workers will run as many builds as possible, workers won't be overloaded.
Cons: we can't use configuration as a code.
### Conclusion
Each described solution has pros and cons and it will be very useful to have
something like HeavyJob Plugin but for pipeline projects. As I've found in
[Jenkins Jira](https://issues.jenkins-ci.org/browse/JENKINS-41940) there are no
such plugins. That is why Pipeline HeavyJob Plugin was developed.
## Implementation details
This plugin implements `nodeWithWeight` step and `nodeWithWeight` agent based on
it. The code of the plugin is based on the code of
[workflow-durable-task-step-plugin](https://github.com/jenkinsci/workflow-durable-task-step-plugin).
[](https://funbox.ru)