Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/hangrybear666/08-devops-bootcamp__jenkins
Collection of Dockerfiles, Jenkinsfiles, Groovy and shell scripts for automating java/node web app test, versioning, publishing and deployment via declarative Jenkins pipelines on remote VPS hosts.
https://github.com/hangrybear666/08-devops-bootcamp__jenkins
Last synced: about 21 hours ago
JSON representation
Collection of Dockerfiles, Jenkinsfiles, Groovy and shell scripts for automating java/node web app test, versioning, publishing and deployment via declarative Jenkins pipelines on remote VPS hosts.
- Host: GitHub
- URL: https://github.com/hangrybear666/08-devops-bootcamp__jenkins
- Owner: hangrybear666
- License: mit
- Created: 2024-08-14T09:09:15.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2024-08-24T10:30:11.000Z (3 months ago)
- Last Synced: 2024-08-25T10:39:52.835Z (3 months ago)
- Language: Groovy
- Homepage:
- Size: 41 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Running Jenkins as docker container on remote VPS and executing multibranch declarative pipelines to test, version, publish and deploy web apps remotely.
Collection of Dockerfiles, Jenkinsfiles and shell scripts for automating java/node web app test, versioning, publishing and deployment via declarative Jenkins pipelines on remote VPS hosts.
The main projects are:
1. Scripts for installing docker, java, node & npm on remote and start a jenkins container with docker capabilities (Docker in Docker / DinD)
2. A `Jenkinsfile` for a declarative and a multibranch jenkins pipeline to build and push a Docker Image of a java maven application automatically on Github pushes via webhook.
3. A `Jenkinsfile_sharedLibrary` containing dynamically parameterized groovy scripts and classes for standardizing and reusing common functionality between pipelines in a git hosted shared library.
4. A `Jenkinsfile_versioning` demonstrating automatic version incrementation, while commiting the new version to Github from the pipeline itself without triggering new builds via SCM webhook triggers.
## Setup
### 1. Pull SCM
Pull the repository locally by running
```
git clone https://github.com/hangrybear666/08-devops-bootcamp__jenkins.git
```
### 2. Create Remote Linux VPS and configure
Generate local ssh key and add to remote VPS's `authorized_keys` file.
### 3. Install additional dependencies on remote
Some Linux distros ship without the `netstat` or `jq` commands we use. In that case run `apt install net-tools` or `dnf install net-tools` on fedora et cetera.
### 4. Create environment files
Add an `.env` file in your repository's root directory and add the following key value-pairs:
```
SERVICE_USER_PW=xxx
```
### 5. Add your remote VPS configuration parameters to `config/remote.properties`
```
REMOTE_ADDRESS=167.99.128.206
ROOT_USER="root"
SERVICE_USER="docker-runner"
```
### 6. Install docker locally.
Make sure to install docker and docker-compose (typically built-in) for local development. See https://docs.docker.com/engine/install/
### 7. Dont forget to open ports in your remote firewall.
50000 and 8080 for Jenkins, 22 for ssh.
### 8. Create Docker Hub Credentials and Repo
a. Sign up to Docker Hub https://hub.docker.com/ and create a personal access token at https://app.docker.com/settings/personal-access-tokens/
b. Save the Docker Hub Credentials in your `.env` file
c. Then create a new private Repository and add the URL as well
```
DOCKER_HUB_USER=xxx
DOCKER_HUB_TOKEN=xxx
DOCKER_HUB_REPO=hangrybear/devops_bootcamp
```
## Usage (Demo Projects)
0. Install docker and start jenkins in docker (dind) on remote VPS and install node and npm inside jenkins-dind container
*NOTE:* The installation script is aimed at Debian 12. See https://docs.docker.com/engine/install/debian/
```bash
cd scripts
./remote-install-docker.sh
./remote-install-java.sh
./remote-run-jenkins-in-docker.sh
./jenkins-container-install-node-npm.sh
```
To setup and configure jenkins, you first need to extract the standard admin password from the docker container on your remote:
```bash
ssh [email protected]
docker ps
# extract the hash from jenkins-dind container
docker exec -it bash
cat /var/jenkins_home/secrets/initialAdminPassword
```
Add your new admin credentials to `.env` file or store in another secure location to not lose access to jenkins.
-----
1. Run a declarative Jenkins Pipeline (and a multibranch pipeline) with embedded groovy script to build and push your java application to Docker Hub with Maven
##### a. Change the default value of `DOCKER_HUB_REPO_URL` in your `Jenkinsfile` file to your own and push the changes.or simply provide it as user input when building the pipeline with parameters.
##### b. Add `docker-hub-repo` credential-id to jenkins with your username and password you can find in your `.env` file after having run setup step 8.
##### c. Add your git credentials with the id `git-creds` and the username `x-token-auth` and fetch a personal access token from your git account.
##### d. Add Maven under Manage Jenkins -> Tools -> Maven and name it `Maven`.
##### e. Create a declarative pipeline under New Item -> Pipeline -> `java-app` and set it to get `java-app/Jenkinsfile` (!) from SCM under Definition and add your Git Credentials with the branch specifier `*/main`.
##### f. Create a multibranch pipeline under New Item -> Multibranch Pipeline -> `java-app-multibranch` and set it to get `java-app/Jenkinsfile` (!) from SCM under Definition and add your Git Credentials with the branch specifier `*`.
##### g. build both Pipelines manually in Jenkins UI. NOTE: You might have to run the multibranch pipeline twice and/or make changes to the branches after initially starting it, as the `DOCKER_HUB_REPO_URL` parameter might not get initialized and exposed correctly before SCM has pulled once.
-----
2. Run a declarative pipeline with dynamically parameterized code hosted in a Jenkins shared library for reusability and avoiding code duplication between microservices or teams.
*Note:* Follow the setup steps from the prior step, specifically b-d .
##### a. Change the default value of `DOCKER_HUB_REPO_URL` in your `Jenkinsfile_sharedLibrary` file to your own or simply provide it as user input when building the pipeline with parameters.
##### b. Change the library identifier and remote in your `Jenkinsfile_sharedLibrary` file to your own and push the changes.
##### c. Create a declarative pipeline under New Item -> Pipeline -> `java-app-sharedLibrary` and set it to get `java-app/Jenkinsfile_sharedLibrary` (!) from SCM under Definition and add your Git Credentials with the branch specifier `*/main`.
##### d. Build the pipeline
-----
3. To setup Github and to trigger both your declarative pipeline and your multibranch pipeline after code has been pushed automatically via webhook, follow these steps
##### a. Navigate to Manage Jenkins -> System -> Add Github Server with name `Github`, check the "Manage Hooks" checkbox and and add the API_URL https://api.github.com with a Github API Token as Jenkins credentials and add it to your `.env` file to not lose access. NOTE: The github token must have only webhook permissions, the rest is optional.
##### b. In your pipeline/s check the "Github Projects" flag and set https://github.com/hangrybear666/08-devops-bootcamp__jenkins. Then add the `GitHub hook trigger for GITScm polling` Flag.
##### c. In your Github Repository add your jenkins repo url on push events as hook, navigate to Settings -> Webhooks -> http://165.227.155.148:8080/github-webhook/
-----
4. To have your pipeline automatically increment the artifact version, add a dynamic docker image tag and commit the version changes to scm without triggered an automatic build via push hook, follow these steps
##### a. Open `Jenkinsfile_versioning` and set your own repository url in `git remote set-url origin` in the commit version update stage.
##### b. Create a declarative pipeline under New Item -> Pipeline -> `java-app-versioning` and set it to get `java-app/Jenkinsfile_versioning` (!) from SCM under Definition and add your Git Credentials with the branch specifier `*/jenkins-jobs`.
##### c. Manage Jenkins -> Available Plugins -> Ignore Committer Strategy -> Install. Configure your pipelines to avoid builds after version commits from jenkins itself via Plugin by navigating to your multibranch pipeline settings and adding `[email protected]` under Configuration -> Branch Sources -> Add -> Ignore Committer Strategy. NOTE: Make sure to check the `Allow builds when a changeset contains non-ignored author(s)` Flag!
##### d. For regular declarative pipeline we can use built-in functionality to ignore certain commits by author `jenkins` under Configuration -> Pipeline -> Additional Behaviors -> Add -> Polling ignores commits from certain users
##### e. Build the pipeline and check the `jenkins-jobs` branch in your github repository for recent version/pom.xml pushes from jenkins user.
-----