{"id":27058943,"url":"https://github.com/odennav/devsecops-pipeline-boardgame-app","last_synced_at":"2025-04-05T12:17:44.594Z","repository":{"id":245352108,"uuid":"818000365","full_name":"odennav/devsecops-pipeline-boardgame-app","owner":"odennav","description":"DevSecOps Pipeline with Jenkins for the Boardgame app build and deployment.","archived":false,"fork":false,"pushed_at":"2025-04-01T23:28:12.000Z","size":901,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T12:17:40.004Z","etag":null,"topics":["aws-ec2","cert-manager","devops","devsecops","docker","gitops","grafana","jenkins","jenkinsfile","kubeadm","kubeaudit","kubernetes","metallb","nexus","prometheus","sonarqube","terraform","traefik","trivy"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/odennav.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-20T22:20:38.000Z","updated_at":"2025-04-04T20:19:08.000Z","dependencies_parsed_at":"2025-04-05T12:17:41.179Z","dependency_job_id":null,"html_url":"https://github.com/odennav/devsecops-pipeline-boardgame-app","commit_stats":null,"previous_names":["odennav/boardgame-devops-pipeline-project","odennav/boardgame-devsecops-pipeline","odennav/devsecops-pipeline-boardgame-app"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/odennav%2Fdevsecops-pipeline-boardgame-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/odennav%2Fdevsecops-pipeline-boardgame-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/odennav%2Fdevsecops-pipeline-boardgame-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/odennav%2Fdevsecops-pipeline-boardgame-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/odennav","download_url":"https://codeload.github.com/odennav/devsecops-pipeline-boardgame-app/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247332547,"owners_count":20921854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-ec2","cert-manager","devops","devsecops","docker","gitops","grafana","jenkins","jenkinsfile","kubeadm","kubeaudit","kubernetes","metallb","nexus","prometheus","sonarqube","terraform","traefik","trivy"],"created_at":"2025-04-05T12:17:43.296Z","updated_at":"2025-04-05T12:17:44.567Z","avatar_url":"https://github.com/odennav.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Boardgame DevOps Pipeline Project\n\nImplement a DevSecOps Pipeline with Jenkins for the Boardgame app.\n\n-----\n\n![](https://github.com/odennav/boardgame-devops-pipeline-project/blob/main/docs/Boardgame-pipeline.png)\n\n## Getting Started\n\n**Pre-requisites**\n\n- Find the soure code for the Boardgame app [here](https://github.com/odennav/boardgame-app)\n\n- Install [Terraform](https://developer.hashicorp.com/terraform/install)\n\n- Install [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)\n\nGenerate `terraform-key` required for connection to EC2 instances in AWS VPC.\n\nChoose `RSA` key pair type and use `.pem` key file format.\n\n\nWe'll implement the following workflow:\n\n- Provision Servers with Terraform\n\n- Ansible Setup and User Configuration\n\n- Setup Kubernetes Cluster with Kubeadm\n\n- Setup MetalLB\n\n- Setup Traefik Proxy\n\n- Setup Cert-manager\n\n- Jenkins Installation and Configuration\n\n- Install Jenkins Plugins\n\n- SonarQube Installation and Setup\n\n- Sonatype Nexus Installation and Setup\n\n- Setup Trivy\n\n- DockerHub Setup\n\n- Setup Mail Notifications\n\n- Pipeline Setup with Jenkinsfile\n\n- Setup Prometheus and Grafana for Monitoring and Observability\n\n\nPlease check the `inventory` list for reference to IP addresses of `EC2` instances used in this project.\n\n-----\n\n## Provision Servers with Terraform\n\nInstall AWS CLI in `local` machine\n```bash\nsudo apt install curl unzip\ncurl \"https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip\" -o \"awscliv2.zip\"\nunzip awscliv2.zip\nsudo ./aws/install -i /usr/local/aws-cli -b /usr/local/bin\n```\n\nConfirm the AWS CLI installation\n```bash\naws --version\n```\n\nClone this repository in the `local` machine\n```bash\ncd /\ngit clone git@github.com:odennav/boardgame-devops-pipeline-project.git\n```\n\nExecute these Terraform commands sequentially in the `local` machine to create the `Virtual Private Cloud` and `EC2` instances.\n\nInitializes terraform working directory\n```bash\ncd boardgame-devops-pipeline-project/terraform\nterraform init\n```\n\nValidate the syntax of the terraform configuration files\n```bash\nterraform validate\n```\n\nCreate an execution plan that describes the changes terraform will make to the infrastructure\n```bash\nterraform plan\n```\n\nApply the changes described in execution plan\n```bash\nterraform apply -auto-approve\n```\n\nCheck AWS console for instances created and running\n\n**SSH access**\n\nUse `.pem` key from AWS to SSH into the public EC2 instance. IPv4 address of public EC2 instance will be shown in terraform outputs.\n```bash\nssh -i private-key/terraform-key.pem ec2-user@\u003cipaddress\u003e\n```\n\nWe can use public EC2 instance as a jumpbox to securely SSH into private EC2 instances within the VPC.\n\nNote, the ansible `inventory` is built dynamically by terraform with the private ip addresses of the `EC2` machines.\n\n-----\n\n## Ansible Setup and User Configuration\n\nGenerate SSH public/private key pair in `build` machine\n```bash\nssh-keygen -t rsa -b 4096\n```\n\nOnce the `RSA` key-pair is generated, manually copy the public key `id_rsa.pub` to the `/root/.ssh/authorized_keys` file in all kube nodes.\n\n\nInstall Ansible in `build` machine\n```bash\nsudo apt install -y software-properties-common\nsudo add-apt-repository --yes --update ppa:ansible/ansible\nsudo apt install -y ansible\n```\n\nThe `bootstrap` and `kubeadm` folders in this repository contain the ansible scripts necessary to set up your servers with the required packages and applications.\n\n\n**Bootstrapping EC2 Nodes**\n\nAll nodes will be bootstrapped using Ansible.\n\nBootstrap the master node\n```bash\ncd boardgame-devops-pipeline-project/bootstrap/\nansible-playbook bootstrap.yml --limit k8s_master\n```\n\nBootstrap the worker nodes\n```bash\nansible-playbook bootstrap.yml --limit k8s_node\n```\n\nOnce the bootstrap is complete, you can log in as `odennav-admin`.\n\nConfirm SSH access to master node\n```bash\nssh odennav-admin@10.33.100.2\n```\n\nTo return to the `build` machine, type exit and press Enter or use Ctrl+D\n\nConfirm SSH access to 1st worker node\n```bash\nssh odennav-admin@10.33.100.3\n```\n\nConfirm SSH access to 2nd worker node\n```bash\nssh odennav-admin@10.33.100.4\n```\n\n-----\n\n## Setup Kubernetes Cluster with Kubeadm\n\n**Setting up Kubernetes Cluster**\n\nThe kube nodes are now ready to have a Kubernetes cluster installed on them.\n\nPlease note amazon `EKS` is preferred for production but it's not deployed in this project to reduce the financial costs incurred in service bill.\n\nExecute ansible role playbook for the kubernetes master node\n```bash\ncd boardgame-devops-pipeline-project/kubeadm/\nansible-playbook k8s.yml  --limit k8s_master\n```\n\nExecute ansible role playbook for the kubernetes worker nodes\n```bash\nansible-playbook k8s.yml  --limit k8s_node\n```\n\nCheck status of your nodes from the `k8smaster` node\n```bash\nkubectl get nodes\n```\n\nThe Kubernetes cluster should be ready as shown below.\n\n![](https://github.com/odennav/boardgame-devops-pipeline-project/blob/main/docs/cluster.png)\n\n\n**Install Kubeaudit**\n\nAudit the Kubernetes clusters for various security concerns with `Kubeaudit` tool.\n\nDownload the binary relase\n```bash\ncd $HOME\nwget https://github.com/Shopify/kubeaudit/releases/download/v0.22.1/kubeaudit_0.22.1_linux_amd64.tar.gz\n```\n\nExtract the tarball file\n```bash\ntar -xzvf kubeaudit_0.22.1_linux_amd64.tar.gz\n```\n\nMove kubeaudit to bin directory in $PATH\n```bash\nsudo mv kubeaudit /usr/local/bin/\n```\n\nUse `cluster` mode and audit all Kubernetes resources in the cluster\n```bash\nkubeaudit all\n```\n\n**Setup RBAC in Kubernetes Cluster**\n\nRole Based Access Control  is a method of regulating access to computer or network resources based on the roles of individual users.\n\nWe'll start by creating the boardgame namespace\n```bash\nkubectl create namespace boardgame\n```\n\nView the manifest for the service account\n```yaml\napiVersion: v1\nkind: ServiceAccount\nmetadata: \n  name: jenkins\n  namespaces: boardgame\n```\n\nCreate a service account for Jenkins \n```bash\nkubectl apply -f boardgame-devops-pipeline-project/kubernetes-manifests/service_account.yaml\n```\n\nCreate a role to assign to the service account\n```bash\nkubectl apply -f boardgame-devops-pipeline-project/kubernetes-manifests/role.yaml\n```\n\nBind the role to service account \n```bash\nkubectl apply -f boardgame-devops-pipeline-project/kubernetes-manifests/role_binding.yaml\n```\n \nCreate API token for Jenkins to athenticate to the kubernetes cluster API\n```bash\nkubectl apply -f boardgame-devops-pipeline-project/kubernetes-manifests/svc_account_token.yaml\n```\n\nView API token created in kubernetes secret\n```bash\nkubectl describe secret jenkins_secret -n boardgame\n```\n\nThis token will be used to create a global credential for kubernetes tool in Jenkins.\n\n\n**Create Kubernetes Secret Credential in Jenkins**\n\nGo to Jenkins `Dashboard` and select `Manage Jenkins`.\n\nUnder the `Security` section, select `Credentials`\n\nClick on `(global)` domain of jenkins `System` store.\n\nNext, click on blue button `+ Add Credentials` at the top right.\n\nAssign the following:\n\nKind -------------------------\u003e Secret text\n\nScope ------------------------\u003e Global\n\nSecret -----------------------\u003e \u003cjenkins_secret token\u003e\n\nID ------------------------- \u003e k8s-cred\n\nDescription ------------------\u003e k8s-cred-token\n\n-----\n\n\n## Setup MetalLB\n\nMetalLB is a load-balancer implementation for bare metal Kubernetes clusters which will allocate IP addresses to services within the Kubernetes cluster.\n\nEnable `strictARP` mode to use layer2 ARP protocol\n```bash\nkubectl get configmap kube-proxy -n kube-system -o yaml | \\\nsed -e \"s/strictARP: false/strictARP: true/\" | \\\nkubectl apply -f - -n kube-system\n```\n\nProceed with installation by manifest\n```bash\nkubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.5/config/manifests/metallb-native.yaml\n```\n\nDefine a pool of addresses to assign to services with `IPAddressPool` resource\n```bash\ncd boardgame-devops-pipeline-project/metallb/\nkubectl apply -f IPAddressPool.yml\n```\n\nConfigure `L2Advertisement` resource to advertise this IP pool to local network.\n```bash\nkubectl apply -f L2Advertisement.yml\n```\n\n-----\n\n## Setup Traefik Proxy\n\nTraefik proxy is a modern cloud native application proxy that will automate the discovery, routing, and load balancing of services in the kubernetes cluster.\n\nCreate namespace for traefik\n```bash\nkubectl create namespace traefik\n```\n\nConfirm traefik namespace created\n```bash\nkubectl get namespaces\n```\n\nAdd traefik repository with helm package manager\n```bash\nhelm repo add traefik https://helm.traefik.io/traefik\n```\n\nUpdate local cache of helm chart repositories\n```bash\nhelm repo update\n```\n\nInstall traefik chart with custom values file `values.yaml` in traefik namespace\n```bash\ncd boardgame-devops-pipeline-project/traefik/\nhelm install --namespace=traefik traefik traefik/traefik --values=values.yaml\n```\n\nVerify Traefik is installed\n```bash\nhelm status traefik --namespace=traefik\n```\n\nVerify the status of the traefik ingress controller service\nkubectl get svc --namespace traefik\n\nConfirm the traefik deployment pods in `traefik` namespace are ready and running\n```bash\nkubectl get pods --namespace traefik\n```\n\n**Setup Route to Traefik Dashboard**\n\nApply middleware configuration for ingress route to the boardgame service\n```bash\nkubectl apply -f default-headers.yaml\n```\n\nConfirm middleware is created\n```bash\nkubectl get middleware\n```\n\nInstall the apache2-utils package to use the `htpassword` tool\n```bash\nsudo apt-get update\nsudo apt-get install -y apache2-utils\n```\n\nGenerate a base64 encoded password\n```bash\nhtpasswd -nb odennav-admin mypassword | openssl base64\n```\nCopy encoded password to the `traefik/traefik-dasboard/secret-dashboard.yaml` file.\n\nApply the secret to the kubernetes cluster\n```bash\ncd boardgame-devops-pipeline-project/traefik/traefik-dasboard/dasboard/\nkubectl apply -f secret-dashboard.yaml\n```\n\nVerify secret created\n```bash\nkubectl get secrets --namespace traefik\n```\n\nApply middleware configuration for ingress route to traefik dashboard\n```bash\nkubectl apply -f middleware.yaml\n```\n\nApply ingress route to the traefik dashboard\n```bash\nkubectl apply -f ingress.yaml\n```\n\n\n## Setup Cert-manager\n\ncert-manager is a powerful \u0026 extensible X.509 certificate controller used to automatically provision and manage TLS certificates in Kubernetes clusters.\n\nCreate namespace for cert-manager\n```bash\nkubectl create namespace cert-manager\n```\n\nConfirm cert-manager namespace created\n```bash\nkubectl get namespaces\n```\n\nAdd jetstack repository with helm package manager\n```bash\nhelm repo add jetstack https://charts.jetstack.io\n```\n\nUpdate local cache of helm chart repositories\n```bash\nhelm repo update\n```\n\nApply CRDs for cert-manager\n```bash\nkubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.15.0/cert-manager.crds.yaml\n```\n\nInstall cert-manager chart with custom values file `values.yaml` in cert-manager namespace\n```bash\ncd boardgame-devops-pipeline-project/traefik/traefik-dasboard/cert-manager/\nhelm install cert-manager jetstack/cert-manager --namespace cert-manager --values=values.yaml --version v1.15.0\n```\n\nConfirm all cert-manager deployment pods in `cert-manager` namespace are ready and running\n```bash\nkubectl get pods --namespace cert-manager\n```\n\n**Configure Staging Certificate**\n\nAt this moment, we'll use the `staging` endpoint for certificate propagation.\n\nThe api url for the `production` endpoint is configured in a way that triggers a block for days/weeks if there are too many failures.\n\nThe `Staging` endpoint wont give us a trusted certificate but it will assign one that is signed from staging servers.\n\nIf we get `staging` certificate, thats good. Then we can use the `caServer` endpoint url to get production certificate.\n\nI've registered a domain `odennav.com` with a domain registrar.\n\nUpdate nameservers set with your registrar to ensure this domain points to the authoritave nameservers received from [Cloudflare](https://www.cloudflare.com).\n\nCloudflare will be used for DNS verification by Let's Encrypt.\n\n\nGo to your cloudflare account and create a custom token. Implement the following:\n\n- On your profile page, select `{} API Tokens`\n\n- Click on `Create Token` and select `Get Started` to create a custom token.\n\nEnter the following:\n\nToken name --------------------\u003e Odennav-Docker-Traefik\n\n1st Permissions ---------------\u003e Zone \u0026 Zone \u0026 Read\n\n2nd Permissions ---------------\u003e Zone \u0026 DNS \u0026 Edit\n\nZone Resources ----------------\u003e Include \u0026 Specific zone \u0026 odennav.com\n\nClick on `Continue to summary` and  on the summary page, click on `Create Token`\n\nCopy the token shown to you for access to the Cloudflare API and save it to the `cert-manager/issuer/secret-cf-token.yaml` file.\n\n\nApply the secrets for cloudflare secret token to the kubernetes cluster\n```bash\ncd boardgame-devops-pipeline-project/traefik/traefik-dasboard/cert-manager/issuers/\nkubectl apply -f secret-cf-token.yaml\n```\n\nApply the staging ClusterIssuer resource\n```bash\nkubectl apply -f letsencrypt-staging.yaml\n```\n\nCreate a staging certificate for the `default` namespace\n```bash\ncd boardgame-devops-pipeline-project/traefik/traefik-dasboard/cert-manager/certificates/staging\nkubectl apply -f local-odennav-com.yaml\n```\n\n**Configure DNS for Traefik Proxy**\n\nIn this step, configure the DNS within your `Cloudflare` account, using a domain, `odennav.com` that you own.\n\nCreate the domain `A` records for each `Host` configured for the ingress route resources:\n\n- `boardgame.odennav.com`\n\n- `traefik.odennav.com`\n\nIdentify the load balancer `EXTERNAL-IP` created by the `traefik` deployment and use it to add required `A` records for the `hosts` above.\n\n```bash\nkubectl get svc -n traefik\n```\n\nView the traefik dashboard on a broswer and note the staging certificate issued.\n\n\n**Configure Production Certificate**\n\nOnce the staging certificate from let's encrypt is confirmed, we can now go to production.\n\nApply production ClusterIssuer. This points to Let's Encrypt production endpoint.\n```bash\ncd boardgame-devops-pipeline-project/traefik/traefik-dasboard/cert-manager/issuers/\nkubectl apply -f letsencrypt-production.yaml\n```\n\nCreate a production certificate resource. This creates a `TXT` record at cloudflare and verifies it to create our production certificate.\n```bash\ncd boardgame-devops-pipeline-project/traefik/traefik-dasboard/cert-manager/certificates/production\nkubectl apply -f local-odennav-com.yaml\n```\n\nCheck for DNS solver challenges. If no challenges after `pending` or `valid` states are resolved, then its working well.\n```bash\nkubectl get challenges\n```\n\nReplace the value of the `secretName` variable defined in the `ingress.yaml` file for traefik dashboard. Use `local-odennav-com-tls` as name of certificate secret.\n\nThis secret stores the production certificate in the `IngressRoute` resource for the traefik dashboard. \n\nRecreate the IngressRoute.\n```bash\ncd cd boardgame-devops-pipeline-project/traefik/traefik-dasboard/dasboard/\nkubectl apply -f ingress.yaml\n```\n-----\n\n## Jenkins Installation and Configuration\n\n**Install Jenkins in Ubuntu**\n\nWe'll use the long term support release which is installed from debian-stable apt repository.\n\n```bash\nsudo wget -O /usr/share/keyrings/jenkins-keyring.asc \\\n  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key\necho \"deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]\" \\\n  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \\\n  /etc/apt/sources.list.d/jenkins.list \u003e /dev/null\nsudo apt-get update\nsudo apt-get install jenkins\n```\n\nEnable jenkins service\n```bash\nsudo systemctl enable jenkins\n```\n\nStart jenkins service\n```bash\nsudo systemctl start jenkins\n```\n\nConfirm jenkins service is active and running\n```bash\nsudo systemctl status jenkins\n```\n\n**Post Installation Setup**\n\nNext we use the post-installation setup wizard to unlock jenkins, customize plugins and create first admin user required to continue accessing jenkins.\n\n - Browse to `10.33.100.7:8080` to see the **`Unlock Jenkins`** page.\n\n- Obtain the automatically-generated alphanumeric password\n  ```bash\n  sudo cat /var/jenkins_home/secrets/initialAdminPassword\n  ```\n- Paste this password into the `Administrator password` field and click `Continue` to access jenkin's main UI.\n\n**Customize Jenkins with Plugins**\n\nAfter unlocking Jenkins, the **`Customize Jenkins`** page appears.\n\nHere you can install any number of useful plugins as part of your initial setup.\n\nClick on `Install suggested plugins` to install the recommended set of plugins, which are based on most common use cases.\n\n**Create First Administrator User**\n\nFinally, after customizing Jenkins with plugins, Jenkins asks you to create your first administrator user.\n\nWhen the **`Create First Admin User`** page appears, specify the details for your administrator user in the respective fields and click `Save and Continue`.\n\nWhen the Jenkins is ready page appears, click `Start using Jenkins`\n\nThis page may indicate Jenkins is almost ready! instead and if so, click `Restart`.\n\nIf the page does not automatically refresh after a minute, use your web browser to refresh the page manually.\n\nIf required, log in to Jenkins with the credentials of the user you just created and you are ready to start using Jenkins.\n\n\n**Install Kubectl using native package management**\n\nThe kubectl version will be the same version with the kubernetes cluster.\n\nWe'll use this ansible script below to install `kubectl` in the jenkins machine.\n\n```bash\ncd boardgame-devops-pipeline-project/jenkins/\nansible-playbook kubectl_install.yml \n```\n\n-----\n\n## Install Plugins\n\nPlugins are required to integrate tools to Jenkins and execute in our pipeline script.\n\nGo to `Plugin Manager` under `Manage Jenkins` section of Jenkins dasboard.\n\nOur next task is to search and install the following plugins below:\n\n- Eclipse Temurin installer\n\n- Config File Provider\n\n- Pipeline Maven Integration\n\n- SonarQube Scanner \n\n- Maven Integration\n\n- Docker\n\n- Docker Pipeline\n\n- docker-build-step\n\n- CloudBees Docker Build and Publish\n\n- Kubernetes\n\n- Kubernetes CLI\n\n- Kubernetes Credentials\n\n- Kubernetes Client API\n\n- Trivy\n\nSelect `Install without restart` at bottom left.\n\n\n**Configure Other Global Tools**\n\nWhen plugins selected are installed, next we configure them.\n\nGo to `Global Tool Configuration` under `Manage Jenkins` section of Jenkins dasboard\n\nNote procedures to configure jdk and docker as global tools below:\n\n\n**Procedure - JDK**: \n\nScroll down and search for **`JDK installations`**\n\nClick on `Add JDK`\n\nEnter or select the following:\n\n - Name -----------------------------\u003e jdk17\n\n- Install automatically `?` ------------\u003e ✔️\n\n- Add Installer --------------------\u003e Install from adoptium.net\n\n- Version --------------------------\u003e jdk-17.0.19+9\n\n\n**Procedure - Docker**:\n\nScroll down and search for **`Docker installations`**  \n\nClick on `Add Docker`\n\nEnter or select the following:\n\n- Name -----------------------------\u003e docker\n\n- Install automatically `?` ------------\u003e ✔️\n\n- Add Installer --------------------\u003e Download from docker.com\n\n- Docker version `?` --------------------------\u003e latest\n\n\n\n**Procedure - SonarQube Scanner**:\n\nScroll down and search for **`SonarQube Scanner installations`** \n\nClick on `Add SonarQube Scanner`\n\nEnter or select the following:\n\n- Name -----------------------------\u003e sonar-scanner\n\n- Install automatically `?` ------------\u003e ✔️\n\n- Version --------------------------\u003e latest(SonarQube Scanner 5.0.1.3006)\n\n\n\n**Procedure - Maven**:\n\nScroll down and search for **`Maven installations`**\n\nClick on `Add Maven`\n\nEnter or select the following:\n\n- Name -----------------------------\u003e maven3\n\n- Install automatically `?` ------------\u003e ✔️\n\n- Version --------------------------\u003e 3.6.1 or latest \n\n\nClick on `Apply` to save configuration.\n\n-----\n\n## SonarQube Installation and Setup\n\nSonarQube is a code quality assurance tool that collects and analyzes source code, providing reports for the code quality of our project.\n\nIt enables us to deploy clean code consistently and reliably.\n\nThe sonarqube machine will be bootstrapped using Ansible.\n\n```bash\ncd boardgame-devops-pipeline-project/bootstrap/\nansible-playbook bootstrap.yml --limit sonarqube\n```\n\nOnce the bootstrap is complete, you can log in as `odennav-admin`.\n\nConfirm SSH access to the `sonarqube` node\n```bash\nssh odennav-admin@10.33.100.5\n```\n\n**Install Docker**\n\nRun the ansible playbook to install docker in the nexus machine\n```bash\ncd boardgame-devops-pipeline-project/docker/\nansible-playbook install-docker.yml --limit sonarqube\n```\n\n**Install SonarQube Container**\n\nWe'll run the long term community version of sonarqube's image.\n```bash\ndocker run -d --name sonar -p 9000:9000 sonarqube:lts-community\n```\n\nConfirm container is running\n```bash\ndocker ps -a --filter \"name=sonar\"\n```\n\nBrowse to UI of SonarQube at `10.33.100.5:9000`\n\nUse `admin` for default username and password. Update to new password when requested.\n\n**Generate Token**\n\nGo to **`Administration`** tab and select `Security` tab.\n\nFrom the drop down, click on `Users`.\n\nThis section is used to create and administer individual users.\n\nClick on button at far right under `Tokens` column.\n\nEnter `Token Name` as `sonar-token` and click on `Generate`. Note period of expiry.\n\nCopy this access code, we'll use it to create credential for SonarQube in jenkins.\n\n\n**Create SonarQube Secret for Jenkins**\n\nGo to Jenkins `Dashboard` and select `Manage Jenkins`.\n\nUnder the `Security` section, select `Credentials`\n\nClick on `(global)` domain of jenkins `System` store.\n\nNext, click on blue button `+ Add Credentials` at the top right.\n\nAssign the following:\n\nKind -------------------------\u003e Secret text\n\nName ------------------------- \u003e sonar-token\n\nScope ------------------------\u003e Global\n\nSecret -----------------------\u003e `sonar-token`\n\nDescription ------------------\u003e sonar-token generated\n\n\n**Configure SonarQube Server**\n\nGo to Jenkins `Dashboard` and select `Manage Jenkins`.\n\nUnder the `System Configuration` section, select `Configure System`.\n\nScroll down and search for **`SonarQube servers`** and `SonarQube installatons`\n\nClick `Add SonarQube` and assign the following:\n\nName ---------------------------------\u003e sonar\n\nServer URL ---------------------------\u003e `https://10.33.100.5:9000`\n\nServer authentication token ----------\u003e sonar-token\n\nClick on `Apply` and `Save`.\n\n\n**SonarQube QualityGate Setup**\n\nGo to **`Administration`** tab and select `Configuration` drop-down tab.\n\nFrom the drop down, click on `Webhooks`.\n\nThis section is used to create webhooks used to notify external services when a project analysis is done.\n\nClick the `Create` button at top-right section.\n\nTo create webhook, assign the following:\n\nName -------------------------\u003e jenkins\n\nURL --------------------------\u003e `https://10.33.100.7:8080/sonarqube-webhook/`\n\nSecret -----------------------\u003e ''\n\nSave this webhook. It will be used in Jenkins pipeine script\n\n\n-----\n\n## Sonatype Nexus Installation and Setup\n\n\n**Bootstrap Nexus Node**\n\nThe nexus machine will be bootstrapped using Ansible.\n\n```bash\ncd boardgame-devops-pipeline-project/bootstrap/\nansible-playbook bootstrap.yml --limit nexus\n```\n\nOnce the bootstrap is complete, you can log in as `odennav-admin`.\n\nConfirm SSH access to the `nexus` node\n```bash\nssh odennav-admin@10.33.100.6\n```\n\n**Install Docker**\n\nRun the ansible playbook to install docker in the nexus machine\n```bash\ncd boardgame-devops-pipeline-project/docker/\nansible-playbook install-docker.yml --limit nexus\n```\n\nRun Nexus Container with the latest version of nexus3 docker image.\n```bash\ndocker run -d --name Nexus -p 8081:8081 sonatype/nexus3\n```\n\nConfirm container is running\n```bash\ndocker ps -a --filter \"name=Nexus\"\n```\n\nBrowse to UI of Nexus at `10.33.100.6:8081`\n\n\nNote, the username to sign into the Sonatype Nexus Repository is `admin` . \n\nThe password is located in `/nexus-data/admin.password` file in the container.\n\n```bash\ndocker -exec -it Nexus /bin/bash \ncat sonatyp-work/nexus-data/admin.password\n```\n\nCopy the password and use it to sign in. Then choose a new password for the `admin` user.\n\n\n\n**Sonatype Nexus Artifact Repository Setup**\n\nTo publish artifacts to Nexus, we'll add the nexus repository urls to the `pom.xml` file in the source code project.\n\n \nAdd the urls of the following nexus repositories to the `pom.xml` file:\n\n- Nexus releases\n\n- Nexus snapshots\n\n```text\n        \u003cdistributionManagement\u003e\n        \u003crepository\u003e\n            \u003cid\u003emaven-releases\u003c/id\u003e\n            \u003curl\u003ehttp://10.33.100.6:8081/repository/maven-releases/\u003c/url\u003e\n        \u003c/repository\u003e\n        \u003csnapshotRepository\u003e\n            \u003cid\u003emaven-snapshots\u003c/id\u003e\n            \u003curl\u003ehttp://10.33.100.6:8081/repository/maven-snapshots/\u003c/url\u003e\n        \u003c/snapshotRepository\u003e\n        \u003c/distributionManagement\u003e\n```\n\n**Configure Nexus Global Settings Credential**\n\nEnsure the `Config File Provider` plugin is already installed.\n\nWhen plugins selected are installed, go to **`Global Tool Configuration`** under `Manage Jenkins` section of Jenkins dasboard\n\nSelect **`Managed files`** and click on `+ Add a new Config` at left tab.\n\nAssign the following:\n\nType -----------------------------\u003e `Global Maven settings.xml` \n\nID of the config file ------------\u003e global-settings\n\nClick `Next`.\n\nNext, add the following to the end of the configuration file, `settings.xml` in the `Content` section\n\nThe `\u003cservers\u003e` section in the `settings.xml` file specifies the authentication information to use when connecting to the Nexus server.\n\n```text\n\u003cserver\u003e\n  \u003cid\u003emaven-releases\u003c/id\u003e\n  \u003cusername\u003eadmin\u003c/username\u003e\n  \u003cpassword\u003epassword\u003c/password\u003e\n\u003c/server\u003e\n\n\u003cserver\u003e\n  \u003cid\u003emaven-snapshots\u003c/id\u003e\n  \u003cusername\u003eadmin\u003c/username\u003e\n  \u003cpassword\u003epassword\u003c/password\u003e\n\u003c/server\u003e\n```\n\nClick on `Submit`\n\nNote the name of this global settings configuration file is `MyGlobalSettings`.\n\n-----\n\n## GitHub Integration\n\nPersonal access token from Github is needed for Jenkins to access source code files.\n\nTo create token, implement the following:\n\n-  Click on your profile at the top right, scroll down and click on `:gear: Settings`\n\n- Scroll down at left bar and select `\u003c\u003e Developer Settings`\n\n- Select the drop down `:key: Personal access tokens` and click on `Tokens (classic)`\n\n- Select drop down 'Generate new token` at top right and click on `Generate new token (classic)`\n\nAssign the following:\n\ntoken name ----------------------------\u003e git-token\n\nExpiration ----------------------------\u003e 30 days\n\nSelect scopes -------------------------\u003e `Select all scopes except delete permissions`\n\n- Click on `Generate token`\n\n- Copy and save token generated. we'll add it as secret credential in Jenkins.\n\n\n\n**Create Github Secret for Jenkins**\n\nGo to Jenkins `Dashboard` and select `:gear: Manage Jenkins`.\n\nUnder the `Security` section, select `Credentials`\n\nClick on `(global)` domain of jenkins `System` store.\n\nNext, click on blue button `+ Add Credentials` at the top right.\n\nAssign the following:\n\nKind -------------------------\u003e Username with password\n\nScope ------------------------\u003e Global\n\nUsername ---------------------\u003e odennav\n\nID -----------------------\u003e `git-cred`\n\nDescription ------------------\u003e boardgame-git-repo\n\nClick on `Create`.\n\n\n-----\n\n## Trivy Setup\n\nTrivy is an open source security scanner used to find security vulnerabilities of dependencies used in source code project and Iac misconfigurations.\n\nIt can also scan the following:\n\n- Container images, filesystem, virtual machine image, git repository, kubernetes cluster and cloud infrastructure.\n\n**Install using apt package manager**\n\nAdd repository to `/etc/apt/sources.list.d` in `Jenkins` machine.\n```bash\nsudo apt-get install wget apt-transport-https gnupg lsb-release\nwget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add -\necho deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list\nsudo apt-get update\nsudo apt-get install -y trivy\n```\n\nIf you intend to use trivy as container image, mount `docker.sock` as from the host into the Trivy container.\n\nConfirm trivy installed\n```bash\ntrivy --version\n```\n\nTrivy will be used to scan the filesystem and generate the scan report with `html.tpl` file in the `/boardgame-devops-pipeline-project/trivy/` directory.\n\nCopy the template to Jenkins machine.\n```bash\nsudo mkdir ~/trivy\nsudo cd trivy\nsudo vi html.tpl\n```\n\nEnsure Jenkins has permission to use the template\n```bash\nsudo chown -R jenkins:jenkins ~/trivy\n```\n\n-----\n\n## DockerHub Setup\n\n**Create DockerHub Secret for Jenkins**\n\nGo to Jenkins `Dashboard` and select `Manage Jenkins`.\n\nUnder the `Security` section, select `Credentials`\n\nClick on `(global)` domain of jenkins `System` store.\n\nNext, click on blue button `+ Add Credentials` at the top right.\n\nAssign the following:\n\nDomain ------------------------\u003e Global credentials\n\nKind --------------------------\u003e Username with password\n\nUsername ----------------------\u003e odennav\n\nPassword ----------------------\u003e **********\n\nID ----------------------------\u003e `docker-cred`\n\nDescription -------------------\u003e dockerhub-cred \n\n-----\n\n## Setup Mail Notifications \n\nAutomate email notifications of the jenkins pipeline build results by configuring Jenkins to send emails through an SMTP server. We'll use SMTP server from Gmail.\n\nTo give Jenkins permission to access our Google account, we'll need an app password. This is a 16-digit passcode.\n\nGenerate your app password with this [guide](https://support.google.com/mail/answer/185833?hl=en) and name the app `Jenkins`.\n\n\n**Create Gmail Secret Credential for Jenkins**\n\nGo to Jenkins `Dashboard` and select `:gear: Manage Jenkins`.\n\nUnder the `Security` section, select `Credentials`\n\nClick on `(global)` domain of jenkins `System` store.\n\nNext, click on blue button `+ Add Credentials` at the top right.\n\nAssign the following:\n\nKind -------------------------\u003e Username with password\n\nScope ------------------------\u003e Global\n\nUsername ---------------------\u003e `odennav@gmail.com`\n\nPassword ---------------------\u003e `\u003cgmail app-password\u003e`\n\nID -----------------------\u003e `mail-cred`\n\nDescription ------------------\u003e gmail-cred\n\nClick on `Create`.\n\n\n\n**Configure Email Notification in Jenkins**\n\nGo to Jenkins `Dashboard` and select `Manage Jenkins`.\n\nUnder the `System Configuration` section, select `Configure System`.\n\n1. Scroll down and search for **`Extended E-mail Notification`**\n\nClick `Add SonarQube` and assign the following:\n\nSMTP server ---------------------\u003e smtp.gmail.com\n\nSMTP Port -----------------------\u003e 465\n\nClick on drop-down `Advanced`\n\nCredentials ---------------------\u003e `mail-cred`\n\nUse SSL -------------------------\u003e :check_mark:\n\n\n\n2. Scroll down and search for **`E-mail Notification`**\n\nClick `Add SonarQube` and assign the following:\n\nSMTP server ---------------------------------\u003e smtp.gmail.com\n\nSMTP Port ---------------------------\u003e 465\n\nClick on drop-down `Advanced`\n\nUser Name ----------------------\u003e `odennav@gmail.com`\n\nPassword -----------------------\u003e `\u003cgmail app-password\u003e`\n\nUse SSL -------------------------\u003e :check_mark:\n\nSMTP Port -----------------------\u003e 465\n\n\nClick on `Apply` and `Save`.\n\n-----\n\n## Jenkins Pipeline Setup \n\nJenkins Pipeline is a suite of plugins which supports continuous integration and continuous delivery operations in Jenkins.\n\nIt's an automated expression of our process for building, testing and deploying source code from Github right through to our staging environment.\n\n**Setup Pipeline**\n\nImplement procedure below:\n\n - Go to Jenkins main dashboard and click on `New Item`\n\n-  Name pipeline as `BoardGame` and select `Pipeline` as type of project, then click `OK`\n\n- Click on the created job  and select `Discard old builds`\n\n  **`Max # of builds to keep`** --------------- \u003e 2\n\n- Scroll down to the `Pipeline` section in the configuration screen.\n\n- Choose `Pipeline script from SCM` and select type of SCM.\n\n- Enter the URL of the Github repository containing the Jenkinsfile.\n\n- Add credentials of the Github repository which contains the personal access token.\n\n- Choose the branch to build from, typically `/main` \n\n- Specify the path of Jenkinsfile in SCM as `/boardgame-devops-pipeline-project/Jenkinsfile`\n\n- Click on `Save` to save this configuration.\n\n\nTo restart Jenkins and apply configuration changes or updates effectively:\n\n- Navigate to the Jenkins **`Dashboard`** and click on `Manage Jenkins` in the sidebar.\n\n- Select `Reload Configuration from Disk` or `Restart Safely`.\n\nThe pipeline block below defines all the actions required for the secure deployment of the BoardGame site.\n\n```text\npipeline {\n    agent any\n\n    tools{\n        jdk 'jdk17'\n        maven 'maven3'\n    }\n\n    environment{\n        SCANNER_HOME= tool 'sonar-scanner'\n        TEMPLATE_PATH=\"@/home/odennav/trivy/html.tpl\"\n        DATE=$(date +\"%Y-%m-%d_%H-%M\")\n    }\n\n    stages {\n        stage('Git Checkout') {\n            steps {\n                git branch: 'main', credentialsId: 'git-cred', url: 'https://github.com/odennav/boardgame-devops-pipeline-project.git'\n            }\n        }\n       \n        stage('Clear Workspace') {\n            steps {\n                sh \"cleanWs()\"\n            }\n        }\n\n        stage('Maven Clean Phase') {\n            steps {\n                sh \"mvn clean\"\n            }\n        }\n\n        stage('Maven Compile Phase') {\n            steps {\n                sh \"mvn compile\"\n            }\n        }\n \n        stage('Maven Test Phase') {\n            steps {\n                sh \"mvn test\"\n            }\n        }\n\n        stage('Trivy Filesystem  Scan') {\n            steps {\n                sh \"\"\"trivy fs --security-checks vuln,secret,misconfig --format template --template ${TEMPLATE_PATH} --output trivy_fs_report_$DATE.html . \"\"\"\n            }\n        }\n\n        stage('SonarQube Scan') {\n            steps {\n                withSonarQubeEnv('sonar'){\n                    sh '''$SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=BoardGame \\ \n                    -Dsonar.java.binaries=. \\\n                    -Dsonar.projectKey=BoardGame '''\n                }    \n            }\n        }\n  \n        stage('SonarQube Quality Gate') {\n            steps {\n                script {\n                  waitForQualityGate abortPipeline: false, credentialsId: 'sonar-token'   \n                }     \n            }\n        }\n\n\n        stage('Maven Build Phase') {\n            steps {\n                sh \"mvn package\"\n            }\n        }\n\n        stage('Publish Artifacts to Nexus') {\n            steps {\n              withMaven(globalMavenSettingsConfig: 'global-settings', jdk: 'jdk17', maven: 'maven3', mavenSettingsConfig:\",traceability: true) {\n                  sh \"mvn deploy\"\n              }  \n            }\n        }\n\n        stage('Build \u0026 Tag Docker Image') {\n            steps {\n                script {\n                    withDockerRegistry(credentialsId: 'docker-cred', toolName: 'docker') {\n                        sh \"docker build -t odennav/boardgame:v1 ./boardgame\"\n                    }\n                }\n                \n            }\n        }\n\n        stage('Scan Docker Image') {\n            steps {\n                sh \"\"\"trivy image --security-checks vuln,secret,misconfig --format template --template ${TEMPLATE_PATH} --output trivy_image_scan_report_$DATE.html . \"\"\"\n\n            }\n        }\n\n\n        stage('Push Docker Image') {\n            steps {\n                script {\n                    withDockerRegistry(credentialsId: 'docker-cred', toolName: 'docker') {\n                        sh \"docker push  odennav/boardgame:v1 \"\n                    }\n                }\n                \n            }\n        }\n\n\n        stage('Deploy to Kubernetes Cluster') {\n            steps {\n                withKubeConfig(caCertificate:\",clusterName: 'kubernetes',contextName:\",credentialsId:'k8s-cred',namespace:'boardgame',restrictKubeConfigAccess:false,serverUrl:'https://172.31.8.1.146:6443'){\n                    sh \"kubectl apply -f kubernetes-manifests/boardgame-manifests\" \n                {\n                \n            }\n        }\n\n        stage('Verify Kubernetes Deployments') {\n            steps {\n                withKubeConfig(caCertificate:\",clusterName: 'kubernetes',contextName:\",credentialsId:'k8s-cred',namespace:'boardgame',restrictKubeConfigAccess:false,serverUrl:'https://172.31.8.1.146:6443'){\n                    sh \"kubectl get pods\" \n                    sh \"kubectl get svc\" \n                {\n                \n            }\n        }\n\n    }\n    post {\n        always {\n            archiveArtifacts artifacts: \"trivy_report_*.html\", fingerprint: true    \n            publishHTML (target: [\n                allowMissing: false,\n                alwaysLinkToLastBuild: false,\n                keepAll: true,\n                reportDir: '.',\n                reportFiles: 'trivy_report_*.html',\n                reportName: 'Trivy Scan',\n                ])\n\n        script {\n            def jobName = env.JOB_NAME\n            def buildNumber = env.BUILD_NUMBER\n            def pipelineStatus = currentBuild.result ?: 'UNKNOWN'\n            def bannerColor = pipelineStatus.toUpperCase() == 'SUCCESS' ? 'green' : 'red'\n\n            def body = \"\"\"\n                \u003chtml\u003e\n                \u003cbody\u003e\n                \u003cdiv style=\"border: 4px solid ${bannerColor}; padding: 10px;\"\u003e\n                \u003ch2\u003e${jobName} - Build ${buildNumber}\u003c/h2\u003e\n                \u003cdiv style=\"background-color: ${bannerColor}; padding: 10px;\"\u003e\n                \u003ch3 style=\"color: white;\"\u003ePipeline Status: ${pipelineStatus.toUpperCase()}\u003c/h3\u003e\n                \u003c/div\u003e\n                \u003cp\u003eCheck the \u003ca href=\"${BUILD_URL}\"\u003econsole output\u003c/a\u003e.\u003c/p\u003e\n                \u003c/div\u003e\n                \u003c/body\u003e\n                \u003c/html\u003e\n            \"\"\"\n\n            emailext (\n                subject: \"${jobName} - Build ${buildNumber} - ${pipelineStatus.toUpperCase()}\",\n                body: body,\n                to: 'odennav@gmail.com',\n                from: 'jenkins@example.com',\n                replyTo: 'jenkins@example.com',\n                mimeType: 'text/html',\n                attachmentsPattern: 'trivy_image_scan_report_$DATE.html'\n            )\n        }                    \n\n        }\n    }\n}\n```\n\n-----\n\n**Build Pipeline**\n\nSelect drop-down of **`BoardGame`** pipeline created above and trigger a build of pipeline job.\n\n- Click on `Build Now`\n\n- Jenkins will fetch the Jenkinsfile from the Github repository and run the jobs defined.\n\n- View the progress of the pipeline job on the Jenkins dashboard.\n\n- Click on the job to view detailed logs and status updates as each stage of the pipeline is executed.\n\nCheck the console output and logs for more info on any failures.\n\n\n**SAST Reports**\n\nTo view reports generated by SonarQube:\n\n- Browse to `10.33.100.5:9000` and click on **`Projects`** tab.\n\n- Select project we created in pipeline script, `BoardGame`.\n\n- View bugs, vulnerabilities, code smells, duplications and hotspots reviews.\n\nTo view scan reports from Trivy:\n\n- Select pipeline job created\n\n- Scroll down and click on `Trivy Scan` in Jenkins.\n\n- View vulnerabilities found with different severity levels.\n\n\n-----\n\n\n## Setup Prometheus and Grafana for Monitoring and Observability\n\n**Install Prometheus**\n\nDownload [Prometheus](https://prometheus.io/download/)\n```bash\ncd ~\nwget https://github.com/prometheus/prometheus/releases/download/v2.53.0/prometheus-2.53.0.linux-amd64.tar.gz\n```\n\nVerify the SHA256 checksum\n```bash\nsha256sum prometheus-2.53.0.linux-amd64.tar.gz\n```\nExtract prometheus from the tarball file\n```bash\ntar -xzvf prometheus-2.53.0.linux-amd64.tar.gz\n```\n\nMove prometheus binary to bin directory\n```bash\nsudo mv ./prometheus-2.53.0.linux-amd64 /usr/local/bin/\n```\n\nBrowse to prometheus UI on port `9090`.\n\n**Manage Prometheus with Systemd**\n\nWe'll create a system service for Prometheus. This enables us to manage it efficiently.\n\nCreate a service file for `prometheus` service\n```bash\nsudo touch /etc/system/service/prometheus.service\n```\nAdd this config to the `prometheus.service` file\n```bash\n[Unit]\nDescription=Prometheus Monitoring System\nWants=network-online.target\nAfter=network-online.target\n\n[Service]\nUser=prometheus\nGroup=prometheus\nType=simple\nExecStart=/usr/local/bin/prometheus-2.53.0.linux-amd64/prometheus\nRestart=on-failure\n\n[Install]\nWantedBy=multi-user.target\n```\n\nCreate Prometheus user and group\n```bash\nsudo adduser --no-create-home --shell /bin/false prometheus\n```\n\nEnable new ownership for prometheus \n```bash\nsudo chown -R prometheus:prometheus /usr/local/bin/prometheus-2.53.0.linux-amd64\n```\n\nReload system to recognize new service created\n```bash\nsudo systemctl daemon-reload\n```\n\nStart the prometheus service\n```bash\nsudo systemctl start prometheus\n```\n\nEnable the service to start on boot\n```\nsudo systemctl enable prometheus\n```\n\nConfirm the status of prometheus service\n```bash\nsudo systemctl status prometheus\n```\n\n-----\n\n**Install Grafana**\n\nDownload and install the [grafana](https://grafana.com/grafana/download)\n```bash\ncd ~\nsudo apt-get install -y adduser libfontconfig1 musl\nwget https://dl.grafana.com/enterprise/release/grafana-enterprise_11.0.0_amd64.deb\nsudo dpkg -i grafana-enterprise_11.0.0_amd64.deb\n```\n\nStart the grafana-server\n```bash\nsudo /bin/systemctl start grafana-server\n```\n\nBrowse to grafana UI on port `3000`.\n\nLog in with the username as `admin` and the password as `admin`.\n\n-----\n\n**Install Blackbox Exporter**\n\nDownload the blackbox exporter\n```bash\ncd ~\nwget https://github.com/prometheus/blackbox_exporter/releases/download/v0.25.0/blackbox_exporter-0.25.0.linux-amd64.tar.gz\n```\n\nVerify the SHA256 checksum\n```bash\nsha256sum blackbox_exporter-0.25.0.linux-amd64.tar.gz\n```\n\nExtract blackbox exporter from the tarball file\n```bash\ntar -xzvf blackbox_exporter-0.25.0.linux-amd64.tar.gz\n```\n\nMove the blackbox exporter to the bin diectory\n```bash\nsudo mv ./blackbox_exporter-0.25.0.linux-amd64 /usr/local/bin\n```\n\n\n**Manage Blackbox Exporter with Systemd**\n\nWe'll create a system service for Blackbox exporter. This enables us to manage it efficiently.\n\nCreate a service file for `blackbox_exporter` service\n```bash\nsudo touch /etc/system/service/blackbox_exporter.service\n```\nAdd this config to the `blackbox_exporter.service` file\n```bash\n[Unit]\nDescription=Blackbox Exporter\nWants=network-online.target\nAfter=network-online.target\n\n[Service]\nUser=blackbox\nGroup=blackbox\nType=simple\nExecStart=/usr/local/bin/blackbox_exporter-0.25.0.linux-amd64/blackbox_exporter\nRestart=on-failure\n\n[Install]\nWantedBy=multi-user.target\n```\n\nCreate blackbox user and group\n```bash\nsudo adduser --no-create-home --shell /bin/false blackbox\n```\n\nEnable new ownership of blackbox binary\n```bash\nsudo chown -R prometheus:prometheus /usr/local/bin/blackbox_exporter-0.25.0.linux-amd64\n```\n\nReload system to recognize new service created\n```bash\nsudo systemctl daemon-reload\n```\n\nStart the prometheus service\n```bash\nsudo systemctl start blackbox_exporter\n```\n\nEnable the service to start on boot\n```\nsudo systemctl enable blackbox_exporter\n```\n\nConfirm the status of prometheus service\n```bash\nsudo systemctl status blackbox_exporter\n```\n\n\nBrowse to grafana UI on port `9115`.\n\n\n**Setup Blackbox Exporter**\n\nPass Boardgame site as a target for the blackbox exporter. Obtain the external IPv4 address assigned by Metallb\n\n```bash\nkubectl get svc --all-namespaces -o wide\n```\n\nAdd the config below to `prometheus-2.53.0.linux-amd64/prometheus.yml` file.\n\n```yaml\nscrape_configs:\n  - job_name: 'blackbox'\n    metrics_path: /probe\n    params:\n      module: [http_2xx]  # Look for a HTTP 200 response.\n    static_configs:\n      - targets:\n        - http://prometheus.io    # Target to probe with http.\n        - http://\u003cload balanced external-ip\u003e:\u003cport\u003e   # Target to probe with http\n    relabel_configs:\n      - source_labels: [__address__]\n        target_label: __param_target\n      - source_labels: [__param_target]\n        target_label: instance\n      - target_label: __address__\n        replacement: 10.33.100.8:9115  # The blackbox exporter's real hostname and port.\n```\n\nRestart prometheus\n```bash\nsudo systemctl restart prometheus\n```\n\n**Create Grafana Data Source**\n\nOpen a web browser and connect to `10.33.100.8:3000`.\n\nRecall the username is `admin` and the password is `admin`\n\nImplement the following:\n\nClick on the :gear: icon in the menu bar on the left. It will take you to the Configuration page for Grafana.\n\nClick on the `Add data sources` button\n\nClick on `Prometheus` as your choice of open-source time series database.\n\nAssign the following as below:\n\nName ----------------\u003e prometheus\n\nDefault--------------\u003e Click to turn the selector on.\n\nURL -----------------\u003e `http://10.33.100.8:9090/`\n\n\nClick the `Save \u0026 Test` button.\n\nNow Grafana can access the metrics from Prometheus.\n\n\n**Import System Dashboard**\n\nHover over the `+` sign in the menu on the left of your screen. It will expand into a menu when you hover over it.\n\nFrom there, click on Import.\n\nImplement the following:\n\nIn the `Grafana dashboard URL or id` field, enter `7587` and click the `Load` button next to it.\n\nOn the next screen in the `Select a Prometheus data source` box, select `prometheus`. \n\nThen click on the `Import` button.\n\nNow you should see a dashboard displaying scraped Prometheus metrics for the Boardgame website.\n\nEnsure to save the imported dashboard.\n\n-----\n\nEnjoy!\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fodennav%2Fdevsecops-pipeline-boardgame-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fodennav%2Fdevsecops-pipeline-boardgame-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fodennav%2Fdevsecops-pipeline-boardgame-app/lists"}