Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/kolman-freecss/devops-jenkins-shellscripting-aws-python
AWS | Jenkins | Python | Docker | Shellscripting | Terraform | Github Actions
https://github.com/kolman-freecss/devops-jenkins-shellscripting-aws-python
Last synced: 1 day ago
JSON representation
AWS | Jenkins | Python | Docker | Shellscripting | Terraform | Github Actions
- Host: GitHub
- URL: https://github.com/kolman-freecss/devops-jenkins-shellscripting-aws-python
- Owner: Kolman-Freecss
- Created: 2024-09-16T18:17:42.000Z (3 months ago)
- Default Branch: master
- Last Pushed: 2024-10-02T19:39:51.000Z (3 months ago)
- Last Synced: 2024-12-25T13:15:57.868Z (1 day ago)
- Language: Python
- Homepage:
- Size: 42 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# DevOps with focus Jenkins
This project is a repository to show the DevOps process with top tech stack.
- Jenkins set up with docker-compose
- Automatization Jenkins jobs with Python.
- Integration Jenkins with AWS
- Terraform to create infrastructure on AWS with a Jenkins implementation.
- Github Actions to CI/CD with Jenkins.
# Index
- [Systems](#systems)
- [Getting Started](#getting-started)
- [Local installation](#local-installation)
- [Configure Jenkins](#configure-jenkins)
- [Configure Environment Variables to execute main.py](#configure-environment-variables-to-execute-mainpy)
- [AWS Configuration](#aws-configuration)
- [Trigger Terraform pipeline](#trigger-terraform-pipeline)
- [Connect to EC2 instance](#connect-to-ec2-instance)
- [Configuration](#configuration)
- [Dependencies](#dependencies)
- [Tech stacks CI/CD](#tech-stacks-ci/cd)
# Systems
- Automate Jenkins implantation on AWS with Terraform
- Automate Jenkins jobs with Python
- Execute local Jenkins with preconfigured plugins with a custom image on Docker Hub with Docker Compose.
# Getting Started
## Local installation
### Configure Jenkins
Config Jenkins on your local machine:
- Initial password located at `/var/jenkins_home/secrets/initialAdminPassword`
1. Execute `docker-compsoe.yml` from `.docker/local` folder.
2. Go to `localhost:8080` and follow the instructions to configure Jenkins. (Create an initialAdminPassword)
3. Install Git plugin for Jenkins. (This is necessary to trigger pipelines with SCM option enabled)
### Configure Environment Variables to execute main.py
1. Create a `local.env` file at `.env` folder.
Add the following variables:
```
JENKINS_URL=
JENKINS_USER=
JENKINS_PASS=
ACCESS_TOKEN=
```
## AWS Configuration
Implantation of Jenkins automated with Terraform on AWS.
Requirements:
- Create your AWS account.
- Create your Access Key in the Security Credentials section.
- Take an AMI valid for your region.
- Configure SSH key pair in your AWS account for EC2 instances.
- Configure VPC.
- Configure Subnet.
1. Configure AWS CLI with your credentials:
```bash
aws configure
# AWS Access Key ID [None]: YOUR_ACCESS_KEY
# AWS Secret Access Key [None]: YOUR_SECRET_ACCESS_KEY
```
2. Go to AMI Catalog and take an AMI ID for your region.
Put your AMI ID in `main.tf` file.
3. Configure your SSH key pair in `main.tf` file.
```bash
aws ec2 create-key-pair --key-name my-ssh-key --query 'KeyMaterial' --output text > my-ssh-key.pem
```
### Trigger Terraform pipeline
Project has different .tf files decoupled by behaviour. Terraform will treat all files as an unique project.
1. Init Terraform:
```bash
terraform init
```
2. Plan Terraform:
```bash
terraform plan
```
3. Apply Terraform:
```bash
terraform apply
```
4. Destroy Terraform:
```bash
terraform destroy
```
### Connect to EC2 instance
Here we've different ways to connect to EC2 instance:
1. Using SSH command:
```bash
# Create your SSH key pair previously in the EC2 AWS section.
ssh -i my-ssh-key.pem ec2-user@YOUR_EC2_PUBLIC_IP
```
# Configuration
In this project, a Jenkins docker image was built and pushed to Docker Hub to store a basic archetypal Jenkins config
with Git, Docker and Pipeline plugins.
You can pulled it from Docker Hub with:
```bash
docker pull kolmanfreecss/jenkins-git
```
(Process to build the image and push it to Docker Hub)
1. Commit the current status container
```bash
docker commit YOUR_CONTAINER_ID kolmanfreecss/jenkins-git
```
2. Login to Docker Hub
```bash
docker login
```
3. Push the image to Docker Hub
```bash
docker push kolmanfreecss/jenkins-git
```
# Dependencies
- Jenkins API
- Plugins:
- Git (Check Configuration section)
- Pipeline (Check Configuration section)
- Docker (Check Configuration section)
# Tech stacks CI/CD
- Jenkins
- Docker & Docker Compose
- AWS
# Troubleshoting
- Script to install Jenkins not working properly.
- Alternative Solution: Connect through SSH to the EC2 instance and install Jenkins
manually. (https://mirrors.jenkins.io/redhat-stable/)
- After that connect to the IPv4 Public EC2 instance with HTTP protocol and port 8080.
- Example: http://YOUR_EC2_PUBLIC_IP:8080
- Check SSH key permissions to connect to EC2 instance.
- `chmod 400 my-ssh-key.pem`
- Remove permissions to other group users or another users because AWS won't let you connect to the EC2 instance if
the permissions are too permissive.
- Check EC2 system log from AWS section to see if Jenkins is running properly or installed.
- BIG Problems installing plugins https://community.jenkins.io/t/issue-while-upgrading-plugins-on-latest-jenkins/9846
- It seems that halifax has blocked the ISP, so we need to install the plugins manually or use another ISP in order
to install them.
- https://community.jenkins.io/t/installing-suggested-plugins-in-jenkins-fails-due-to-connection-timed-out/12564
- Another solution is to use a VPN to change the IP address and try to install the plugins again.
- Another
solution: https://stackoverflow.com/questions/77096022/jenkins-cli-to-install-jenkins-plugins-error-unknownhostexception
- Manual installation of plugins. (https://www.jenkins.io/doc/book/managing/plugins/#advanced-installatio)
- To install them to have Git for example you will need to install before Git plugin all its dependencies.
Follow this order:
1. https://plugins.jenkins.io/instance-identity/releases/
2. https://plugins.jenkins.io/mailer/releases/
3. https://plugins.jenkins.io/credentials/releases/
4. https://plugins.jenkins.io/plain-credentials/releases/
5. https://plugins.jenkins.io/variant/releases/
6. https://plugins.jenkins.io/ssh-credentials/releases/
7. https://plugins.jenkins.io/credentials-binding/releases/
8. https://plugins.jenkins.io/git-client/releases/
- It takes its time to start even if the instance is running. Be patient. :)
- Check logs with
- ```bash
aws ec2 get-console-output --instance-id YOUR_INSTANCE_ID --output text
```
---
Shield: [![CC-BY-NC-ND 4.0][CC-BY-NC-ND-shield]][CC-BY-NC-ND]
This work is licensed under
a [Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.][CC-BY-NC-ND]
[![CC-BY-NC-ND 4.0][CC-BY-NC-ND-image]][CC-BY-NC-ND]
[CC-BY-NC-ND-shield]: https://img.shields.io/badge/License-CC--BY--NC--ND--4.0-lightgrey
[CC-BY-NC-ND]: http://creativecommons.org/licenses/by-nc-nd/4.0/
[CC-BY-NC-ND-image]: https://i.creativecommons.org/l/by-nc-nd/4.0/88x31.png