{"id":21379569,"url":"https://github.com/mfkimbell/end-to-end-devops","last_synced_at":"2025-08-18T09:38:34.820Z","repository":{"id":206703728,"uuid":"717514435","full_name":"mfkimbell/end-to-end-DevOps","owner":"mfkimbell","description":"End-to-End CI/CD Pipeline. Jenkins polls for changes to GitHub source code for a Tomcat server hosted web application and then executes jobs with Ansible scripts. These scripts create new docker images with the changes, upload those images to DockerHub, and then deploy the new image into multiple pods on Kubernetes with a loadbalancer. ","archived":false,"fork":false,"pushed_at":"2023-11-26T21:58:20.000Z","size":136,"stargazers_count":1,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-02T05:46:19.283Z","etag":null,"topics":["ansible","aws","docker","ec2","eks","eksctl","jenkins","kubectl","kubernetes","maven","tomcat"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mfkimbell.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-11T17:38:48.000Z","updated_at":"2023-11-27T12:31:46.000Z","dependencies_parsed_at":"2024-11-22T10:23:09.903Z","dependency_job_id":null,"html_url":"https://github.com/mfkimbell/end-to-end-DevOps","commit_stats":null,"previous_names":["mfkimbell/dev-ops","mfkimbell/end-to-end-devops"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mfkimbell/end-to-end-DevOps","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mfkimbell%2Fend-to-end-DevOps","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mfkimbell%2Fend-to-end-DevOps/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mfkimbell%2Fend-to-end-DevOps/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mfkimbell%2Fend-to-end-DevOps/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mfkimbell","download_url":"https://codeload.github.com/mfkimbell/end-to-end-DevOps/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mfkimbell%2Fend-to-end-DevOps/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270974182,"owners_count":24678246,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","aws","docker","ec2","eks","eksctl","jenkins","kubectl","kubernetes","maven","tomcat"],"created_at":"2024-11-22T10:21:18.360Z","updated_at":"2025-08-18T09:38:34.777Z","avatar_url":"https://github.com/mfkimbell.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"## End-to-End DevOps Pipeline\n\n\u003cimg width=\"1742\" alt=\"workflow devops\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/e2ddc927-202c-4431-9469-b84d6a3855b0\"\u003e\n\n\n### **Tools Used:**\n* `AWS` Hosting of all CI/CD resources and permissions to those resources\n* `EC2` Manage instances that run various servers for the application architecture\n* `Maven` Managing dependencies and compiling Java code\n* `Jenkins` Build and monitor automation of the application deployment process\n* `Ansible` Scripts for uploading new Docker Images and Kubernetes pulling of Images\n* `Tomcat` Manage server for Java application\n* `Docker` Containerize the application and server\n* `Kubernetes` Docker container management and fault tolerance with load balancer\n* `EKS` AWS management of Kubernetes\n* `eksctl` Command line management of EKS clusters on AWS CLI\n* `kubectl` Command line management of Kubernetes clusters\n\n### Purpose\n\nPurpose was to make an end-to-end CICD pipeline for an application stored on Github through AWS. Changes are made to the application on Github. Jenkins polls every minute to see if changes occur, so, in under a minute, Jenkins will execute a job  via Ansible to build and upload a new version of the Docker container onto DockerHub. After this is complete, another Jenkins CD job will use Ansible to tell the Kubernetes Bootstrap EC2 to pull the new image from DockerHub and deploy it on multiple pods (for high availability) and create a Load balancer users can access to use the application. \n\n\nHere are all of the EC2 Instances running:\n\n\u003cimg width=\"725\" alt=\"Screenshot 2023-11-13 at 6 33 45 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/21eddf1b-5cac-4162-8888-8f49a67aa72f\"\u003e\n\nHere are all of the Jenkins Jobs working: \n\n\u003cimg width=\"600\" alt=\"Screenshot 2023-11-13 at 6 42 04 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/5aa1e59c-26cf-499e-b14c-4b2e70d05096\"\u003e\n\n(For the working version of the app, only `RegApp_CI_Job` and `RegApp_CD_Job` are used)\n\n\n\n\n\n\n\n## Documentation from start to end:\n\n## **As I will be deleting all of these AWS resources to prevent charges on my account, I have documented thoroughly this entire process.**\n\n\nSSH into an ec2 instance, and download jenkins and java (we create security groups that allow access from port 8080 for when we need to access our webapps\n\n\"service jenkins start\" to start the jenkins server\n\naccess jenkins with public ip and port\n\n   \n\u003cimg width=\"639\" alt=\"Screenshot 2023-11-11 at 5 52 00 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/22fc8cfa-0573-4cc2-aa7b-2628ce88f948\"\u003e\n\nNext, need to integrate github with jenkins\n\nInstall git on jenkins instance\n\nInstall github plugin on jenkins GUI\n\n**Roadblock: jenkins uses \"master\" not \"main\" by default**\n\n\u003cimg width=\"500\" alt=\"Screenshot 2023-11-11 at 9 56 04 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/8e2e0385-9864-46c2-aa41-cfc0b5c768a6\"\u003e\n\nConfiure git on jenkins GUI\n\n\u003cimg src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/f6cf2e22-05e8-4f6a-8bbd-915752163a42\"  width=\n\"500\"\u003e\n\nJava JDK on Maven from our EC2 server\n\n\u003cimg src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/ea7f7ba8-979c-4ba5-a0d9-da96edfd6b16\" width=\"500\" \u003e\n   \nIntegrate maven with jenkins\n\n\u003cimg src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/0728c5e7-a434-426e-990a-c146e947649a\" width=\"500\" \u003e\n\nHere are the three Jenkins Jobs I've created\n\n![Screenshot 2023-11-11 at 4 49 34 PM](https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/c28fcad5-c0bb-4188-bafd-bc8e78068a27)\n\nHere we can see our build artifacts from the maven webapp job\n\n\u003cimg src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/4360d000-df4a-4217-bb2d-4dad8e6fcb1e\" width=\"500\" \u003e\n\nNow we setup the Tomcat server similarly\n    \nHad to remove default permissions, and update it so we can have admin access. Also created some shortcuts for starting and stoppping the server\n\n \u003cimg width=\"863\" alt=\"Screenshot 2023-11-11 at 6 10 35 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/3582e4a1-4067-4d7e-8f02-3e2c0d62a36b\"\u003e\n\nAnd then we login to the tomcat manager console\n\n\u003cimg width=\"700\" alt=\"Screenshot 2023-11-11 at 6 14 07 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/494d759c-0c28-475e-9130-bb13b6e51889\"\u003e\n\nThen we set up jenkins with some secrets to access the tomcat server\n\n\u003cimg width=\"700\" alt=\"Screenshot 2023-11-11 at 9 52 54 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/f828f35a-db47-4d59-bb97-75080f06b8c6\"\u003e\n\nNext we create a jenkins job \"buildAndDeployServer\" that builds the webapp on the tomcat server. After run, we see \"webapp\" now appears on the tomcat server.\n\n\u003cimg width=\"700\" alt=\"Screenshot 2023-11-11 at 10 13 44 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/d30cc4ce-297b-4591-ba3f-af66ca193214\"\u003e\n\nAnd now we can see our webapp running on Tomcat:\n\n\u003cimg width=\"651\" alt=\"Screenshot 2023-11-11 at 10 31 34 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/c1303646-ac7d-466c-ac6c-3ed26dab81d2\"\u003e\n\nBut I ran this job MANUALLY. I want it to run every time I make changes to my code. So we set a build trigger for code changes in Jenkins. We can poll the software configuration management, which is a cron job that checks to see if the there are changes to the code, and only runs if there are changes. The current configuration I have set up uses a wildcard syntax to check the server every minute:\n\n\u003cimg width=\"700\" alt=\"Screenshot 2023-11-11 at 10 38 07 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/b8a0e8c9-e6cd-4b66-a873-df43c1fd465c\"\u003e\n\nAnd here is a job running automatically after a code change:\n\n\n\n\u003cimg width=\"270\" alt=\"Screenshot 2023-11-11 at 10 52 11 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/309f62a1-c3c6-489c-9881-ca31f949cb04\"\u003e\n\n-\u003e\n\n\u003cimg width=\"375\" alt=\"Screenshot 2023-11-11 at 10 43 09 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/2e3c248e-dbbb-4f25-bf78-48926aa63c06\"\u003e\n\n\n\n\nI want to deploy Tomcat into a docker container\n\nSo I make a new EC2 instance and install docker on it. \n\nWe have the interal port as 8080, and the external port as 8081. So we need to update the security group of the docker-host. \n\n\u003cimg width=\"721\" alt=\"Screenshot 2023-11-12 at 10 20 13 AM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/e748fdae-d813-428a-8bd4-4b0c6d181881\"\u003e\n\nHere we open ports 8081-9000:\n\n\u003cimg width=\"731\" alt=\"Screenshot 2023-11-12 at 10 22 27 AM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/5ad66480-3606-4703-912d-b10294907e99\"\u003e\n\nI write a dockerfile to install java and tomcat on centos and then start the Tomcat server. This was a nightmare to do for some reason. Normally you would just pull a Official Docker Tomcat image, but there was a well-known issue with that apparently:\n\n\u003cimg width=\"7070\" alt=\"Screenshot 2023-11-12 at 11 01 05 AM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/fdc1ea2e-74d6-46ee-bd91-765d2a284354\"\u003e\n\nChanged my docker container to be accessed via password so I can connect to it via jenkins. SSH is best practice, but this was easier.\n\n\u003cimg width=\"7072\" alt=\"Screenshot 2023-11-12 at 11 47 47 AM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/4b489f71-50a5-4908-9ec6-55069cbe6765\"\u003e\n\nNow we need to build a Jenkins job to deploy artifacts on our docker container. Specifically, we are sending the .war webapp file over SSH:\n\n\u003cimg width=\"735\" alt=\"Screenshot 2023-11-12 at 12 11 12 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/a6745334-e784-4b13-b490-59702fe25c85\"\u003e\n\nSpent a couple hours experimenting here since I again ran into some issues:\n\n\u003cimg width=\"563\" alt=\"Screenshot 2023-11-12 at 1 07 29 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/033a636b-7137-4f5d-8751-a7d1f5570824\"\u003e\n\nFinally managed to get the webapp.war file on my docker instance\n\n\u003cimg width=\"599\" alt=\"Screenshot 2023-11-12 at 1 23 42 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/ff8b7661-8f35-484c-b9df-a4d30016a680\"\u003e\n\nNow we create a new folder to copy the artifacts to in the EC2 instance:\n\n\u003cimg width=\"682\" alt=\"Screenshot 2023-11-12 at 1 39 47 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/2a361316-80d4-471d-8fef-fa9d295805a4\"\u003e\n\nWe also update our Jenkins job to reflect that:\n\n\u003cimg width=\"542\" alt=\"Screenshot 2023-11-12 at 1 41 58 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/d175fb11-3b3c-45d0-8f35-1d693650079a\"\u003e\n\nNow that the file is with our dockerfile, we can update it to send the .war file to the tomcat webapp folder. \n\n\u003cimg width=\"633\" alt=\"Screenshot 2023-11-12 at 1 45 20 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/d37b8a0c-1eae-48b2-9064-971351162308\"\u003e\n\nCreate a new container with these updates:\n\n\u003cimg width=\"650\" alt=\"Screenshot 2023-11-12 at 1 48 32 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/8074b5c2-d5b2-4f72-afef-cd14b8fd9959\"\u003e\n\nNow  if we go to the URL, (publicIP):8086/webapp we can see our webapp:\n\n\u003cimg width=\"635\" alt=\"Screenshot 2023-11-12 at 1 49 39 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/25d0d6f5-da32-4882-a4e9-cf7582a14e8b\"\u003e\n\nHowever, I built this container manually, and I'd like jenkins to build it for me. So we set it to check for updates every minute, and remove and rebuild if there are changes to the code:\n\n\u003cimg width=\"538\" alt=\"Screenshot 2023-11-12 at 2 12 29 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/57299653-db93-4d41-8802-d77b180518a3\"\u003e\n\nNow we are going to integrate Ansible as a deployment tool. We set up access from our Ansible EC2 to the Docker EC2 with it's private IP since we are in a VPC, so that's allowed:\n\n\u003cimg width=\"546\" alt=\"Screenshot 2023-11-12 at 3 25 39 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/abfe1103-ae30-4cd4-9d55-943d25b41ef2\"\u003e\n\nWe can see that Ansible was successfully able to acces the private IP:\n\n\u003cimg width=\"874\" alt=\"Screenshot 2023-11-12 at 3 58 41 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/e0a0e0de-bd87-4856-adfa-5174fc1ae1ff\"\u003e\n\nNow we need to make it so Jenkins can integrate with Ansible. We need to make a new Jenkins job that this time excludes the \"exec\" jobs, since we will be handling the deployment on Ansible.\n\n\u003cimg width=\"918\" alt=\"Screenshot 2023-11-12 at 4 27 27 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/cd810e4b-4ccb-4e1b-ad94-8bad043b37e9\"\u003e\n\nWe give our Anisble EC2 it's own dockerfile with the same instructions as before on building the Tomcat server, build that image, and then build that container. \n\n\u003cimg width=\"828\" alt=\"Screenshot 2023-11-12 at 4 48 32 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/a960d07d-1261-4efe-89ab-69369ab90b07\"\u003e\n\nAnd the webapp sucessfully runs on the docker container in the Ansible EC2:\n\n\u003cimg width=\"538\" alt=\"Screenshot 2023-11-12 at 4 50 43 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/78a0d960-1cd2-4d22-a7fe-a596dba842d5\"\u003e\n\nNow we need to create an Ansible Playbook to execute these commands automatically. We need to instruct Ansible to instruct the Docker container to pull a container from DockerHub. First off, we need to change the host IPs on the Ansible server so we can access a playbook file on the Ansible server, altering vi/ansible/hosts. We create groups based on where we want the execution to occur:\n\n\u003cimg width=\"592\" alt=\"Screenshot 2023-11-12 at 5 01 53 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/6b706ac2-9a6b-496c-a5bd-b705e58177fe\"\u003e\n\nThis is my basic playbook. I create a task to create an image and specify that if this playbook is available it can be executed in the docker directory. \n\n\u003cimg width=\"433\" alt=\"Screenshot 2023-11-12 at 5 17 45 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/3677e783-742d-4381-8480-92ff5fd1780e\"\u003e\n\nAfter running the task, we can see the image has been added:\n\n\u003cimg width=\"944\" alt=\"Screenshot 2023-11-12 at 5 20 24 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/10eb9d93-ec0d-4483-8d07-09245422fbc5\"\u003e\n\nNow, we want to upload our image onto DockerHub. So I login to DockerHub on my Ansible EC2, and then manually push the image of interest. \n\n\u003cimg width=\"758\" alt=\"Screenshot 2023-11-12 at 5 35 11 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/051e4e84-5f4c-46e7-ab9b-a474c103b80f\"\u003e\n\nAnd now we see it on DockerHub:\n\n\u003cimg width=\"963\" alt=\"Screenshot 2023-11-12 at 5 35 48 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/36774ef0-1ec5-4f69-934a-100e8157da9e\"\u003e\n\nNow I'd like to incorporate these steps into Ansible Playbook. I create two new tasks on the playbook to tag and upload the image:\n\n\u003cimg width=\"572\" alt=\"Screenshot 2023-11-12 at 5 40 45 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/a5f7b9a2-6ad7-487c-86ee-4a700e02b2b6\"\u003e\n\nNow I alter my jenkins job to execute the Ansible Playbook, as well as adding a \"Poll ACM: * * * * *\" so it's run automatically with code changes:\n\n\u003cimg width=\"545\" alt=\"Screenshot 2023-11-12 at 5 51 07 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/82072d28-0c0e-4d20-86a7-b03209c94393\"\u003e\n\nNext, I create another playbook, deploy_regapp.yml, that graps the container from DockerHub.\n\n\u003cimg width=\"799\" alt=\"Screenshot 2023-11-12 at 6 24 21 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/b7a588c2-a132-4357-ac49-811f33353f69\"\u003e\n\nAnd once again, the webapp is up and running, this time through Ansible Playbook.\n\n\u003cimg width=\"550\" alt=\"Screenshot 2023-11-12 at 6 34 38 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/52ce3440-34d9-42e9-95eb-bcf7becd021e\"\u003e\n\nNext, we need to update our playbook to remove current versions so that the container actually gets updated when there are changes. \n\n\u003cimg width=\"765\" alt=\"Screenshot 2023-11-12 at 6 43 28 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/be6c3646-3e2b-4f63-bfd2-ccdfc9c3531f\"\u003e\n\nAnd we add that playbook to the Jenkins job execs:\n\n\u003cimg width=\"537\" alt=\"Screenshot 2023-11-12 at 6 49 04 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/14fbc7ab-c0c6-4adc-ab61-ef44c6565a43\"\u003e\n\nFinally, our webapp is up and running again, all done automatically through Jenkins and Ansible whenever a change is made.\n\n\u003cimg width=\"480\" alt=\"Screenshot 2023-11-12 at 6 53 08 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/d69a6c73-201a-463b-870e-5f8fb883356e\"\u003e\n\nHere are all of the Jenkins jobs so far, Copy_Artifacts_onto_Ansible is the recent one I was referring to:\n\n\u003cimg width=\"707\" alt=\"Screenshot 2023-11-12 at 6 58 13 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/b1855a5f-0d38-4c45-b129-4c4f8c81076d\"\u003e\n\nNow, I want to incorporate Kubernetes for recovery/high-availability in case something happens to the Docker container. I make a new EC2 instance, and download awscli and eksctl.\n\n\u003cimg width=\"604\" alt=\"Screenshot 2023-11-13 at 11 52 26 AM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/aae210a4-475a-47a3-aaaa-a061caede435\"\u003e\n\n\nThen I make an IAM role \"eksctl\" with EC2, Cloudformation, and IAM Full access, as well as administrator access for now. \n\n\u003cimg width=\"453\" alt=\"Screenshot 2023-11-13 at 11 50 07 AM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/965c7121-b485-4534-be61-10b176cd1ded\"\u003e\n\nI run \"eksctl create cluster\" along with other specifications, and we can see the cluster up and running in Cloudformation:\n\n\u003cimg width=\"613\" alt=\"Screenshot 2023-11-13 at 12 49 45 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/440f741c-1950-4f53-a320-6cb80bd07c1d\"\u003e\n\nI run \"kubectl run webapp --image=http\" to create a webapp pod, and I run \"kubectl create deployment demo-nginx --image=nginx --port=80 --replicas=2\" to create an \"nginx\" deployment of two replica pods. \n\n\u003cimg width=\"453\" alt=\"Screenshot 2023-11-13 at 1 03 08 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/e3026ce5-a741-481c-b698-6800cfc93a4b\"\u003e\n\nThen, I expose the application with a load balancer on port 80:\n\n\u003cimg width=\"914\" alt=\"Screenshot 2023-11-13 at 1 06 11 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/c10fa513-6fb6-474e-b9ea-7b41eb43bea7\"\u003e\n\n\u003cimg width=\"624\" alt=\"Screenshot 2023-11-13 at 1 07 08 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/ac52a1de-ac54-4dda-816d-f94b2741205e\"\u003e\n\nThrough the external address, we can see the Nginx page:\n\n\u003cimg width=\"738\" alt=\"Screenshot 2023-11-13 at 1 08 44 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/0bc80cb7-14a2-4090-ab82-6b368b50aa5a\"\u003e\n\nIt's better practice to use Manifest Files, so we'll implement that now. First we create a yml file with pod creation instructions: \n\n\u003cimg width=\"484\" alt=\"Screenshot 2023-11-13 at 1 32 36 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/c4f39f17-d61a-4597-a579-3368a3aa98dc\"\u003e\n\n-\u003e\n\n\u003cimg width=\"382\" alt=\"Screenshot 2023-11-13 at 1 25 17 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/9d1bd8c0-d5b9-429d-a6ef-626a17c0189c\"\u003e\n\nAs well as for a load balancer:\n\n\u003cimg width=\"429\" alt=\"Screenshot 2023-11-13 at 1 38 11 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/75b7028d-4257-4e56-9136-2c365f0074b2\"\u003e\n\n-\u003e\n\n\u003cimg width=\"808\" alt=\"Screenshot 2023-11-13 at 1 34 38 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/4dbc0241-b02d-4cd8-b634-d9152a36bf0c\"\u003e\n\nNow we want to make a deployment from the Docker container on DockerHub. Previously, we were using a known Docker Container (nginx), however, I want to use MY docker container. \n\n\u003cimg width=\"363\" alt=\"Screenshot 2023-11-13 at 4 27 48 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/3d8f86fd-f024-4fde-8d77-b37b5935882f\"\u003e\n\nAccessing our webapp from our new load balancer (produced by aws):\n\n\u003cimg width=\"534\" alt=\"Screenshot 2023-11-13 at 4 40 00 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/d70a7ef7-37a9-4210-bb6d-2973d71e2e1c\"\u003e\n\nNow, because of the Rolling deployment, even if one pod goes down, a new pod will be deployed to take it's place:\n\n\u003cimg width=\"718\" alt=\"Screenshot 2023-11-13 at 4 41 43 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/74340c23-c300-4c31-b1a0-e8781253a9fa\"\u003e\n\n\nNext, we want Ansible to run these commands instead of running them manually. First we want to create groups so Ansible can connect to the private IPs of the EC2 instances:\n\n\n\u003cimg width=\"227\" alt=\"Screenshot 2023-11-13 at 4 56 51 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/48fa425d-45a2-485b-aed7-94ba1246a020\"\u003e\n\n\nAnd now we can see Ansible is able to access all of these servers (by checking uptime):\n\n\n\u003cimg width=\"867\" alt=\"Screenshot 2023-11-13 at 5 00 51 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/f7759f7b-d13b-4814-ba58-94b06d13df36\"\u003e\n\n\nI make an ansible playbook to execute on kubernetes server:\n\n\u003cimg width=\"521\" alt=\"Screenshot 2023-11-13 at 5 20 06 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/11def968-fc26-4e34-8e3c-2e75533a2890\"\u003e\n\nWe also need a playbook for the service/loadbalancer: \n\n\u003cimg width=\"466\" alt=\"Screenshot 2023-11-13 at 5 21 50 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/2baa9964-3660-4002-9b14-7b976306e7aa\"\u003e\n\nNow that it's working, I want to execute these playbooks from Jenkins like before:\n\n\u003cimg width=\"533\" alt=\"Screenshot 2023-11-13 at 5 30 00 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/098b88ce-efab-4f14-9701-346711471f7b\"\u003e\n\n\u003cimg width=\"704\" alt=\"Screenshot 2023-11-13 at 5 32 41 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/743f6f0d-11d0-426a-b8c4-3f67788f7207\"\u003e\n\nWe can see before I had no pods up and running, and after running the job they are up and running again:\n\n\u003cimg width=\"712\" alt=\"Screenshot 2023-11-13 at 5 33 28 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/5f9ff6ab-4ea1-4b2e-84d2-e2edd29a5c11\"\u003e\n\nIn hindsight, these should be merged, so now we only need to run kube_deploy.yml:\n\n\u003cimg width=\"497\" alt=\"Screenshot 2023-11-13 at 5 35 36 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/b93805d7-1f68-4761-8294-117fcd888cf8\"\u003e\n\n\u003cimg width=\"531\" alt=\"Screenshot 2023-11-13 at 5 36 16 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/1e1421cd-740c-4bbe-9d21-eb7d064a33f8\"\u003e\n\nNow, on Jenkins, we need to update our (previously) deployment job and change it to instead use ansible to create and upload a new image onto DockerHub. The highlighted section is what will be deleted that was previously from the \"Copy_Artifacts_Onto_Ansible\" job:\n\n\u003cimg width=\"532\" alt=\"Screenshot 2023-11-13 at 5 49 10 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/c57953a1-6643-4c76-bb05-e3744500bceb\"\u003e\n\nWe rename our \"Deploy on Kubernetes\" to \"RegApp_CD_Job\" and we call our altered \"Copy_Artifacts_Onto_Ansible\" job \"RegApp_CI_Job\" for clairty:\n\n\u003cimg width=\"694\" alt=\"Screenshot 2023-11-13 at 5 53 05 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/566269c1-517b-49a5-b3b4-e8fee2e23e36\"\u003e\n\nI ran these jobs manually, I would like them to run automatically and sequentially:\n\n\u003cimg width=\"405\" alt=\"Screenshot 2023-11-13 at 5 59 29 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/ff535bbd-9cdc-46a6-97e2-597725651221\"\u003e\n\n\u003cimg width=\"647\" alt=\"Screenshot 2023-11-13 at 6 00 34 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/2f0cf3a4-ec3f-4b4a-95d4-020df150d91b\"\u003e\n\nAdditionally, I update my kube_deploy.yml to create new pods if the DockerHub Image has changed:\n\n\u003cimg width=\"694\" alt=\"Screenshot 2023-11-13 at 6 07 20 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/e109c8b8-a9b1-4135-8902-dceca5d03bcc\"\u003e\n\nFinally, we can test the full pipeline. We access the webapp via our loadbalancer on AWS:\n\n\u003cimg width=\"637\" alt=\"Screenshot 2023-11-13 at 6 19 39 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/831a2461-471b-4c42-a271-466846f049fa\"\u003e\n\nHere is the webapp loaded. I'm going to make a change in VS code and commit a change to the title.\n\n\u003cimg width=\"554\" alt=\"Screenshot 2023-11-13 at 6 20 48 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/ced4667f-0a83-4418-929f-3f499503d585\"\u003e\n\n\u003cimg width=\"743\" alt=\"Screenshot 2023-11-13 at 6 22 38 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/28838264-326b-484f-957c-c3850c45903e\"\u003e\n\nWe can see the DockerHub Image get updated:\n\n\u003cimg width=\"939\" alt=\"Screenshot 2023-11-13 at 6 24 41 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/2c063fd5-8678-4827-9f02-a9c87e13dae4\"\u003e\n\nWe see the CI and CD jobs get executed:\n\n\u003cimg width=\"693\" alt=\"Screenshot 2023-11-13 at 6 28 43 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/8dbb6c98-0cc2-4749-8aa5-dcf1777a508c\"\u003e\n\nAnd we can see the new deployment of the pods:\n\n\u003cimg width=\"732\" alt=\"Screenshot 2023-11-13 at 6 41 30 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/c1ad36d7-9678-4062-aae8-8285ba7e2264\"\u003e\n\nFinally, we can see the updated WebApp:\n\n\u003cimg width=\"539\" alt=\"Screenshot 2023-11-13 at 6 30 21 PM\" src=\"https://github.com/mfkimbell/end-to-end-DevOps/assets/107063397/4589564f-046a-4acf-a53b-5b92feee0be7\"\u003e\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmfkimbell%2Fend-to-end-devops","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmfkimbell%2Fend-to-end-devops","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmfkimbell%2Fend-to-end-devops/lists"}