{"id":17442424,"url":"https://github.com/kolman-freecss/kf_envorchestatortool","last_synced_at":"2026-02-08T16:04:44.026Z","repository":{"id":258021294,"uuid":"868073649","full_name":"Kolman-Freecss/KF_EnvOrchestatorTool","owner":"Kolman-Freecss","description":"Orchestator tool that automatically provides a pre-configured environment either on-premises or in the cloud","archived":false,"fork":false,"pushed_at":"2024-10-16T17:42:20.000Z","size":69,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-10-17T10:14:02.746Z","etag":null,"topics":["aws","docker","groovy","jenkins","python","shellscript","terraform"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kolman-Freecss.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-CC-BY-NC-ND","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-05T12:09:15.000Z","updated_at":"2024-10-16T17:47:00.000Z","dependencies_parsed_at":"2024-10-17T10:14:10.861Z","dependency_job_id":"1fa2adbd-2e37-4884-943b-c4bca94ad7e0","html_url":"https://github.com/Kolman-Freecss/KF_EnvOrchestatorTool","commit_stats":null,"previous_names":["kolman-freecss/kf_envorchestatortool"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kolman-Freecss%2FKF_EnvOrchestatorTool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kolman-Freecss%2FKF_EnvOrchestatorTool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kolman-Freecss%2FKF_EnvOrchestatorTool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kolman-Freecss%2FKF_EnvOrchestatorTool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kolman-Freecss","download_url":"https://codeload.github.com/Kolman-Freecss/KF_EnvOrchestatorTool/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":220097838,"owners_count":16594211,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","docker","groovy","jenkins","python","shellscript","terraform"],"created_at":"2024-10-17T16:06:01.386Z","updated_at":"2026-02-08T16:04:43.992Z","avatar_url":"https://github.com/Kolman-Freecss.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# KF_EnvOrchestatorTool\n\nThis tool is used to serve an automated environment in local or cloud platform to easily develop applications monolith or any architecture, it doesn't matter.\n\n# Index\n\n- [Systems](#systems)\n- [Getting Started](#getting-started)\n- [Local installation](#local-installation)\n    - [Configure Jenkins](#configure-jenkins)\n    - [Configure Environment Variables to execute main.py](#configure-environment-variables-to-execute-mainpy)\n    - [Configure SSH](#configure-ssh)\n- [AWS Configuration](#aws-configuration)\n    - [Trigger Terraform pipeline](#trigger-terraform-pipeline)\n    - [Connect to EC2 instance](#connect-to-ec2-instance)\n- [Configuration](#configuration)\n- [Dependencies](#dependencies)\n- [Tech stacks CI/CD](#tech-stacks-ci/cd)\n- [Tips](#tips)\n- [Troubleshoting](#troubleshoting)\n    - [Jenkins](#jenkins)\n    - [Scripts](#scripts)\n    - [AWS](#aws)\n    - [Settings](#settings)\n\n# Systems\n\n- Jenkins set up with docker-compose\n- Automate Jenkins implantation on AWS with Terraform\n- Automate Jenkins jobs with Python\n- Execute local Jenkins with preconfigured plugins with a custom image on Docker Hub with Docker Compose.\n- Github Actions to CI/CD with Jenkins.\n- Second service to initialize Jenkins with Python and Shellscript.\n\n# Getting Started\n\n## Local installation\n\n### Configure Jenkins\n\n1. Execute `docker-compsoe.yml` from `.docker/local` folder.\n2. Go to `localhost:8080` and follow the instructions to configure Jenkins. (Create an initialAdminPassword)\n3. Install Git plugin for Jenkins. (This is necessary to trigger pipelines with SCM option enabled)\n\nConfig Jenkins on your local machine:\n\n- Initial password located at `/var/jenkins_home/secrets/initialAdminPassword`\n\n### Configure Environment Variables to execute main.py\n\n1. Create a `local.env` file at `.env` folder.\n\nAdd the following variables:\n\n```\nJENKINS_URL=\u003cYOUR_JENKINS_URL\u003e\nJENKINS_USER=\u003cYOUR_JENKINS_USER\u003e\nJENKINS_PASS=\u003cYOUR_JENKINS_PASSWORD\u003e\nACCESS_TOKEN=\u003cYOUR_GITHUB_ACCESS_TOKEN\u003e\n```\n\nNote: ACCESS_TOKEN is necessary to pull the repositories configured in the pipelines from Github.\n\n### Configure SSH\n\n- Install OpenSSH Server on your local machine.\n- (Optional: main.py automatically will start the OpenSSH Server if its installed) Start the service.\n\n```bash\n# Windows\nStart-Service sshd\n```\n\n- (Optional) Try to connect from container to your local machine with SSH.\n\n```bash\ndocker exec -it jenkins-git bash\nssh -i /var/jenkins_home/.ssh/id_rsa admin@host.docker.internal -vvv\n```\n\n## AWS Configuration\n\nImplantation of Jenkins automated with Terraform on AWS.\n\nRequirements:\n\n- Create your AWS account.\n- Create your Access Key in the Security Credentials section.\n- Take an AMI valid for your region.\n- Configure SSH key pair in your AWS account for EC2 instances.\n- Configure VPC.\n- Configure Subnet.\n\n1. Configure AWS CLI with your credentials:\n\n```bash\naws configure\n\n# AWS Access Key ID [None]: YOUR_ACCESS_KEY\n# AWS Secret Access Key [None]: YOUR_SECRET_ACCESS_KEY\n```\n\n2. Go to AMI Catalog and take an AMI ID for your region.\n\nPut your AMI ID in `main.tf` file.\n\n3. Configure your SSH key pair in `main.tf` file.\n\n```bash\naws ec2 create-key-pair --key-name my-ssh-key --query 'KeyMaterial' --output text \u003e my-ssh-key.pem\n```\n\n### Trigger Terraform pipeline\n\nProject has different .tf files decoupled by behaviour. Terraform will treat all files as an unique project.\n\n1. Init Terraform:\n\n```bash\nterraform init\n```\n\n2. Plan Terraform:\n\n```bash\nterraform plan\n```\n\n3. Apply Terraform:\n\n```bash\nterraform apply\n```\n\n4. Destroy Terraform:\n\n```bash\nterraform destroy\n```\n\n### Connect to EC2 instance\n\nHere we've different ways to connect to EC2 instance:\n\n1. Using SSH command:\n\n```bash\n# Create your SSH key pair previously in the EC2 AWS section.\nssh -i my-ssh-key.pem ec2-user@YOUR_EC2_PUBLIC_IP\n```\n\n# Configuration\n\nIn this project, a Jenkins docker image was built and pushed to Docker Hub to store a basic archetypal Jenkins config\nwith Git, Docker and Pipeline plugins.\n\nYou can pulled it from Docker Hub with:\n\n```bash\ndocker pull kolmanfreecss/jenkins-git\n```\n\n(Process to build the image and push it to Docker Hub)\n\n1. Commit the current status container\n\n```bash\ndocker commit YOUR_CONTAINER_ID kolmanfreecss/jenkins-git\n```\n\n2. Login to Docker Hub\n\n```bash\ndocker login\n```\n\n3. Push the image to Docker Hub\n\n```bash\ndocker push kolmanfreecss/jenkins-git\n```\n\n# Dependencies\n\n- Jenkins API\n    - Plugins:\n        - Git (Check Configuration section)\n        - Pipeline (Check Configuration section)\n        - Docker (Check Configuration section)\n\n# Tech stacks CI/CD\n\n- Jenkins\n- Docker \u0026 Docker Compose\n- AWS\n- Python\n- Shellscript\n- Terraform\n\n# Tips\n\n- Check Event Viewer on Windows to see if SSH Server is running properly.\n  - `Applications and Services Logs \u003e OpenSSH \u003e Operational`\n- Remember that Jenkins needs SSH private key and the local machine needs its public key to validate the connection stored in\n  the `authorized_keys` file.\n\n# Troubleshoting\n\n## Jenkins\n- Script to install Jenkins not working properly.\n    - Alternative Solution: Connect through SSH to the EC2 instance and install Jenkins\n      manually. (https://mirrors.jenkins.io/redhat-stable/)\n        - After that connect to the IPv4 Public EC2 instance with HTTP protocol and port 8080.\n            - Example: http://YOUR_EC2_PUBLIC_IP:8080\n- Check EC2 system log from AWS section to see if Jenkins is running properly or installed.\n- BIG Problems installing plugins https://community.jenkins.io/t/issue-while-upgrading-plugins-on-latest-jenkins/9846\n    - It seems that halifax has blocked the ISP, so we need to install the plugins manually or use another ISP in order\n      to install them.\n        - https://community.jenkins.io/t/installing-suggested-plugins-in-jenkins-fails-due-to-connection-timed-out/12564\n    - Another solution is to use a VPN to change the IP address and try to install the plugins again.\n    - Another\n      solution: https://stackoverflow.com/questions/77096022/jenkins-cli-to-install-jenkins-plugins-error-unknownhostexception\n        - Manual installation of plugins. (https://www.jenkins.io/doc/book/managing/plugins/#advanced-installatio)\n            - To install them to have Git for example you will need to install before Git plugin all its dependencies.\n              Follow this order:\n                1. https://plugins.jenkins.io/instance-identity/releases/\n                2. https://plugins.jenkins.io/mailer/releases/\n                3. https://plugins.jenkins.io/credentials/releases/\n                4. https://plugins.jenkins.io/plain-credentials/releases/\n                5. https://plugins.jenkins.io/variant/releases/\n                6. https://plugins.jenkins.io/ssh-credentials/releases/\n                7. https://plugins.jenkins.io/credentials-binding/releases/\n                8. https://plugins.jenkins.io/git-client/releases/\n- It takes its time to start even if the instance is running. Be patient. :)\n    - Check logs with\n        - ```bash\n      aws ec2 get-console-output --instance-id YOUR_INSTANCE_ID --output text\n      ```\n      \n## Scripts\n\n- Use `dos2unix` to convert the scripts to Unix format.\n    - ```bash\n      dos2unix YOUR_SCRIPT.helpers\n      ```\n- Create SSH credentials on Jenkins through Python with Jenkins API. Problem with the XML tag using incorrect format for the implementation\n  - Solution: Use `com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey`\n      \n## AWS\n- Check SSH key permissions to connect to EC2 instance.\n    - `chmod 400 my-ssh-key.pem`\n    - Remove permissions to other group users or another users because AWS won't let you connect to the EC2 instance if\n      the permissions are too permissive.\n\n## Settings\n- Install SSH Server on local machine.\n    - If you have a Windows Server you can follow Microsoft official documentation to install OpenSSH Server. Or check https://github.com/PowerShell/Win32-OpenSSH/releases\n    - Also you could use WSL to install OpenSSH Server.\n      - ```bash\n        sudo apt-get install openssh-server\n        ```\n    - Configure permissions to the id_rsa file to not be too permissive.\n        - ```bash\n          chmod 600 /var/jenkins_home/.ssh/id_rsa\n          ```\n    - Create an authorized_keys file in the .ssh folder with the public key of the local machine.\n        - ```bash\n          cat /var/jenkins_home/.ssh/id_rsa.pub \u003e\u003e /var/jenkins_home/.ssh/authorized_keys\n          ```\n    - Create an sshd_config file in the .ssh folder with the following IMPORTANT configurations UNCCOMMENTED.:\n        - ```bash\n          Port 22\n          AuthorizedKeysFile /var/jenkins_home/.ssh/authorized_keys\n          PubkeyAuthentication yes\n          PasswordAuthentication no\n          ```\n\n---\n\nShield: [![CC-BY-NC-ND 4.0][CC-BY-NC-ND-shield]][CC-BY-NC-ND]\n\nThis work is licensed under\na [Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.][CC-BY-NC-ND]\n\n[![CC-BY-NC-ND 4.0][CC-BY-NC-ND-image]][CC-BY-NC-ND]\n\n[CC-BY-NC-ND-shield]: https://img.shields.io/badge/License-CC--BY--NC--ND--4.0-lightgrey\n\n[CC-BY-NC-ND]: http://creativecommons.org/licenses/by-nc-nd/4.0/\n\n[CC-BY-NC-ND-image]: https://i.creativecommons.org/l/by-nc-nd/4.0/88x31.png\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkolman-freecss%2Fkf_envorchestatortool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkolman-freecss%2Fkf_envorchestatortool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkolman-freecss%2Fkf_envorchestatortool/lists"}