Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/heyvaldemar/amazon-lightsail-instance-pipeline-terraform

Amazon Lightsail Instance Pipeline with Terraform
https://github.com/heyvaldemar/amazon-lightsail-instance-pipeline-terraform

aws blueprint devops dynamodb ec2 github-action gitlab gitlab-ci hosting key-management-service lightsail linux pipeline pre-commit s3 security server static-ip-address terraform terraform-state-backend

Last synced: 28 days ago
JSON representation

Amazon Lightsail Instance Pipeline with Terraform

Awesome Lists containing this project

README 

        
# Amazon Lightsail Instance Pipeline with Terraform



You can easily deploy an app: WordPress, LAMP, Node.js, Joomla, Magento, MEAN, Drupal, GitLab CE, Redmine, Nginx, Ghost, Django, PrestaShop, Plesk Hosting Stack, cPanel & WHM for AlmaLinux.



Or you can deploy a clean operating system: Amazon Linux, Ubuntu, Debian, FreeBSD, openSUSE, CentOS.



The Terraform script performs the following operations to set up an Amazon Lightsail instance and its related resources:



1. **Creating an AWS Lightsail instance**: The first resource block creates an Amazon Lightsail instance with defined parameters such as instance name, availability zone, key pair name, blueprint ID (specifying the OS of the instance), and bundle ID (specifying the plan of the instance). It also sets up an automatic snapshot addon that will take daily backups of the instance at a specified time. Lastly, it adds a tag of "Environment" to the instance.



2. **Public ports definition on the AWS Lightsail instance**: The second resource block specifies the public ports of the Lightsail instance created in step 1. This configuration opens TCP ports 22 (standard SSH port), 80 (standard HTTP port), and 443 (standard HTTPS port), allowing incoming connections on these ports. By opening port 22, you're also enabling secure shell (SSH) access to the instance, which is essential for administrative tasks.



3. **Creating a key pair for AWS Lightsail instances**: The third resource block creates a key pair in AWS Lightsail. Key pairs are used to log into Lightsail instances securely, and the name of the key pair is supplied via a variable.



4. **Static IP address creation in AWS Lightsail**: The fourth resource block creates a static IP address in AWS Lightsail. Static IPs are used to ensure that the IP address associated with your instance does not change if you stop and restart your instance.



5. **Static IP address attachment to a Lightsail instance**: The fifth resource block attaches the static IP address created in step 4 to the Lightsail instance created in step 1.



# Requirements



Install AWS CLI by following the [guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html).



Configure AWS CLI by following the [guide](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html).



Install Terraform by following the [guide](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli).



Install pre-commit by following the [guide](https://pre-commit.com/#install)



Install tflint by following the [guide](https://github.com/terraform-linters/tflint)



Install tfsec by following the [guide](https://github.com/aquasecurity/tfsec)



Install tfupdate by following the [guide](https://github.com/minamijoyo/tfupdate)



# Blueprint



When creating an instance in AWS Lightsail, two crucial parameters need to be specified: `blueprint_id` and `bundle_id` in the `00-variables.tf`.



The `blueprint_id` parameter determines the operating system or application that will be installed on your instance. Essentially, it's the blueprint for your instance. For example, if you want to set up an instance running Ubuntu, the `blueprint_id` would correspond to that. You can retrieve a list of all available blueprint IDs using the following command:



`aws lightsail get-blueprints`



On the other hand, the `bundle_id` parameter specifies the plan for your Lightsail instance, which includes aspects such as RAM, CPU, storage, and data transfer allowance. It's akin to the instance type in other AWS services. To obtain a list of all available bundle IDs, use the following command:



`aws lightsail get-bundles`



In essence, the `blueprint_id` and `bundle_id` parameters together define the software and hardware configuration of your Lightsail instance.



# Pre-commit Hooks



`.pre-commit-config.yaml` is useful for identifying simple issues before submission to code review. Pointing these issues out before code review, allows a code reviewer to focus on the architecture of a change while not wasting time with trivial style nitpicks. Make sure you have all tools from the requirements section installed for pre-commit hooks to work.



# Manual Installation



Make sure you have all tools from the requirements section installed.



You may change variables in the `00-variables.tf` to meet your requirements.



Initialize a working directory containing Terraform configuration files using the command:



`terraform init`



Run the pre-commit hooks to check for formatting and validation issues:



`pre-commit run --all-files`



Review the changes that Terraform plans to make to your infrastructure using the command:



`terraform plan`



Deploy using the command:



`terraform apply -auto-approve`



# SSH



Once you've run `terraform apply` and the resources are successfully created, a private key file will be generated in your project root directory (where your Terraform files are located). This key can be used to securely connect to the created Amazon Lightsail instance via SSH.



Here's an example of how to use the key to connect via SSH (replace myuser with your username and myinstance with your instance's public IP address or hostname):



`ssh -i key-pair-wordpress-1.pem bitnami@instance-static-ip`



# Backend for Terraform State



The `backend` block in the `01-providers.tf` must remain commented until the bucket and the DynamoDB table are created.



After all your resources will be created, you will need to replace empty values for `region` and `bucket` in the `backend` block of the `01-providers.tf` since variables are not allowed in this block.



For `region` you need to specify the region where the S3 bucket and DynamoDB table are located. You need to use the same value that you have in the `00-variables.tf` for the `region` variable.



For `bucket` you will get its values in the output after the first run of `terraform apply -auto-approve`.



After your values are set, you can then uncomment the `backend` block and run again `terraform init` and then `terraform apply -auto-approve`.



In this way, the `terraform.tfstate` file will be stored in an S3 bucket and DynamoDB will be used for state locking and consistency checking.



# GitHub Actions



`.github` is useful if you are planning to run a pipeline on GitHub and implement the GitOps approach.



Remove the `.example` part from the name of the files in `.github/workflow` for the GitHub Actions pipeline to work.



Note, that you will need to add variables such as AWS_ACCESS_KEY_ID, AWS_DEFAULT_REGION, and AWS_SECRET_ACCESS_KEY in your GitHub projects CI/CD settings section to run your pipeline.



Therefore, you will need to create a service user in advance, using AWS Identity and Access Management (IAM) to get values for these variables and assign an access policy to the user to be able to operate with your resources.



You can delete `.github` if you are not planning to use the GitHub pipeline.



1. **Terraform Unit Tests**



This workflow executes a series of unit tests on the infrastructure code and is triggered by each commit. It begins by running [terraform fmt]( https://www.terraform.io/cli/commands/fmt) to ensure proper code formatting and adherence to terraform best practices. Subsequently, it performs [terraform validate](https://www.terraform.io/cli/commands/validate) to check for syntactical correctness and internal consistency of the code.



To further enhance the code quality and security, two additional tools, tfsec and tflint, are utilized:



tfsec: This step checks the code for potential security issues using tfsec, an open-source security scanner for Terraform. It helps identify any security vulnerabilities or misconfigurations in the infrastructure code.



tflint: This step employs tflint, a Terraform linting tool, to perform additional static code analysis and linting on the Terraform code. It helps detect potential issues and ensures adherence to best practices and coding standards specific to Terraform.



2. **Terraform Plan / Apply**



This workflow runs on every pull request and on each commit to the main branch. The plan stage of the workflow is used to understand the impact of the IaC changes on the environment by running [terraform plan](https://www.terraform.io/cli/commands/plan). This report is then attached to the PR for easy review. The apply stage runs after the plan when the workflow is triggered by a push to the main branch. This stage will take the plan document and [apply](https://www.terraform.io/cli/commands/apply) the changes after a manual review has signed off if there are any pending changes to the environment.



3. **Terraform Drift Detection**



This workflow runs on a periodic basis to scan your environment for any configuration drift or changes made outside of terraform. If any drift is detected, a GitHub Issue is raised to alert the maintainers of the project.



If you have paid version of GitHub and you wish to have the approval process implemented, please refer to the provided [guide](https://docs.github.com/actions/deployment/targeting-different-environments/using-environments-for-deployment#creating-an-environment) to create an environment called production and uncomment this part in the `02-terraform-plan-apply.yml`:



```

on:

  push:

    branches:

    - main

  pull_request:

    branches:

    - main

```



And comment out this part in the `02-terraform-plan-apply.yml`:



```

on:

  workflow_run:

    workflows: [Terraform Unit Tests]

    types:

      - completed

```



Once the production environment is created, set up a protection rule and include any necessary approvers who must approve production deployments. You may also choose to restrict the environment to your main branch. For more detailed instructions, please see [here](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#environment-protection-rules).



If you have a free version of GitHub no action is needed, but approval process will not be enabled.



# GitLab CI/CD



`.gitlab-ci.yml` is useful if you are planning to run a pipeline on GitLab and implement the GitOps approach.



Remove the `.example` part from the name of the file `.gitlab-ci.yml` for the GitLab pipeline to work.



Note, that you will need to add variables such as AWS_ACCESS_KEY_ID, AWS_DEFAULT_REGION, and AWS_SECRET_ACCESS_KEY in your GitLab projects CI/CD settings section to run your pipeline.



Therefore, you will need to create a service user in advance, using AWS Identity and Access Management (IAM) to get values for these variables and assign an access policy to the user to be able to operate with your resources.



You can delete `.gitlab-ci.yml` if you are not planning to use the GitLab pipeline.



1. **Terraform Unit Tests**



This workflow executes a series of unit tests on the infrastructure code and is triggered by each commit. It begins by running [terraform fmt]( https://www.terraform.io/cli/commands/fmt) to ensure proper code formatting and adherence to terraform best practices. Subsequently, it performs [terraform validate](https://www.terraform.io/cli/commands/validate) to check for syntactical correctness and internal consistency of the code.



To further enhance the code quality and security, two additional tools, tfsec and tflint, are utilized:



tfsec: This step checks the code for potential security issues using tfsec, an open-source security scanner for Terraform. It helps identify any security vulnerabilities or misconfigurations in the infrastructure code.



tflint: This step employs tflint, a Terraform linting tool, to perform additional static code analysis and linting on the Terraform code. It helps detect potential issues and ensures adherence to best practices and coding standards specific to Terraform.



2. **Terraform Plan / Apply**



To ensure accuracy and control over the changes made to your infrastructure, it is essential to manually initiate the job for applying the configuration. Before proceeding with the application, it is crucial to carefully review the generated plan. This step allows you to verify that the proposed changes align with your intended modifications to the infrastructure. By manually reviewing and approving the plan, you can confidently ensure that only the intended modifications will be implemented, mitigating any potential risks or unintended consequences.



# Committing Changes and Triggering Pipeline



Follow these steps to commit changes and trigger the pipeline:



1. **Install pre-commit hooks**: Make sure you have all tools from the requirements section installed.



2. **Clone the Git repository** (If you haven't already):



`git clone `



3. **Navigate to the repository directory**:



`cd `



4. **Create a new branch**:



`git checkout -b `



5. **Make changes** to the Terraform files as needed.



6. **Run pre-commit hooks**: Before committing, run the pre-commit hooks to check for formatting and validation issues:



`pre-commit run --all-files`



7. **Fix any issues**: If the pre-commit hooks report any issues, fix them and re-run the hooks until they pass.



8. **Stage and commit the changes**:



`git add .`



`git commit -m "Your commit message describing the changes"`



9. **Push the changes** to the repository:



`git push origin `



Replace `` with the name of the branch you are working on (e.g., `new-feature-branch-name`).



10.  **Monitor the pipeline**: After pushing the changes, the pipeline will be triggered automatically. You can monitor the progress of the pipeline and check for any issues in the CI/CD interface.



11.  **Merge Request**: If the pipeline is successful and the changes are on a feature branch, create a Merge Request to merge the changes into the main branch. If the pipeline fails, investigate the issue, fix it, and push the changes again to re-trigger the pipeline. Once the merge request is created, your team can review the changes, provide feedback, and approve or request changes. After the merge request has been reviewed and approved, it can be merged into the main branch to apply the changes to the production infrastructure.



# Author



Iā€™m Vladimir Mikhalev, the [Docker Captain](https://www.docker.com/captains/vladimir-mikhalev/), but my friends can call me Valdemar.



šŸŒ My [website](https://www.heyvaldemar.com/) with detailed IT guides\

šŸŽ¬ Follow me on [YouTube](https://www.youtube.com/channel/UCf85kQ0u1sYTTTyKVpxrlyQ?sub_confirmation=1)\

šŸ¦ Follow me on [Twitter](https://twitter.com/heyValdemar)\

šŸŽØ Follow me on [Instagram](https://www.instagram.com/heyvaldemar/)\

šŸ˜ Follow me on [Mastodon](https://mastodon.social/@heyvaldemar)\

šŸ§Š Follow me on [Bluesky](https://bsky.app/profile/heyvaldemar.bsky.social)\

šŸŽø Follow me on [Facebook](https://www.facebook.com/heyValdemarFB/)\

šŸŽ„ Follow me on [TikTok](https://www.tiktok.com/@heyvaldemar)\

šŸ’» Follow me on [LinkedIn](https://www.linkedin.com/in/heyvaldemar/)\

šŸˆ Follow me on [GitHub](https://github.com/heyvaldemar)



# Communication



šŸ‘¾ Chat with IT pros on [Discord](https://discord.gg/AJQGCCBcqf)\

šŸ“§ Reach me at [email protected]



# Give Thanks



šŸ’Ž Support on [GitHub](https://github.com/sponsors/heyValdemar)\

šŸ† Support on [Patreon](https://www.patreon.com/heyValdemar)\

šŸ„¤ Support on [BuyMeaCoffee](https://www.buymeacoffee.com/heyValdemar)\

šŸŖ Support on [Ko-fi](https://ko-fi.com/heyValdemar)\

šŸ’– Support on [PayPal](https://www.paypal.com/paypalme/heyValdemarCOM)