https://github.com/chinchalinchin/cf-deploy
A collection of reusable CloudFormation stack templates
https://github.com/chinchalinchin/cf-deploy
Last synced: 6 months ago
JSON representation
A collection of reusable CloudFormation stack templates
- Host: GitHub
- URL: https://github.com/chinchalinchin/cf-deploy
- Owner: chinchalinchin
- Created: 2021-11-21T13:15:48.000Z (almost 4 years ago)
- Default Branch: master
- Last Pushed: 2022-05-10T13:34:38.000Z (over 3 years ago)
- Last Synced: 2025-02-02T07:13:50.921Z (8 months ago)
- Language: Python
- Size: 1010 KB
- Stars: 0
- Watchers: 2
- Forks: 1
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# cf-deploy
**cf-deploy** is CLI for automating the deployment of **CloudFormation** stacks through the Python library **boto3**. **cf-deploy** reads in a configuration YAML, parses it into a series of stacks and then posts the stacks to **CloudFormation**.
```shell
aws cloudformation create-stack
--stack-name
--template-body
--parameters ParameterKey=,ParameterValue=
ParameterKey=,ParameterValue=
# ...
```
```python
import boto3
client = boto3.client('cloudformation')
client.create_stack(
StackName=stack,
TemplateBody=template,
Parameters=parameters,
Capabilities=capabilities
)
```
# Provisioning
0. Copy the */env/.sample.env* environment file into a new environment file and configure the values. See notes in the sample file for more information on the purpose of each variable,
```shell
cp ./env/.sample.env ./env/.env
```
1. Create a new **CloudFormation** stack template in the */templates/* directory or select an existing template.
2. Add the stack and its parameters to the *deployments.yml* configuration file,
```yaml
MyNewStack:
template:
parameters:
- ParameterKey:
ParameterValue:
- ParameterKey:
ParameterValue:
## ... as many as it takes ...
```
If the parameter contains sensitive information, such as credentials, put the value in the *.env* file and then reference the variable name in the *deployments.yml* using the `!env` YAML object,
```yaml
MyNewStack:
template:
parameters:
- ParameterKey: secretKey
ParameterValue: !env ENVIRONMENT_SECRET
## ... as many as it takes ...
```
In the above example, the template has a parameter `secretKey` in the `Parameters` section, and the *deployments.yml* passes in the value of the environment variable `ENVIRONMENT_SECRET` for this parameter.
**NOTE**: All environment variables that are required locally are also required within the **Azure DevOps** pipeline. [Refer to the official documentation for information on how to provision variables and secrets within the pipeline](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch)
3. Invoke the python *deployer.py* script, which in turn will use the **boto3** python library to post the contents of *deployments.yml* to **CloudFormation**,
```shell
cf-deploy
```
**NOTE**: In order for this script to succeed, you must have your **AWS CLI** authenticated with an **IAM** account that has permission to deploy resources through **CloudFormation**. Similarly, the **Azure DevOps** pipeline requires an **IAM** account with the appropriate policies attached.
## Predeployment
To prevent the pipeline from having permission to edit its own permissions, the **IAM** resources for the **Innovation Lab** cloud environment are provisioned outside of the **Azure DevOps** pipeline. Similar to the actual deplyoments, resources that need provisioned before the pipeline takes over can be specified and configured in the *predeployments.yml*. This has a corresponding argument in the *deployer.py* script,
```shell
python ./src/deploy/deployer.py predepoloy
```
## Development
When adding a new template, before you push to the remote, make sure you run a local linter against templates and scan them for vulnerabilities.
TODO: pre-commit git hook for cfn-linter.
### Security Scan
Use the [snyk iac test](https://docs.snyk.io/snyk-cli/commands/iac-test) CLI utility to analyze the new template configuration,
```shell
./scripts/scan
```
A report will be output into */reports/* as well as printed to console. Address any security vulnerabilites before pushing, as the CI/CD pipeline for provisioning infrastructure will fail if the security scan fails.
### Linting
Ensure the new template is formatted correctly by running the official [CloudFormation linter](https://github.com/aws-cloudformation/cfn-lint) against the new templates,
## Notes
1. Before provisioning the **VPCStack**, ensure the SSH key has been generated locally and imported into the **AWS EC2** keyring,
```shell
aws ec2 import-key-pair --key-name --public-key-material fileb://
```
**NOTE**: Ensure you import the *public* key, not the *private* key. The *private* key is used to establish the identity of the person initiating an SSH connection.
The bastion host acts as a gateway into the **InnoLab** VPC. After the key has been imported into the bastion host and added to the **SecretsManager**, you can pull the *private* SSH key for tunneling into the bastion host from the **SecretsManager** and initiate a connection with any of the instances in the **VPC** with,
```shell
eval $(ssh-agent -s)
ssh-add ~/.ssh/$KEYNAME
ssh -i $KEYNAME -f -N -L \
:: \
ec2-user@ -v
```
2. Before provisioning the **RDSStack**, ensure secrets for the username and password have been created in the **SecretsManager**. See */templates/rds.yml* lines 77 -78 for the secret naming convention.
## Stack Dependencies
The following tables detail the cross stack dependencies between different stacks. The `@env` table implies the stack is deployed each time into a separate environment, i.e. `Dev`, `Staging` or `Prod`. If a stack does not have `@env` in the following table, this implies the stack's resources do not depend on the environment into which it is deployed; in other words, these resources are global. If the stack explicitly declares an environment (as in the case of `SonarStack` and it's cross stack dependency, `VPCStack-Dev`), this implies this stack is only deployed into that environment.
### DevOps Stacks
| Stack | Dependency |
| ----- | ---------- |
| IAMStack | None |
| RepoStack | None |
| DNSStack | None |
| Doc-PipelineStack | IAMStack, RepoStack |
| Frontend-PipelineStack-@env | IAMStack, RepoStack |
| Lambda-PipelineStack-@env | IAMStack, RepoStack |
### Core Stacks
| Stack | Dependency |
| ----- | ---------- |
| CognitoStack-@env | None |
| VPCStack-@env | None |
| RDSStack-@env | VPCStack-@env, IAMStack |
| ClusterStack-@env | VPCStack-@env |
### Serverless Stacks
| Stack | Dependency |
| ----- | ---------- |
| DynamoStack-@env | None |
| CloudFrontStack | None |
| LambdaStack-@env | VPCStack-@env, RepoStack, CognitoStack-@env, IAMStack |
### Services Stacks
**Note**: These stacks are deployed into the **Fargate ECS** cluster provisioned through the `ClusterStack-@env`.
| Stack | Dependency |
| ----- | ---------- |
| Frontend-ServiceStack-@env | IAMStack, RepoStack, VPCStack-@env, ClusterStack-@env |
| Backend-ServiceStack-@env | IAMStack, RepoStack, VPCStack-@env, ClusterStack-@env |
| Sonar-ServiceStack | IAMStack, RepoStack, VPCStack-Dev, ClusterStack-Dev |
### Application Stacks
| Stack | Dependency |
| ----- | --------- |
| AlationStack | VPCStack-Dev |
## References
- [AWS CloudFormation Resource Reference](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html)
- [boto3 CloudFormation Client](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cloudformation.html)