Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/d-w-arnold/aws-scripts-examples
Scripts for an AWS eco-system, to support: Examples of Infrastructure as Code (IaC) source code, using AWS CDK.
https://github.com/d-w-arnold/aws-scripts-examples
aws-cli boto3 python3 shell
Last synced: 27 days ago
JSON representation
Scripts for an AWS eco-system, to support: Examples of Infrastructure as Code (IaC) source code, using AWS CDK.
- Host: GitHub
- URL: https://github.com/d-w-arnold/aws-scripts-examples
- Owner: d-w-arnold
- License: gpl-3.0
- Created: 2024-11-14T16:56:47.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2024-11-20T12:01:18.000Z (about 2 months ago)
- Last Synced: 2024-11-20T13:18:45.367Z (about 2 months ago)
- Topics: aws-cli, boto3, python3, shell
- Language: Python
- Homepage: https://github.com/d-w-arnold/aws-cdk-examples
- Size: 91.8 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# aws-scripts-examples
This repo is complimentary to my AWS CDK repo: [aws-cdk-examples](https://github.com/d-w-arnold/aws-cdk-examples)
### Prerequisites
1) [Installing or updating the latest version of the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
2) [AWS CLI Configuration basics](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html), or simply see below:
```bash
# Configure AWS CLI
aws configure# Input AWS Access Key ID
# Input AWS Secret Access Key
# Input Default region name: eu-west-2 (London)
# Input Default output format: json# Check AWS CLI Configuration
aws configure list
```3) [Install the Session Manager plugin for the AWS CLI](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html)
`**IMPORTANT**: In the next step, please ask the DevOps Engineer to share access to the correct PEM file needed for accessing the Bastion Hosts.`
4) Complete step 2 only, [Allowing SSH connections for Session Manager](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html#ssh-connections-enable).
[//]: # (TODO: [IMPORTANT] Add AWS CLI region arg to SSH config, so the `aws-private.sh` region arg is honoured in the SSH config SSM start-session command: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html#ssh-connections-enable)
### Install Python dependencies
Python dependencies (mainly the AWS Python SDK - `boto3` ) should be setup in a Python virtual environment (e.g. 'venv'):
```bash
# pwd: `aws-scripts` directorypython3 -m venv venv
source ./venv/bin/activate
python3 -m pip install -r requirements.txt
# Optional: install Python dev dependencies
python3 -m pip install -r requirements-dev.txt
```### Example usages:
```bash
cd# Shell Script(s)
./.sh -h# Python Script(s)
# IMPORTANT: IF there is a Shell script similarly named
# to a Python script, in the same directory,
# the Shell script will invoke the Python script.
#
# e.g. In directory `foobar`, the Shell script `foobar/tmp.sh`
# will invoke the Python script `foobar/tmp.py`
python3 .py -h
```### Support / Output files
For the latest list of files/folders generated by the many scripts in this git repo, see:
`.gitignore`
(specifically lines preceding the comment:
`# --- Files and folders generated by scripts in this git repo ---`)
### AWS AmazonMQ Broker
#### [aws-amazonmq-broker/aws-amazonmq-broker-web.py](aws-amazonmq-broker/aws-amazonmq-broker-web.py)
For each Amazon MQ (RabbitMQ) broker, find the managed AWS VPC endpoint for said broker, and generate a SSM Parameter Store parameter for each subnet ENI, recording the private IPv4 address of each ENI.
### AWS Clean-Up
#### [aws-clean-up/aws-clean-up-dns.py](aws-clean-up/aws-clean-up-dns.py)
Clean-up AWS Route53 DNS records, which are no longer in use by: Amazon Certificate Manager (ACM), AWS Amplify, etc.
#### [aws-clean-up/aws-clean-up-logs.py](aws-clean-up/aws-clean-up-logs.py)
Clean-up AWS CloudWatch Logs log groups which are not provisioned by an AWS CDK stack and do not have a corresponding AWS resource sending logs: Lambda function, ECS cluster, RDS instance, etc."
### AWS Cost Explorer
#### [aws-cost-explorer/aws-cost-explorer.py](aws-cost-explorer/aws-cost-explorer.py)
Collect cost and usage from AWS Cost Explorer. This is useful for accounting purposes, for looking into the last 'n' months of AWS cloud costs.
### AWS Create
#### [aws-create/aws-create-amplify.sh](aws-create/aws-create-amplify.sh)
Create a new AWS Amplify app. This is useful for creating a (Web) Portal component for a project/product.
#### [aws-create/aws-create-amplify-notifications.sh](aws-create/aws-create-amplify-notifications.sh)
Create an MS Teams notifications setup for a newly created AWS Amplify app. This is useful for allowing the AWS Amplify build pipeline notifications, to be sent to a corresponding MS Teams channel - rather than having to really on email-only notification (which currently is all AWS Amplify supports setting up in the AWS management console).
#### [aws-create/aws-create-codeartifact.sh](aws-create/aws-create-codeartifact.sh)
Create a new CodeArtifact repository, for storing of npm packages.
#### [aws-create/aws-create-sns-mob-push.sh](aws-create/aws-create-sns-mob-push.sh)
Create a new SNS platform application. This is useful for creating an SNS platform application, to be used by a Gw component, to send push notifications directly to mobile apps.
### AWS Delete
#### [aws-delete/aws-delete-amplify.sh](aws-delete/aws-delete-amplify.sh)
Delete an existing AWS Amplify app. This is useful for deleting a (Web) Portal component for a project/product.
#### [aws-delete/aws-delete-codeartifact.sh](aws-delete/aws-delete-codeartifact.sh)
Delete an existing CodeArtifact repository, for storing of npm packages.
#### [aws-delete/aws-delete-sns-mob-push.sh](aws-delete/aws-delete-sns-mob-push.sh)
Delete a new SNS platform application. This is useful for deleting an SNS platform application.
### AWS Deploy
#### [aws-deploy/aws-deploy.sh](aws-deploy/aws-deploy.sh)
Git tag a new commit, and deploy (release) changes via the corresponding AWS CodePipeline pipeline. This is useful for releasing changes on AWS CodePipeline pipelines which need to be manually started, and make reference of a deployment git tag, specified as a git tag in the commit history of a git repo.
#### [aws-deploy/aws-deploy-amplify.sh](aws-deploy/aws-deploy-amplify.sh)
Git tag a new commit, and deploy (release) changes via the corresponding AWS Amplify app build pipeline. This is useful for releasing changes on AWS Amplify app build pipelines which need to be manually started (via web-hook), and make reference of a deployment git tag, specified as a git tag in the commit history of a git repo.
### AWS NAT Gateway
#### [aws-nat-gateway/aws-nat-gateway-public-ips.sh](aws-nat-gateway/aws-nat-gateway-public-ips.sh)
Generate an updated AWS Systems Manager Parameter Store parameter, used by AWS CDK to reference all NAT Gateway Public IPs (of all custom AWS VPCs), across all AWS regions.
### AWS OpenVPN-VPN-Server-NLB
#### [aws-openvpn-vpn-server-nlb/aws-openvpn-vpn-server-nlb.py](aws-openvpn-vpn-server-nlb/aws-openvpn-vpn-server-nlb.py)
For all OpenVPN Server NLBs (of all custom AWS VPCs), across all AWS regions, generate a SSM Parameter Store parameter recording all OpenVPN Server NLB Public IPs.
### AWS Private
#### [aws-private/aws-private.sh](aws-private/aws-private.sh)
Connect to (one or multiple) private AWS resources (resources which not accessible via the public internet).
This script simplifies accessing private AWS resources, by utilising multiple private Bastion Hosts setup in each (regional) AWS VPC, and an AWS IAM user's permissions to use AWS System Manager (SSM) Session Manager. See [original blog post](https://aws.amazon.com/premiumsupport/knowledge-center/systems-manager-ssh-vpc-resources/) for more info.
The result of running this script is an open SSH connection to a private Bastion Host, and each supplied arg (each arg representing a single private AWS VPC resource) gets mapped to a localhost port, so long as the SSH connection persists.
#### (Optional) [aws-private/aws-private-London.sh](aws-private/aws-private-London.sh)
Invokes the `aws-private/aws-private.sh` script, for the London (e.g. `eu-west-2`) AWS region.
#### [aws-private/aws-private-ecs.sh](aws-private/aws-private-ecs.sh)
Connect to a single private AWS ECS container resource (not accessible via the public internet).
The result of running this script is a secure connection to the specified ECS container for debugging purposes (NB. All commands executed while connected to an ECS container are logged).
#### (Optional) [aws-private/aws-private-ecs-London.sh](aws-private/aws-private-ecs-London.sh)
Invokes the `aws-private/aws-private-ecs.sh` script, for the London (e.g. `eu-west-2`) AWS region.
### AWS Query
#### [aws-query/aws-query-sns-topics-with-no-subs.py](aws-query/aws-query-sns-topics-with-no-subs.py)
Query for the list of AWS SNS topics, which have no subscriptions.
### AWS Tag
#### [aws-tag/aws-tag-amplify-app-resources.py](aws-tag/aws-tag-amplify-app-resources.py)
Tag all AWS Amplify app additional resources.
This is useful for propagating changes in the tags on an AWS Amplify app, to all it's additional associated resources.
(e.g. CloudWatch alarm(s), and default the EventBridge rule and SNS topic of the AWS Amplify app).
#### [aws-tag/aws-tag-amplify-apps.py](aws-tag/aws-tag-amplify-apps.py)
Tag all AWS Amplify apps, specified in a local config file (see Python script help info for more details).
This is useful for updating AWS tags on an AWS Amplify app, especially one which uses a Back-End from a different project name, and so the AWS Amplify app during creation will inherit most AWS tags from the Back-End specified - this Python script can be used to update the AWS Amplify app's AWS tags to show the correct project name for the AWS Amplify app.
### ECR Retagging
#### [ecr-retagging/ecr-retagging.sh](ecr-retagging/ecr-retagging.sh)
Re-tag an Elastic Container Registry (ECR) image.
### PyPi Server
Generate `.pypirc` and `pip.conf` files for use with PyPi server(s).
#### [pypi/pypirc.sh](pypi/pypirc.sh)
For uploading python packages..
Get the latest PyPi server details and credentials, for a given AWS region (for a given AWS account).
Args:
1) The AWS region
2) (Optional) For when the script is being run in an AWS CodePipeline pipeline. Valid options: [`"pipeline"`] - stops the script writing to stdout```bash
cd pypi
./pypirc.sh eu-west-2
```#### [pypi/pip-conf.sh](pypi/pip-conf.sh)
For downloading python packages..
Get the latest PyPi server details and credentials, for a given AWS region (for a given AWS account).
Args:
1) The AWS region
2) (Optional) For when the script is being run in an AWS CodePipeline pipeline. Valid options: [`"pipeline"` | `"pipeline-internal"`] - both options stop the script writing to stdout, and the `"pipeline-internal"` substitutes the PyPi server public URL for the private (to AWS VPC) DNS name (and any host port detail)```bash
cd pypi
./pip-conf.sh eu-west-2
```### RDS Backup
#### [rds-backup/rds-backup.py](rds-backup/rds-backup.py)
For a specific database (DB), on an RDS instance, backup all DB tables to CSV files - one CSV file per DB table.
### RDS Init
#### [rds-init/rds-init.sh](rds-init/rds-init.sh)
Invoke Relational Database Service (RDS) init Lambda functions for specific database (DB) servers.
This is useful for resetting or incrementing changes to DB schemas on RDS instance DB servers, per a specified SQL file.
### React Native
#### [react-native/react-native-dev-to-main-pre-merge.sh](react-native/react-native-dev-to-main-pre-merge.sh)
For a specified (react-native) git repo, update package.json, git commit and push, and create a new Pull Request with repo default reviewers.
#### [react-native/react-native-prod-release.sh](react-native/react-native-prod-release.sh)
Git tag a new commit, and deploy (release) changes via Fastlane for prod release to public app stores.
### S3 Backup
[//]: # (# s3-backup/s3-backup.sh)
#### [s3-backup/s3-backup.sh](s3-backup/s3-backup.sh)
Backup all AWS S3 Buckets matching a name prefix.
### S3 Encrypt
#### [s3-encrypt/s3-encrypt.sh](s3-encrypt/s3-encrypt.sh)
Encrypt S3 buckets with a different KMS key.
This is useful for projects such as Dog, where the Dog Gw can create S3 buckets per institution,
meaning these newly created S3 buckets are not moderated by AWS CDK. So if the encryption KMS key were to change,
existing S3 buckets would still be encrypted using an old KMS key (possibly one even set to 'Pending Deletion').### S3 Upload
#### [s3-upload/s3-upload.sh](s3-upload/s3-upload.sh)
Upload files from git repos to AWS S3 buckets.
This is useful for pushing SQL files in a project git repo needed by any AWS Lambda Functions (setup in AWS CDK), which run as part of an AWS CDK deployment of a database server stack, to initialise database server schemas per the SQL file.
### User Details
#### [user-details/proxy-user-details.sh](user-details/proxy-user-details.sh)
Get the latest Proxy server details and credentials, for a given AWS region.
Args:
1) The AWS region```bash
cd user-details
./proxy-user-details.sh eu-west-2
```#### [user-details/vpn-user-details.sh](user-details/vpn-user-details.sh)
Get the latest VPN server details and credentials, for a given VPN user, on a VPN server in a given AWS region.
Args:
1) The AWS region
2) The VPN username (normally, a person's first name and first initial of any last names)```bash
cd user-details
# VPN user: John Smith
./vpn-user-details.sh eu-west-2 johns
# VPN user: John Taylor-Smith
./vpn-user-details.sh eu-west-2 johnts
```