https://github.com/tmusabbir/glue-utils
Few AWS Glue Utility Scripts
https://github.com/tmusabbir/glue-utils
amazon-web-services aws emr etl glue lakeformation
Last synced: 3 months ago
JSON representation
Few AWS Glue Utility Scripts
- Host: GitHub
- URL: https://github.com/tmusabbir/glue-utils
- Owner: tmusabbir
- License: apache-2.0
- Created: 2020-10-08T17:03:10.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2020-10-09T17:12:52.000Z (over 4 years ago)
- Last Synced: 2024-08-13T07:15:56.886Z (6 months ago)
- Topics: amazon-web-services, aws, emr, etl, glue, lakeformation
- Language: Shell
- Homepage:
- Size: 169 KB
- Stars: 2
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- jimsghstars - tmusabbir/glue-utils - Few AWS Glue Utility Scripts (Shell)
README
# AWS Glue & AWS Lake Formation Utility Scripts
A few AWS Glue & Lake Formation scripts that you might find useful and can save your development time.
## Delete all Glue resources associated with a Lake Formation Blueprint
When you create a Blueprint through AWS Lake Formation, it creates a set of Glue resources - Glue Crawler, few
Spark & Python shell jobs under Glue ETL, few Glue Triggers, and a Glue workflow. However, when you delete that
Blueprint (as of 10/2020 ), corresponding Glue resources don't get deleted automatically. Also, through Glue console, you cannot delete
multiple Glue jobs together, you need to delete them individually. So, the clean-up process can take time.
The following script deletes all Blueprint related Glue resources based on the Blueprint name. Before proceeding, make
sure
you configure AWS CLI for the account where you want to clean up your Glue resources.
[deleteResourceByBlueprintName.sh](scripts/deleteResourcesByBlueprintName.sh)
It takes three parameters:
- -d: [Optional parameter] It dry-runs the command without executing it
- -b: Name of the blueprint
Example:
With dry-run flag turned on:
```shell script
./deleteResourcesByBlueprintName.sh -b test-workflow -d
```
Without dry-run flag:
```shell script
./deleteResourcesByBlueprintName.sh -b test-workflow
```
