https://github.com/soumilshah1995/How-to-receive-notifications-when-your-Glue-ETl-scripts-fail-Email-Alerts

Set up a job alert to notify you each time your job succeeds or fails if you're concerned about the status of your Glue job and are weary of constantly checking the job console for problems or success. Today, we'll put up a mechanism for notifications that will let us know when one of our Glue jobs has failed or halted.
https://github.com/soumilshah1995/How-to-receive-notifications-when-your-Glue-ETl-scripts-fail-Email-Alerts

Last synced: 5 months ago
JSON representation

Set up a job alert to notify you each time your job succeeds or fails if you're concerned about the status of your Glue job and are weary of constantly checking the job console for problems or success. Today, we'll put up a mechanism for notifications that will let us know when one of our Glue jobs has failed or halted.

• Host: GitHub
• URL: https://github.com/soumilshah1995/How-to-receive-notifications-when-your-Glue-ETl-scripts-fail-Email-Alerts
• Owner: soumilshah1995
• License: apache-2.0
• Created: 2022-11-02T16:45:07.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2022-11-02T20:13:41.000Z (over 2 years ago)
• Last Synced: 2024-08-13T07:08:54.746Z (8 months ago)
• Language: Python
• Size: 28.3 KB
• Stars: 2
• Watchers: 2
• Forks: 4
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• jimsghstars - soumilshah1995/How-to-receive-notifications-when-your-Glue-ETl-scripts-fail-Email-Alerts - Set up a job alert to notify you each time your job succeeds or fails if you're concerned about the status of your Glue job and are weary of constantly checking the job console for problems or success (Python)

README

        
# How-to-receive-notifications-when-your-Glue-ETl-scripts-fail-Email-Alerts

![image](https://user-images.githubusercontent.com/39345855/199550250-5fb8e90d-6855-4e89-a038-d51f124f556d.png)

* Set up a job alert to notify you each time your job succeeds or fails if you're concerned about the status of your Glue job and are exhausted of constantly checking the job console for errors or success this solution will really help you

---------------------------------------------------------------------

#### Solution:

* Whenever the state changes for glue job that event shall be sent to AWS Event Bridge and where we shall have a rule if the event matches with given rule then event shall be passed to Lambda function which will process and send details to SNS Topic where subscribed candidates can be notified via email 

#### Steps to use 

##### Step 1: Download the repository

```

git clone https://github.com/soumilshah1995/How-to-receive-notifications-when-your-Glue-ETl-scripts-fail-Email-Alerts.git

```

##### Step 2: change the Env Values 

```

ACCOUNT=XXXXX

TopicName=glue-jobs-topics

DEV_ACCESS_KEY=XXXXXXXXX

DEV_SECRET_KEY=XXX

DEV_REGION=us-east-1

```

* replace where it says XXXX with your AWS Account ID

##### Step 3: Add your Email Address

```

    MySubscription:

      Type: AWS::SNS::Subscription

      Properties:

        Endpoint: 

        Protocol: email

        TopicArn: !Ref 'SNSTopic'

```

##### Step 4 Deploy

```

serverless config credentials --provider aws --key   --secret   -o

sls deploy

OR

npx sls deploy

```

### Enjoy 

![image](https://user-images.githubusercontent.com/39345855/199551616-eaedf846-6f59-480d-8eaf-73eccb5bdab8.png)

### Special Thanks 

* https://aws.amazon.com/premiumsupport/knowledge-center/glue-job-fail-retry-lambda-sns-alerts/

* https://aarfahrayees.medium.com/easy-way-for-setting-up-notification-for-aws-glue-jobs-that-failed-using-sns-a002a52a1b72