https://github.com/bmlt-enabled/aggregator-failed-import-notifier
Checks for failed aggregator imports and then sends slack notification to the BMLT-Enabled #root-status channel.
https://github.com/bmlt-enabled/aggregator-failed-import-notifier
Last synced: 5 days ago
JSON representation
Checks for failed aggregator imports and then sends slack notification to the BMLT-Enabled #root-status channel.
- Host: GitHub
- URL: https://github.com/bmlt-enabled/aggregator-failed-import-notifier
- Owner: bmlt-enabled
- License: mit
- Created: 2020-03-05T02:26:09.000Z (over 6 years ago)
- Default Branch: main
- Last Pushed: 2024-04-24T03:56:42.000Z (about 2 years ago)
- Last Synced: 2024-04-24T08:16:43.411Z (about 2 years ago)
- Language: HCL
- Homepage:
- Size: 29.3 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Aggregator Import Failure Notifier
Checks for failed aggregator imports and then sends slack notification to the BMLT-Enabled `root-status` channel.
## What does this do?
This creates a lambda that gets triggered whenever the aggregator ECS task state changes, It then checks if the desired
status is stopped so we know that the task has ended. We then hit the root server endpoint of aggregator and check if any of
last successful import times is less than the ECS task start time. If it is then it sends a slack notification to the
BMLT-Enabled `root-status` channel.
## Why?
I had previously created a php script that accomplishes this similarly but was never really happy with it. It ran on a cron
and wasn't precise enough. Adding to 47 bot would of been more of the same. With this it gets triggered as soon as the import
is done and we will know right away if a import failed.
## Setup
To install clone this repo, then from the terraform directory. You then should login to console and set the hook in the
lambdas env variable. Setting it in the variable can be dangerous because if you then commit the state file you would have
also commited the hook. Then run.
```
terraform init
```
This will initialize the terraform. now run.
```
terraform plan
```
This will zip the lambda using the archive data source, As well as give you the plan for what the apply stage will
perform. If all looks good you can now run.
```
terraform apply
```
## TODO:
If a root server fails an import, it would be interesting to look into the possibility of pulling the cloudwatch logs and
attaching the error message along with the slack notification. Most of the time they are just connection errors.