https://github.com/tosh2230/aws-glue-crawlflow
Run AWS Glue Crawler and check the status by AWS Step functions.
https://github.com/tosh2230/aws-glue-crawlflow
aws-glue aws-lambda aws-sam aws-sqs aws-step-functions python
Last synced: about 1 month ago
JSON representation
Run AWS Glue Crawler and check the status by AWS Step functions.
- Host: GitHub
- URL: https://github.com/tosh2230/aws-glue-crawlflow
- Owner: tosh2230
- License: mit
- Created: 2020-08-13T06:48:16.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2021-11-19T23:07:14.000Z (about 4 years ago)
- Last Synced: 2025-05-16T17:40:06.842Z (6 months ago)
- Topics: aws-glue, aws-lambda, aws-sam, aws-sqs, aws-step-functions, python
- Language: Python
- Homepage:
- Size: 105 KB
- Stars: 2
- Watchers: 1
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# aws-glue-crawlflow
aws-glue-crawlflow helps you to run AWS Glue Crawler automatically.
## Description
AWS Services what to be called are below.
- AWS Lambda
- Amazon SQS
- AWS Step Functions
- AWS Glue
## Install
This app is created to be deployed by AWS SAM(Serverless Application Model).
To install the AWS SAM CLI, see following pages.
[Installing the AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)
```bash
git clone https://github.com/tosh223/aws-glue-crawlflow.git
cd ./aws-glue-crawlflow
sam build
sam deploy --guided
```
## Usage
Call a Lambda function 'enqueue_glue_crawlflow' with event like below.
```json
{
"CrawlerName": "your-crawler-name",
"WaitTime": "60",
"CheckCount": "0"
}
```
## References
[Invoke Lambda with Step Functions](https://docs.aws.amazon.com/step-functions/latest/dg/connect-lambda.html)
[Glueの使い方的な⑦(Step Functionsでジョブフロー)](https://qiita.com/pioho07/items/f8a2fd946fc391f89c97)