https://github.com/gborn/Serverless-ETL-Pipeline-on-AWS

Design of an ETL Pipeline to process and transform incrementally loaded data in datalake using AWS Lambda, Glue Jobs, EMR, and Athena.
https://github.com/gborn/Serverless-ETL-Pipeline-on-AWS

asyncio aws-athena aws-ecr aws-glue aws-s3 boto3

Last synced: 6 months ago
JSON representation

Design of an ETL Pipeline to process and transform incrementally loaded data in datalake using AWS Lambda, Glue Jobs, EMR, and Athena.

• Host: GitHub
• URL: https://github.com/gborn/Serverless-ETL-Pipeline-on-AWS
• Owner: gborn
• License: mit
• Created: 2022-05-10T07:37:30.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2022-05-10T08:02:26.000Z (over 3 years ago)
• Last Synced: 2024-08-02T22:21:48.712Z (about 1 year ago)
• Topics: asyncio, aws-athena, aws-ecr, aws-glue, aws-s3, boto3
• Language: Python
• Homepage:
• Size: 445 KB
• Stars: 3
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Serverless-ETL-Pipeline-on-AWS

Design of an ETL Pipeline to process and transform incrementally loaded data in datalake using AWS Lambda, Glue Jobs, EMR,  and Athena.