Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/leo-the-nardo/combopurifier
Data Pipeline from AWS SQS/S3 to Kubernetes w/ Spark using Airflow, EKS & Data Lakehouse
https://github.com/leo-the-nardo/combopurifier
airflow argocd aws-glue-catalog aws-lake-formation aws-s3 aws-sqs data-lake delta-lake eks minio spark terraform
Last synced: 16 days ago
JSON representation
Data Pipeline from AWS SQS/S3 to Kubernetes w/ Spark using Airflow, EKS & Data Lakehouse
- Host: GitHub
- URL: https://github.com/leo-the-nardo/combopurifier
- Owner: leo-the-nardo
- Created: 2024-10-31T12:02:26.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2024-11-09T08:19:57.000Z (2 months ago)
- Last Synced: 2024-11-09T08:20:24.538Z (2 months ago)
- Topics: airflow, argocd, aws-glue-catalog, aws-lake-formation, aws-s3, aws-sqs, data-lake, delta-lake, eks, minio, spark, terraform
- Language: Python
- Homepage:
- Size: 27 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
### just another POC to play with Data Engineering concepts and some techs. ![golango.png](assets/golango.png)
![poc.png](./assets/poc.png)