Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/leo-the-nardo/combopurifier

Data Pipeline from AWS SQS/S3 to Kubernetes w/ Spark using Airflow, EKS & Data Lakehouse
https://github.com/leo-the-nardo/combopurifier

airflow argocd aws-glue-catalog aws-lake-formation aws-s3 aws-sqs data-lake delta-lake eks minio spark terraform

Last synced: 16 days ago
JSON representation

Data Pipeline from AWS SQS/S3 to Kubernetes w/ Spark using Airflow, EKS & Data Lakehouse

Awesome Lists containing this project

README

        

### just another POC to play with Data Engineering concepts and some techs. ![golango.png](assets/golango.png)
![poc.png](./assets/poc.png)