https://github.com/enricogoerlitz/explore-airflow

Apache Airflow project for orchestrating ETL workflows in a data warehouse environment. Implements a medallion architecture (Bronze, Silver, Gold layers) with Postgres integration for scalable and modular data processing pipelines.
https://github.com/enricogoerlitz/explore-airflow

airflow airflow-dags airflow-docker pandas postgres stored-procedures

Last synced: 6 months ago
JSON representation

Apache Airflow project for orchestrating ETL workflows in a data warehouse environment. Implements a medallion architecture (Bronze, Silver, Gold layers) with Postgres integration for scalable and modular data processing pipelines.

• Host: GitHub
• URL: https://github.com/enricogoerlitz/explore-airflow
• Owner: enricogoerlitz
• Created: 2025-03-12T20:09:37.000Z (7 months ago)
• Default Branch: main
• Last Pushed: 2025-03-31T10:15:03.000Z (6 months ago)
• Last Synced: 2025-03-31T11:24:42.959Z (6 months ago)
• Topics: airflow, airflow-dags, airflow-docker, pandas, postgres, stored-procedures
• Language: Python
• Homepage:
• Size: 1.33 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Explore Airflow

## DAG Architecture

### Master DAG

![Master DAG](./resources/architecture/01%20Master%20DAG.png)

### Bronze Layer

![Bronze Master DAG](./resources/architecture/02%20Bronze%20Master%20DAG.png)

![Bronze PT Import DAG](./resources/architecture/03%20Bronze%20PT%20Import%20DAG.png)

### Silver Layer

![Silver Master DAG](./resources/architecture/04%20Silver%20Master%20DAG.png)

![Silver PT Transoformations DAG](./resources/architecture/05%20Silver%20Transformations%20DAG.png)

### Gold Layer

![Gold Master DAG](./resources/architecture/06%20Gold%20Master%20DAG.png)

![Gold Materialization DAG](./resources/architecture/07%20Gold%20Materialization%20DAG.png)