https://github.com/vedanthv/data-engineering-code-snippets

Repo of reusable code snippets for your data engineering projects
https://github.com/vedanthv/data-engineering-code-snippets

Last synced: about 6 hours ago
JSON representation

Repo of reusable code snippets for your data engineering projects

• Host: GitHub
• URL: https://github.com/vedanthv/data-engineering-code-snippets
• Owner: vedanthv
• Created: 2024-06-13T02:02:51.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-06-29T05:53:46.000Z (5 months ago)
• Last Synced: 2024-06-29T06:44:38.895Z (5 months ago)
• Language: HTML
• Size: 2.87 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Reusable Data Engineering Code Snippets

- [x] [ETL Pipeline with Mage Orchestrator and S3 storage](https://github.com/vedanthv/data-engineering-code-snippets/tree/main/mage-s3)

- [x] dbt Integration with Athena

  - [x] Creating materialized tables in Athena using data from S3

  - [x] Creating materialized tables in Athena using data from S3

  - [x] Using references from another model in current view/table

  - [x] Custom tests for views and models

  - [x] Running only one model @ a time

- [x] dbt and duckdb integration