https://github.com/nabojyoti/elt-ipl

This is an End-to-End Data Engineering Project that using the IPL Dataset.
https://github.com/nabojyoti/elt-ipl

apache-spark databricks-notebooks pyspark snowflake snowsql

Last synced: about 1 month ago
JSON representation

This is an End-to-End Data Engineering Project that using the IPL Dataset.

• Host: GitHub
• URL: https://github.com/nabojyoti/elt-ipl
• Owner: nabojyoti
• Created: 2024-04-29T04:09:18.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-04-29T04:20:36.000Z (9 months ago)
• Last Synced: 2024-12-18T03:42:13.361Z (about 1 month ago)
• Topics: apache-spark, databricks-notebooks, pyspark, snowflake, snowsql
• Language: Jupyter Notebook
• Homepage:
• Size: 1.67 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        

# IPL Data Engineering

This is an End-to-End Data Engineering Project that using the IPL Dataset.

## Pipeline Architecture

[![IPL.png](https://i.postimg.cc/9fpNxFRk/IPL.png)](https://postimg.cc/FfYZzvxb)

## Screenshots

### Snowflake Environment

[![Screenshot-from-2024-04-27-14-09-35.png](https://i.postimg.cc/kM03RGxN/Screenshot-from-2024-04-27-14-09-35.png)](https://postimg.cc/9zYNvWgz)

### Databricks Environment

[![Screenshot-from-2024-04-28-13-01-33.png](https://i.postimg.cc/ZKSshd8Q/Screenshot-from-2024-04-28-13-01-33.png)](https://postimg.cc/Z0wx80DP)

### Data Visualization

[![Screenshot-from-2024-04-28-13-02-10.png](https://i.postimg.cc/yxvZ5tXs/Screenshot-from-2024-04-28-13-02-10.png)](https://postimg.cc/644q4Hmm)

[![Screenshot-from-2024-04-28-13-02-26.png](https://i.postimg.cc/8ktf18MV/Screenshot-from-2024-04-28-13-02-26.png)](https://postimg.cc/s1GgcNTn)

## Tech Stack

**Dataset:** IPL-dataset from @data.world

**Extract & Load Data:** SnowSQL & Snowflake DB 

**Transform Data:** Pyspark & Databricks