Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/trannhatnguyen2/bi_datalake_azure

Building Data Lake on the Microsoft Azure Cloud Platform
https://github.com/trannhatnguyen2/bi_datalake_azure

azure databricks datalake powerbi sql-server

Last synced: 15 days ago
JSON representation

Building Data Lake on the Microsoft Azure Cloud Platform

Host: GitHub
URL: https://github.com/trannhatnguyen2/bi_datalake_azure
Owner: trannhatnguyen2
Created: 2023-10-31T17:34:21.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2023-10-31T17:45:23.000Z (over 1 year ago)
Last Synced: 2024-11-16T11:42:16.838Z (3 months ago)
Topics: azure, databricks, datalake, powerbi, sql-server
Homepage:
Size: 72.3 KB
Stars: 0
Watchers: 1
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

        # [Building Data Lake on the Microsoft Azure Cloud Platform](hhttps://github.com/trannhatnguyen2/BI_DataLake_Azure)

## Member of group

| student_id | class   | full_name        |

| ---------- | ------- | ---------------- |

| K204061440 | K20406T | Tran Nhat Nguyen |

# 📕 Table of contents

- 🛠️ [Tools](#️-tools)

- 🚀 [Solution](#-solution)




# 🛠️ Tools

| Tools                        |

| ---------------------------- |

| SQL Server                   |

| Azure Data Factory           |

| Azure Data Lake Storage Gen2 |

| Azure Databricks             |

| Azure Synapse Analytics      |

| Azure Key Vault              |

| Azure Active Directory       |

| Microsoft Power BI           |

# 🚀 Solution







    Solution



The use case for this project is building an end-to-end solution by ingesting the tables from the on-premise SQL Server database using Azure Data Factory and then storing the data in Azure Data Lake Storage Gen2. Then Azure Databricks is used to transform the RAW data to the cleanest form of data and then we use Azure Synapse Analytics to load the clean data and finally use Microsoft Power BI to integrate with Azure Synapse Analytics to build an interactive dashboard. Also, we are using Azure Active Directory and Azure Key Vault for monitoring and governance purposes.

---

© 2023 NhatNguyen