Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/datawithbaraa/sql-modern-warehouse-and-analytics
A comprehensive guide to building a modern data warehouse with SQL Server, including ETL processes, data modeling, and analytics.
https://github.com/datawithbaraa/sql-modern-warehouse-and-analytics
data-analysis data-analytics data-cleaning data-engineering data-lake data-lakehouse data-science data-warehouse data-warehousing database datalake datascience datawarehouse datawarehousing etl medallion-architecture pipeline sql sql-query sql-server
Last synced: 13 days ago
JSON representation
A comprehensive guide to building a modern data warehouse with SQL Server, including ETL processes, data modeling, and analytics.
- Host: GitHub
- URL: https://github.com/datawithbaraa/sql-modern-warehouse-and-analytics
- Owner: DataWithBaraa
- License: mit
- Created: 2024-12-22T15:41:10.000Z (14 days ago)
- Default Branch: main
- Last Pushed: 2024-12-22T16:26:36.000Z (14 days ago)
- Last Synced: 2024-12-22T16:30:51.814Z (14 days ago)
- Topics: data-analysis, data-analytics, data-cleaning, data-engineering, data-lake, data-lakehouse, data-science, data-warehouse, data-warehousing, database, datalake, datascience, datawarehouse, datawarehousing, etl, medallion-architecture, pipeline, sql, sql-query, sql-server
- Homepage:
- Size: 0 Bytes
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Data Warehouse and Analysis Project
Welcome to the **Data Warehouse and Analysis Project** repository! š This project showcases a complete data warehousing solution, from building a data warehouse to performing insightful analytics and creating dashboards. Designed as a portfolio piece, this repository highlights industry best practices in data engineering and analytics.
---
## š Project Overview
This project involves:
1. **Data Architecture**: Establishing a robust data warehouse architecture with Bronze, Silver, and Gold layers.
2. **ETL Pipelines**: Extracting, transforming, and loading data from source systems into the data warehouse.
3. **Data Modeling**: Creating fact and dimension tables to support analytical queries.
4. **Analytics & Reporting**: Developing SQL-based reports and dashboards for actionable insights.
This repository is an excellent resource for students and professionals aiming to demonstrate their skills in:
- SQL Development
- ETL Process Design
- Data Modeling
- Data Analytics
---
## šļø Repository Structure
```
data-warehouse-and-analysis/
ā
āāā README.md
āāā data/
ā āāā source_crm/ # CRM source data files
ā āāā source_erp/ # ERP source data files
ā
āāā scripts/
ā āāā database_setup/ # Scripts for database and schema creation
ā āāā bronze_layer/ # Scripts for the Bronze layer
ā āāā silver_layer/ # Scripts for the Silver layer
ā āāā gold_layer/ # Scripts for the Gold layer
ā
āāā analytics/
ā āāā reports/ # SQL scripts for analytical reports
ā āāā dashboards/ # Dashboard designs and descriptions
ā
āāā docs/
ā āāā diagrams/ # Architecture, data lineage, and data model diagrams
ā āāā naming_conventions.md # Naming conventions for tables, columns, etc.
ā āāā project_overview.md # Detailed project overview
ā āāā database_design.md # Database schema documentation
ā āāā ETL_process.md # ETL process documentation
ā āāā analytics_overview.md # Analytical process documentation
ā āāā dashboard_design.md # Dashboard design documentation
ā
āāā LICENSE
āāā .gitignore
```
---
## š ļø Setup Instructions
### Prerequisites
- **SQL Server**: Install SQL Server or a compatible database.
- **SQL Client**: Tools like SQL Server Management Studio (SSMS).
## š Key Features
### 1. **Data Warehouse Architecture**
- Bronze Layer: Raw data from CRM and ERP systems.
- Silver Layer: Cleaned and enriched data.
- Gold Layer: Fact and dimension tables for analytics.
### 2. **ETL Processes**
- Automated loading and transformation of data across layers.
- Error handling and validation mechanisms.
### 3. **Analytics & Reporting**
- Customer segmentation and retention analysis.
- Product performance and profitability analysis.
- Monthly and yearly sales trends.
### 4. **Visual Diagrams**
- **System Architecture**: High-level overview of the data flow.
- **Data Lineage**: End-to-end data transformation journey.
- **Data Model**: ER diagram of fact and dimension tables.
---
## š Documentation
Explore detailed documentation in the `docs/` folder:
- **Project Overview**: Goals and methodology.
- **Database Design**: Explanation of schemas and table relationships.
- **ETL Process**: Step-by-step ETL pipeline details.
- **Analytics Overview**: KPIs and reporting logic.
- **Dashboard Design**: Insights and visualizations.
- **Naming Conventions**: Standards for tables, columns, and scripts.
---
## š”ļø License
This project is licensed under the [MIT License](LICENSE). You are free to use, modify, and share this project with proper attribution.
---
## š About the Author
Hi! Iām **Baraa Khatib Salkini**, also known as **Data With Baraa**. I am passionate about data and love sharing knowledge through my projects and tutorials.
- **YouTube**: [Data With Baraa](http://bit.ly/3GiCVUE)
- **LinkedIn**: [Baraa Khatib Salkini](https://linkedin.com/in/baraa-khatib-salkini)
- **Website**: [www.datawithbaraa.com](https://www.datawithbaraa.com)
---
## š§ Contact
For questions or feedback, reach out to me via [LinkedIn](https://linkedin.com/in/baraa-khatib-salkini) or email.
Happy learning and analyzing! š