{"id":28297175,"url":"https://github.com/k178412/sql-data-warehouse-project","last_synced_at":"2026-04-25T12:32:37.210Z","repository":{"id":292667432,"uuid":"981577349","full_name":"k178412/sql-data-warehouse-project","owner":"k178412","description":"A hands-on data warehouse project using SQL Server, covering ETL processes, and data modeling.","archived":false,"fork":false,"pushed_at":"2025-06-28T06:12:56.000Z","size":1833,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-28T07:25:05.495Z","etag":null,"topics":["bronze-layer","data-analysis","data-analytics","data-cleaning","data-engineering","data-warehouse","database","datalake","dataset","datawarehouse","etl","etl-pipeline","etl-process","gold-layer","silver-layer","sql","sql-query","sql-server","sqlserver"],"latest_commit_sha":null,"homepage":"","language":"TSQL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/k178412.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-11T12:31:21.000Z","updated_at":"2025-06-28T06:13:00.000Z","dependencies_parsed_at":null,"dependency_job_id":"a8bc36d7-1907-481c-a465-a98278134cb5","html_url":"https://github.com/k178412/sql-data-warehouse-project","commit_stats":null,"previous_names":["k178412/sql-data-warehouse-project"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/k178412/sql-data-warehouse-project","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k178412%2Fsql-data-warehouse-project","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k178412%2Fsql-data-warehouse-project/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k178412%2Fsql-data-warehouse-project/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k178412%2Fsql-data-warehouse-project/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/k178412","download_url":"https://codeload.github.com/k178412/sql-data-warehouse-project/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k178412%2Fsql-data-warehouse-project/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32262801,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-25T09:15:33.318Z","status":"ssl_error","status_checked_at":"2026-04-25T09:15:31.997Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bronze-layer","data-analysis","data-analytics","data-cleaning","data-engineering","data-warehouse","database","datalake","dataset","datawarehouse","etl","etl-pipeline","etl-process","gold-layer","silver-layer","sql","sql-query","sql-server","sqlserver"],"created_at":"2025-05-23T01:17:43.953Z","updated_at":"2026-04-25T12:32:37.205Z","avatar_url":"https://github.com/k178412.png","language":"TSQL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SQL Data Warehouse Project\nA hands-on data warehouse project using SQL Server, covering ETL processes, and data modeling.\n\n---\n\n## 🏗️ Data Architecture\nThis project follows the Medallion Data Architecture, breaking the data pipeline into three layers (Bronze, Silver, and Gold layers) to ensure clarity, maintainability, and scalability.\n\n![data architecture](https://github.com/k178412/sql-data-warehouse-project/blob/main/docs/data%20architecture.png)\n\n1. **Bronze Layer** - Stores raw data exactly as received from source systems.\n2. **Silver Layer** - Cleans and transforms data for consistency, applying standardization and normalization.\n3. **Gold Layer** - Contains business-ready data, optimized for reporting and insights.\n\n---\n\n## 🔍 Project Overview\nThis project shows the full data warehouse lifecycle, from source data ingestion to business-ready data models.  \nKey components include:\n\n1. 🧱 **Data Architecture** - Designing a structured data warehouse using Medallion Architecture.\n2. 🔄 **ETL Pipelines** - Extracting, transforming, and loading data using SQL scripts.\n3. 🧮 **Data Modeling** - Creating fact and dimension tables for optimized querying and analytics.\n\n---\n\n## 📂 Project Files\n\n1. [**Datasets/**](https://github.com/k178412/sql-data-warehouse-project/tree/main/datasets) - Source CRM and ERP data stored as CSV files, used for ingestion into the warehouse.\n2. [**Docs/**](https://github.com/k178412/sql-data-warehouse-project/tree/main/docs) - Diagrams (created in Draw.io) for architecture, data flow, and data modeling.\n3. [**Scripts/**](https://github.com/k178412/sql-data-warehouse-project/tree/main/scripts) - SQL scripts for database setup, table creation, ETL processes, and transformations.\n\n---\n\n## 📊 Diagrams\n\n1. [**Data Architecture**](https://github.com/k178412/sql-data-warehouse-project/blob/main/docs/data%20architecture.png) - Defines the structural flow of the data warehouse.\n2. [**Data Flow**](https://github.com/k178412/sql-data-warehouse-project/blob/main/docs/data%20flow.png) - Illustrates the journey from raw to refined data.\n   ![](https://github.com/k178412/sql-data-warehouse-project/blob/main/docs/data%20flow.png)\n4. [**Data Integration**](https://github.com/k178412/sql-data-warehouse-project/blob/main/docs/data%20integration.png) - Highlights how different source systems connect.\n5. [**Data Model**](https://github.com/k178412/sql-data-warehouse-project/blob/main/docs/data%20model.png) - Represents logical schema for fact and dimension tables.\n6. [**ETL Pipeline**](https://github.com/k178412/sql-data-warehouse-project/blob/main/docs/etl.png) - Shows different extraction, transformation, and loading processes.\n\n---\n\n## 🛠️ Tools \u0026 Technologies\n\n1. **SQL Server** - Core database platform for data storage and transformation.\n2. **Notion** - For planning and tracking project progress.\n3. **Draw.io** - Used to design diagrams and workflows.\n4. **Git** - To manage version control and repository tracking.\n\n---\n\n## 📌 Project Tracking\n\nYou can view the detailed plan, and progress here:\n\n**Notion Project Link**: [Data Warehouse Project](https://glimmer-ricotta-2b2.notion.site/Data-Warehouse-Project-1ee49f37ed3480e1812dc7b441fb2b11?pvs=4)\n\n---\n\n## 🔒 License\nThis project is licensed under MIT License.\n\n---\n\n## 🤝 Contributing\nContributions, issues, and feature requests are welcome!\n\n---\n\n⭐️ If you find this project useful, please consider giving it a star!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk178412%2Fsql-data-warehouse-project","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fk178412%2Fsql-data-warehouse-project","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk178412%2Fsql-data-warehouse-project/lists"}