https://github.com/datacody/dbt-jaffle-shop
A hands-on project built to deepen understanding of dbt modeling, testing, and documentation. Based on the Jaffle Shop dataset, the project showcases best practices in transforming and validating source data for business analytics using the modern data stack.
https://github.com/datacody/dbt-jaffle-shop
analytics bigquery data-eng data-modeling dbt etl-pipeline sql transformation
Last synced: 11 months ago
JSON representation
A hands-on project built to deepen understanding of dbt modeling, testing, and documentation. Based on the Jaffle Shop dataset, the project showcases best practices in transforming and validating source data for business analytics using the modern data stack.
- Host: GitHub
- URL: https://github.com/datacody/dbt-jaffle-shop
- Owner: DataCody
- License: mit
- Created: 2025-05-14T01:59:22.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-05-22T02:17:30.000Z (about 1 year ago)
- Last Synced: 2025-07-02T08:43:29.645Z (12 months ago)
- Topics: analytics, bigquery, data-eng, data-modeling, dbt, etl-pipeline, sql, transformation
- Homepage:
- Size: 724 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## 🥪 dbt-jaffle-shop
A Modern Data Stack Project for End-to-End Analytics Engineering
This repository is a customized implementation of the [dbt-labs/jaffle-shop](https://github.com/dbt-labs/jaffle-shop) project, tailored to demonstrate proficiency in data modeling, testing, documentation, and orchestration using dbt. It serves as a comprehensive example of building a robust analytics pipeline from raw data ingestion to curated data marts.
## 🚀 Project Overview
- **Source Data**: Simulated e-commerce datasets representing customers, orders, and payments.
- **TransformationLayers**:
- **Staging Models**: Clean and standardize raw data.
- **Intermediate Models**: Join and enrich data across multiple sources.
- **Mart Models**: Provide business-ready datasets for analytics and reporting.
- **Testing**: Implemented data quality tests including unique, not_null, and referential integrity checks.
- **Documentation**: Auto-generated documentation with detailed model and column descriptions.
- **Orchestration**: Configured dbt Cloud jobs for scheduled runs and testing.
## 🧰 Tech Stack
- **dbt Core**: SQL-based data transformation framework.
- **dbt Cloud**: Hosted environment for development and job orchestration.
- **Data Warehouse**: BigQuery.
- **Version Control**: GitHub for source code management.
- **CI/CD**: [Optional—mention if integrated].
## 🗺️ Project Structure
dbt-jaffle-shop/
├── analyses
├── docs/
├── models/
│ ├── staging/
│ ├── intermediate/
│ └── marts/
├── seeds/
├── tests/
├── macros/
├── snapshots/
├── dbt_project.yml
├── LICENSE
└── README.md
## 🧪 Testing & Quality Assurance
Implemented comprehensive testing strategies to ensure data reliability:
- **Schema Tests**: Enforced constraints like unique and not_null.
- **Custom Tests**: Developed bespoke tests for business logic validation.
- **Data Freshness**: Configured freshness checks on source data.
## 📊 Documentation & Lineage
- Auto-Generated Docs: Utilized dbt docs generate for creating interactive documentation.
- Data Lineage: Visualized model dependencies and data flow.
### 📸 Documentation screenshot
### 📸 DAG visualization
## 🗓️ Scheduling & Orchestration
Configured dbt Cloud jobs with the following characteristics:
- **Environment**: Production
- **Schedule**: Daily runs at 9 AM AEST
- **Commands**:
- `dbt seed`
- `dbt run`
- `dbt test`
## 🧩 Key Features
- Modular and scalable model architecture.
- Adherence to dbt best practices and naming conventions.
- Comprehensive testing and documentation.
- Automated workflows for continuous integration and deployment.
## 📈 Future Enhancements
- Integration with BI tools like Looker or Tableau.
- Implementation of snapshots for slowly changing dimensions.
- Expansion of test coverage with more complex scenarios.
## 📄 License
This project is licensed under the [MIT License](LICENSE).