{"id":49319722,"url":"https://github.com/gblack98/dbt-advanced-patterns","last_synced_at":"2026-04-26T17:02:32.123Z","repository":{"id":350560785,"uuid":"1197777400","full_name":"Gblack98/dbt-advanced-patterns","owner":"Gblack98","description":"Production-grade dbt patterns: incremental models, SCD Type 2, custom macros, and slim CI for fintech data platforms","archived":false,"fork":false,"pushed_at":"2026-04-10T22:17:41.000Z","size":100979,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-11T00:19:35.754Z","etag":null,"topics":["data-engineering","data-warehouse","dbt","fintech","incremental-models","scd","sql"],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Gblack98.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-31T21:58:21.000Z","updated_at":"2026-04-10T22:17:50.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Gblack98/dbt-advanced-patterns","commit_stats":null,"previous_names":["gblack98/dbt-advanced-patterns"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Gblack98/dbt-advanced-patterns","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gblack98%2Fdbt-advanced-patterns","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gblack98%2Fdbt-advanced-patterns/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gblack98%2Fdbt-advanced-patterns/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gblack98%2Fdbt-advanced-patterns/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Gblack98","download_url":"https://codeload.github.com/Gblack98/dbt-advanced-patterns/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gblack98%2Fdbt-advanced-patterns/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32305040,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T09:34:17.070Z","status":"ssl_error","status_checked_at":"2026-04-26T09:34:00.993Z","response_time":129,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-engineering","data-warehouse","dbt","fintech","incremental-models","scd","sql"],"created_at":"2026-04-26T17:02:28.591Z","updated_at":"2026-04-26T17:02:32.097Z","avatar_url":"https://github.com/Gblack98.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# dbt Fintech — Advanced Patterns\n\nA hands-on dbt learning project using DuckDB on a real-world loan default dataset (148,670 rows).\nCovers advanced dbt patterns: incremental models, SCD Type 2, data contracts, custom macros, tests, exposures, groups \u0026 access, and slim CI.\n\n---\n\n## Stack\n\n| Tool | Version |\n|------|---------|\n| dbt-core | 1.11.7 |\n| dbt-duckdb | 1.10.1 |\n| DuckDB | local (`dev.duckdb`) |\n| Python | 3.12 |\n\n---\n\n## Dataset\n\n**Source:** [Loan Default Dataset](https://www.kaggle.com/datasets/yasserh/loan-default-dataset) (Kaggle)\n**File:** `seeds/Loan_Default.csv` — 148,670 loans from 2019\n**Key columns:** `loan_id`, `region`, `credit_score`, `loan_amount`, `is_default`\n**Regions:** south, North, central, North-East\n\n---\n\n## Project Structure\n\n```\nseeds/\n└── Loan_Default.csv              ← raw data\n\nmodels/\n├── staging/\n│   └── stg_loans.sql             ← cleaning + casting (view)\n└── marts/\n    ├── mart_loan_summary.sql     ← aggregation by region/year (table)\n    └── mart_risk_profile.sql     ← credit score bucketing (incremental)\n\nsnapshots/\n└── loan_status_snapshot.sql      ← SCD Type 2 history tracking\n\nmacros/\n├── audit_log.sql                 ← post-hook: logs row count after each run\n└── test_not_empty.sql            ← reusable generic test\n\ntests/\n├── assert_default_rate_reasonable.sql\n└── assert_loan_summary_positive_loans.sql\n```\n\n---\n\n## Concepts Covered\n\n| Concept | File(s) |\n|---------|---------|\n| Incremental model | `mart_risk_profile.sql` |\n| SCD Type 2 snapshot | `snapshots/loan_status_snapshot.sql` |\n| Data Contracts | `models/marts/schema.yml` — `contract: enforced: true` |\n| Post-hook macro | `macros/audit_log.sql` — uses `{{ this }}` to avoid DAG cycles |\n| Custom generic test | `macros/test_not_empty.sql` |\n| Singular tests | `tests/assert_*.sql` |\n| Unit tests | `schema.yml` — top-level `unit_tests:` block |\n| Exposures | `models/marts/exposures.yml` |\n| Groups \u0026 Access | `models/staging/groups.yml` |\n| Slim CI | `.github/workflows/dbt_ci.yml` |\n\n---\n\n## Setup\n\n```bash\n# Clone the repo\ngit clone git@github.com:Gblack98/dbt-advanced-patterns.git\ncd dbt-advanced-patterns\n\n# Create and activate virtualenv\npython3 -m venv .venv\nsource .venv/bin/activate\n\n# Install dependencies\npip install dbt-core==1.11.7 dbt-duckdb==1.10.1\n\n# Configure dbt profile (~/.dbt/profiles.yml)\ncat \u003e\u003e ~/.dbt/profiles.yml \u003c\u003c 'EOF'\ndbt_fintech:\n  target: dev\n  outputs:\n    dev:\n      type: duckdb\n      path: dev.duckdb\n      threads: 1\nEOF\n```\n\n---\n\n## Usage\n\n```bash\ncd dbt_fintech\n\n# Load seed data\ndbt seed\n\n# Run all models\ndbt run\n\n# Run all tests\ndbt test\n\n# Run snapshots\ndbt snapshot\n\n# Inspect exposure DAG\ndbt ls --select +exposure:loan_risk_dashboard\n\n# List models in a group\ndbt ls --select group:marts\n```\n\n---\n\n## CI/CD — Slim CI\n\nThe `.github/workflows/dbt_ci.yml` workflow implements a **Slim CI** strategy:\n\n- **First run:** full `dbt build` → saves `manifest.json` as a GitHub Actions artifact\n- **Subsequent runs:** downloads previous manifest → runs only **modified models and their dependents** (`state:modified+`)\n\n```\npush / PR → seed → slim build (state:modified+) → upload manifest\n```\n\n---\n\n## Key Results\n\n### `mart_loan_summary` — Default rate by region\n\n| Region | Default Rate |\n|--------|-------------|\n| North-East | 30.45% |\n| central | ~27% |\n| south | ~25% |\n| North | 22.51% |\n\n### `mart_risk_profile` — Default rate by credit score bucket\n\n| Bucket | Score Range | Default Rate |\n|--------|-------------|-------------|\n| Bad | \u003c 580 | 24.7% |\n| Fair | 580–669 | 24.5% |\n| Good | 670–739 | 24.2% |\n| Very Good | 740–799 | 24.5% |\n| Excellent | ≥ 800 | 25.2% |\n\n\u003e Note: similar default rates across buckets are an artifact of the synthetic Kaggle dataset.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgblack98%2Fdbt-advanced-patterns","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgblack98%2Fdbt-advanced-patterns","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgblack98%2Fdbt-advanced-patterns/lists"}