Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

awesome-dbt

A curated list of awesome dbt resources
https://github.com/Hiflylabs/awesome-dbt

Last synced: 16 days ago
JSON representation

CI/CD

Community

Data Quality

  • BigQuery Stale data detection - Stale data detection with dbt and BigQuery dataset metadata.
  • Environment-dependent Unit Testing in dbt - Guide on how to run unit tests in dbt dynamically.
  • How do you test your data - Suggestions on testing your data powered by the community.
  • How to unit test sql transforms in dbt - Unit test using source defer and generic custom tests.
  • dq-tools - Make simple storing test results and visualisation of these in a BI dashboard leveraging 6 Data Quality KPIs.
  • Elementary - A dbt package that provides data anomaly detection as dbt tests.
  • dbt-expectations - Port between dbt and great_expectations to extend out-of-the-box tests.
  • DataKitchen Open Source Data Observability - Data breaks. Servers break. dbt and other tools break. Observability and alerting across and down your data estate. Save time with simple, fast data quality test generation and execution.
  • re_data - A dbt package for montioring metrics and detect anomalies.
  • Scherlok - Zero-config data quality CLI that complements dbt test with auto-detected anomalies (volume, schema drift, freshness, distribution, cardinality) on every materialized model after dbt run.
  • Misata - Generate known-answer seed and test data for dbt models: declare the expected aggregates (revenue curves, rates, rollups) and assert your models return them exactly.
  • data-contract-validator - Fails the pull request when a model stops producing what a downstream consumer expects. Reads columns from catalog.json/manifest.json (falling back to a sqlglot parse of the SQL) and compares them against reverse-ETL destinations such as HubSpot CRM and against FastAPI/Pydantic services, in CI or pre-commit.
  • Aegis DQ - Agentic data quality framework that runs YAML rules against your warehouse (DuckDB, BigQuery, Snowflake, Databricks, Athena, Postgres), then uses LLMs to diagnose failures, trace root causes, and propose SQL fixes. Pairs naturally with dbt models as the validation layer after dbt run.

Get Started

How To

Integrations

  • Auto Alert - diqu - Automate and streamline the alerting/ notification process for dbt test results using this versatile CLI companion tool. Receive detailed alerts & test metadata seamlessly on various platforms, promoting improved collaboration on dbt project issues 🐞🚀.
  • Tabula - Tabula is an end-to-end automation platform for data management tasks.
  • Grai - Expose warehouse dbt tests in CI to upstream data consumers so production changes never break the warehouse.
  • Cube - APIs, Caching, and Access Control on top of dbt Metrics.
  • FlexIt Analytics - Business Intelligence platform with deep dbt Cloud and CLI integration.
  • Raycast dbt Jobs - Raycast integration to monitor dbt Cloud Jobs.
  • Metaplane - Data Observaibility layer on top of your dbt + BI project.
  • Dbt + Machine Learning: What makes a great baton pass? - Landscape of ML utilities around dbt.
  • Supported Adapters - Offically supported database adapters.
  • Superset - Open source visualization layer for your Modern Data Stack.
  • dbt-streamdeck - Stream Deck plugin enables you to view the status of models and jobs as actions in your Stream Deck.
  • modal-dbt - This repo gives some code to run dbt jobs/actions using modal which is a serverless application framework.
  • Raycast dbt Metadata - Queries the dbt Cloud API to return some useful information about your models (number of tests, time they took to run etc…).
  • Lightdash - Open source Looker alternative with deep dbt integration.
  • fal - Add multi-language support (Python) to your dbt project.
  • prefect-dbt - Collection of Prefect integrations for working with dbt with your Prefect flows.
  • Datafold - Gives a quick print out summary of changes so you can move fast and (not) break stuff!
  • dbt-cli-mcp - MCP for dbt CLI.
  • dbt MCP Server - MCP tools to interact with dbt.
  • Soda - Integration of Soda's data observability platform and dbt.
  • dbt-agent-skills - A curated collection of agent skills, built and maintained by dbt Labs, to help AI coding agents work more effectively with dbt.
  • ingestr - CLI tool to copy data from any source to any destination with a single command. Load data into your warehouse before dbt transforms it. Supports 50+ sources including Postgres, MongoDB, Salesforce, Shopify.
  • Privacy Dynamics - Anonymize data in your dbt project.
  • Dagster and dbt: Better Together - Getting started with the dagster-dbt library.
  • Acryl DataHub - A unified data catalog, governance, and observability platform. Use it to view models, docs, test results, and column-level lineage across your dbt projects and downstream dashboards.

Orchestration

Packages

  • dbt-incremental-stream - A dbt package for Snowflake Streams.
  • dbt-dag-monitoring - A dbt package for monitoring airflow DAGs and tasks.
  • dbt-data-diff - Data-diff solution for dbt-ers with Snowflake.
  • dbt-tags - Tag-based masking policies management in Snowflake.
  • dbt-testgen - Generate dbt tests based on sample data.
  • dbt_otel_export - Takes dbt runs and turns them into OpenTelemetry traces.
  • dbt-assertions - Package to assert rows in-line with dbt macros.
  • dbt-ibis - Write your dbt models using Ibis, the portable Python dataframe library.
  • dbt-timescaledb - The TimescaleDB adapter plugin for dbt.
  • dbt-fake - Daily updated fake data for dbt learning and projects.
  • dbt_cloud_run_cost - Package to calculate dbt Cloud usage-based cost.
  • dbt-reconfigured - A dbt package containing reconfigured macros.
  • dbt-census-utils - A collection of dbt macros for working with Census data.
  • dq-vault - Data Quality Observation of Data Vault layer.
  • dbt-translate - Translate numbers into words.
  • dbt-excel - A dbt adapter for working with Excel.
  • dbt_linreg - Linear regression in SQL using dbt.
  • dbt-snowflake-query-tags - Automatically tag dbt-issued queries with informative metadata.
  • snowflake-resource-monitoring - Yet another package to monitor Snowflake usage.
  • usagedata - Provides insights on the database/table level usage informations from Snowflake.
  • dbt_ml - Package for dbt that allows users to train, audit and use BigQuery ML models.
  • ddbt - This repo represents my attempt to build a fast version of DBT which gets very slow on large projects (3000+ data models). This project attempts to be a direct drop in replacement for DBT at the command line.
  • dbt-snowflake-monitoring - A dbt package to help you monitor Snowflake performance and costs.
  • datavault4dbt - Macros for staging and creation of all DataVault-Entities you need, to build your own DataVault2.0 solution.
  • DDO - Perform DataOps & administrative CI/CD on your data warehouse.
  • dbt-yaml-check - Checks that columns defined in YAML also exist in SQL.
  • data-diff - A command-line tool and Python library to efficiently diff rows across two different databases.
  • dbt-project-evaluator - This package highlights areas of a dbt project that are misaligned with dbt Labs' best practices.
  • dbt_constraints - Generate database constraints based on the tests in a dbt project.
  • dbt-privacy - Macros to make it easier to protect your customers' data.
  • dbt-fivetran-utils - General macros and helpers.
  • dbt_metrics - Macros to support secondary calculations and generate business metrics.
  • dbt-metabase - Model synchronization from dbt to Metabase.
  • dbt-ml-preprocessing - A SQL port of python's scikit-learn preprocessing module, provided as cross-database dbt macros.
  • dbt-external-tables - Macros to stage your external sources.
  • dbt-feature-store - Macros to build a feature store right within your dbt project.
  • dbt-codegen - Macros that generate dbt code, and log it to the command line.
  • dbt-init - Create a project and populate as much of the dbt project as possible.
  • dbt-artifacts - This package builds a mart of tables from dbt artifacts loaded into a table.
  • dbt-erdiagram-generator - This packages generate ERD diagrams from a dbt project.
  • dbt2looker - Generate Looker views for dbt models.
  • dbt-coverage - Checks dbt docs and tests coverage.
  • dbt-meta-testing - Yet another coverage testing.
  • dbt-superset-lineage - Push and pull metadata between dbt to Superset.
  • dbt-invoke - CLI for creating, updating, and deleting dbt property files.
  • dbt-unit-testing - Package which contains macros to support unit testing.
  • dbt-snow-mask - A dbt package for Snowflake Dynamic Data Masking.
  • dbt-fabric - A dbt adapter for working with Microsoft Fabric Data Warehouses.
  • dbt-fabricspark - A code package from Microsoft for enabling dbt to work with Synapse Spark in Microsoft Fabric.
  • dbtvault - Package for generating and executing ETL for Data Vault 2.0.
  • dbt-ml-inline-preprocessing - Feature Engineering in dbt.
  • dbt_semantic_view/
  • dbt_audit_helper - Macros for data audits that compare columns values and schemas between tables.
  • Terraform-dbt Cloud Module - IAC in dbt Cloud via Terraform.
  • dbt-date - Date logic and calendar functionality.
  • dbt_diving - A free to use dbt package with helper macros
  • dbt-unirate - Currency exchange rate macros and models for dbt, powered by the UniRate API.
  • dbt-fabricsparknb - Dbt Fabric Spark Notebook Generator by Insight based on/forked form dbt-fabricspark by Microsoft.
  • dbt-coves - CLI tool for generating a scaffold for your dbt project.
  • dbt-profiler - Data profiling and doc block generator.
  • dbt_utils - General macros library. A must have.

Sample Projects