https://github.com/databrickslabs/dqx

Databricks framework to validate Data Quality of pySpark DataFrames
https://github.com/databrickslabs/dqx

data-profiling data-quality data-quality-checks data-quality-monitoring databricks dlt spark spark-streaming

Last synced: 3 months ago
JSON representation

Databricks framework to validate Data Quality of pySpark DataFrames

• Host: GitHub
• URL: https://github.com/databrickslabs/dqx
• Owner: databrickslabs
• License: other
• Created: 2024-04-23T18:28:43.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2025-03-28T12:18:15.000Z (3 months ago)
• Last Synced: 2025-04-01T10:06:30.030Z (3 months ago)
• Topics: data-profiling, data-quality, data-quality-checks, data-quality-monitoring, databricks, dlt, spark, spark-streaming
• Language: Python
• Homepage: https://databrickslabs.github.io/dqx
• Size: 1.88 MB
• Stars: 243
• Watchers: 7
• Forks: 34
• Open Issues: 27
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE
  • Codeowners: CODEOWNERS

Awesome Lists containing this project

README

        
DQX by Databricks Labs

===



    

        

    



Simplified Data Quality checking at Scale for PySpark Workloads on streaming and standard DataFrames.

[![build](https://github.com/databrickslabs/dqx/actions/workflows/push.yml/badge.svg)](https://github.com/databrickslabs/dqx/actions/workflows/push.yml) 

[![codecov](https://codecov.io/github/databrickslabs/dqx/graph/badge.svg)](https://codecov.io/github/databrickslabs/dqx) 

![linesofcode](https://aschey.tech/tokei/github/databrickslabs/dqx?category=code)

[![PyPI](https://img.shields.io/pypi/v/databricks-labs-dqx?label=pypi%20package&cacheSeconds=3600)](https://pypi.org/project/databricks-labs-dqx/) 

![PyPI - Downloads](https://img.shields.io/pypi/dm/databricks-labs-dqx?cacheSeconds=3600)

# Documentation

The complete documentation is available at: [https://databrickslabs.github.io/dqx/](https://databrickslabs.github.io/dqx/)

# Contribution

Please see the contribution guidance [here](https://databrickslabs.github.io/dqx/docs/dev/contributing/) on how to contribute to the project (build, test, and submit a PR).

# Project Support

Please note that this project is provided for your exploration only and is not 

formally supported by Databricks with Service Level Agreements (SLAs). They are 

provided AS-IS, and we do not make any guarantees. Please do not 

submit a support ticket relating to any issues arising from the use of this project.

Any issues discovered through the use of this project should be filed as GitHub 

[Issues on this repository](https://github.com/databrickslabs/dqx/issues). 

They will be reviewed as time permits, but no formal SLAs for support exist.