{"id":42307978,"url":"https://github.com/datachecks/dcs-core","last_synced_at":"2026-03-03T01:01:27.510Z","repository":{"id":182763280,"uuid":"666821713","full_name":"datachecks/dcs-core","owner":"datachecks","description":"Open Source Data Quality Monitoring.","archived":false,"fork":false,"pushed_at":"2026-02-09T22:18:28.000Z","size":5233,"stargazers_count":168,"open_issues_count":24,"forks_count":23,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-02-23T22:17:45.791Z","etag":null,"topics":["data-engineering","data-governance","data-observability","data-ops","data-quality-monitor","data-quality-monitoring","data-validation","database","dataops","dataquality","elasticsearch","etl","metrics","mlops","monitoring","mysql","postgres","postgresql","python","sql"],"latest_commit_sha":null,"homepage":"https://docs.datachecks.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/datachecks.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"docs/support/contact.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-07-15T17:29:44.000Z","updated_at":"2026-01-29T11:22:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"ab256892-fcaf-44bb-bfc9-6306a1d2ed23","html_url":"https://github.com/datachecks/dcs-core","commit_stats":null,"previous_names":["waterdipai/datachecks","datachecks/datachecks","datachecks/dcs-core"],"tags_count":20,"template":false,"template_full_name":null,"purl":"pkg:github/datachecks/dcs-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datachecks%2Fdcs-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datachecks%2Fdcs-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datachecks%2Fdcs-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datachecks%2Fdcs-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/datachecks","download_url":"https://codeload.github.com/datachecks/dcs-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/datachecks%2Fdcs-core/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30028228,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-03T00:31:48.536Z","status":"ssl_error","status_checked_at":"2026-03-03T00:30:56.176Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-engineering","data-governance","data-observability","data-ops","data-quality-monitor","data-quality-monitoring","data-validation","database","dataops","dataquality","elasticsearch","etl","metrics","mlops","monitoring","mysql","postgres","postgresql","python","sql"],"created_at":"2026-01-27T11:12:46.288Z","updated_at":"2026-03-03T01:01:27.384Z","avatar_url":"https://github.com/datachecks.png","language":"Python","readme":"\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"Logo\" src=\"https://raw.githubusercontent.com/waterdipai/datachecks/main/docs/assets/datachecks_banner_logo.svg\" width=\"1512\"\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\u003cb\u003eOpen Source Data Quality Monitoring.\u003c/b\u003e\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003cimg align=\"center\" alt=\"License\" src=\"https://img.shields.io/badge/License-Apache%202.0-blue.svg\"/\u003e\n    \u003cimg align=\"center\" src=\"https://img.shields.io/pypi/pyversions/datachecks\"/\u003e\n    \u003cimg align=\"center\" alt=\"Versions\" src=\"https://img.shields.io/pypi/v/datachecks\"/\u003e\n    \u003cimg align=\"center\" alt=\"coverage\" src=\"https://static.pepy.tech/personalized-badge/datachecks?period=total\u0026units=international_system\u0026left_color=black\u0026right_color=green\u0026left_text=Downloads\"/\u003e\n    \u003cimg align=\"center\" alt=\"coverage\" src=\"https://codecov.io/gh/waterdipai/datachecks/branch/main/graph/badge.svg?token=cn6lkDRXpl\"\u003e\n    \u003cimg align=\"center\" alt=\"Status\" src=\"https://github.com/waterdipai/datachecks/actions/workflows/ci.yml/badge.svg?branch=main\"/\u003e\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n⭐️ If you like it, star the repo \u003ca href=\"https://github.com/waterdipai/waterdip/stargazers\"\u003e\u003c/a\u003e ⭐\n\n\u003ch3\u003e|\n\u003ca href=\"https://docs.datachecks.io/\"\u003eDocumentations\u003c/a\u003e\n|\n\u003ca href=\"https://join.slack.com/t/datachecks/shared_invite/zt-1zqsigy4i-s5aadIh2mjhdpVWU0PstPg\"\u003eSlack Community\u003c/a\u003e\n|\n\u003c/h3\u003e\n\u003c/div\u003e\n\n## Why Data Monitoring?\n\nAPM (Application Performance Monitoring) tools are used to monitor the performance of applications. APM tools are mandatory part of dev stack. Without AMP tools, it is very difficult to monitor the performance of applications.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"why_data_observability\" src=\"https://raw.githubusercontent.com/datachecks/dcs-core/main/docs/assets/datachecks_why_data_observability.svg\" width=\"800\"\u003e\n\u003c/p\u003e\n\nBut for Data products regular APM tools are not enough. We need a new kind of tools that can monitor the performance of Data applications.\nData monitoring tools are used to monitor the data quality of databases and data pipelines. It identifies potential issues, including in the databases and data pipelines. It helps to identify the root cause of the data quality issues and helps to improve the data quality.\n\n## What is `datachecks`?\n\nDatachecks is an open-source data monitoring tool that helps to monitor the data quality of databases and data pipelines.\nIt identifies potential issues, including in the databases and data pipelines. It helps to identify the root cause of the data quality issues and helps to improve the data quality.\n\nDatachecks can generate several reliability, uniqueness, completeness metrics from several data sources\n\n### Reports: Data Quality Visualisation\n\nYou can generate with just one command. It generates a beautiful data quality report with all the metrics.\nThis html report can be shared with the team.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"why_data_observability\" src=\"https://raw.githubusercontent.com/datachecks/dcs-core/main/docs/assets/datachecks_dashboard.png\" width=\"800\"\u003e\n\u003c/p\u003e\n\n### CLI: Data Quality Visualisation in Bash\n\nData quality report can be generated in the terminal. It is very useful for debugging. All it takes is one command.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"why_data_observability\" src=\"https://raw.githubusercontent.com/datachecks/dcs-core/main/docs/assets/datachecks_cli_output.png\" width=\"800\"\u003e\n\u003c/p\u003e\n\n## Getting Started\n\nInstall `datachecks` with the command that is specific to the database.\n\n### Install Datachecks\n\nTo install all datachecks dependencies, use the below command.\n\n```shell\npip install dcs-core -U\n```\n\n### Create the config file\n\nWith a simple config file, you can generate data quality reports for your data sources. Below is the sample config example.\nFor more details, please visit the [config guide](https://docs.datachecks.io/dcs-oss/config/validation-config)\n\n\n### Run from CLI\n\n**Generate Report in Terminal**\n\n```shell\ndcs-core inspect -C config.yaml\n```\n\n**Generate HTML Report**\n\n```shell\ndcs-core inspect -C config.yaml  --html-report\n```\n\nPlease visit the [Quick Start Guide](https://docs.datachecks.io/dcs-oss/introduction/getting-started)\n\n## Supported Data Sources\n\nDatachecks supports sql and search data sources. Below are the list of supported data sources.\n\n| Data Source                                                                           | Type                   | Supported  |\n|---------------------------------------------------------------------------------------|------------------------|------------|\n| [Postgres](https://docs.datachecks.io/dcs-oss/integrations/transactional/postgres)    | Transactional Database | :thumbsup: |\n| [MySql](https://docs.datachecks.io/dcs-oss/integrations/transactional/mysql)          | Transactional Database | :thumbsup: |\n| [MS SQL Server](https://docs.datachecks.io/dcs-oss/integrations/transactional/mssql)  | Transactional Database | :thumbsup: |\n| [Oracle](https://docs.datachecks.io/dcs-oss/integrations/transactional/oracle)        | Transactional Database | :thumbsup: |\n| [DB2](https://docs.datachecks.io/dcs-oss/integrations/transactional/db2)              | Transactional Database | :thumbsup: |\n| [SAP Sybase](https://docs.datachecks.io/dcs-oss/integrations/warehouse/sybase)    | Transactional Database         | :thumbsup: |\n| [OpenSearch](https://docs.datachecks.io/dcs-oss/integrations/search/opensearch)       | Search Engine          | :thumbsup: |\n| [Elasticsearch](https://docs.datachecks.io/dcs-oss/integrations/search/elasticsearch) | Search Engine          | :thumbsup: |\n| [GCP BigQuery](https://docs.datachecks.io/dcs-oss/integrations/warehouse/bigquery)    | Data Warehouse         | :thumbsup: |\n| [DataBricks](https://docs.datachecks.io/dcs-oss/integrations/warehouse/databricks)    | Data Warehouse         | :thumbsup: |\n| [Snowflake](https://docs.datachecks.io/dcs-oss/integrations/warehouse/snowflake)      | Data Warehouse         | :thumbsup: |\n| [AWS RedShift](https://docs.datachecks.io/dcs-oss/integrations/warehouse/redshift)    | Data Warehouse         | :thumbsup: |\n\n## Metric Types\n\n| Validation Funtions                                                                      | Description                                                                                                        |\n|------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|\n| **[Reliability](https://docs.datachecks.io/dcs-oss/validations/reliability)**                   | Reliability functions detect whether tables/indices/collections are updating with timely data                      |\n| **[Numeric Distribution](https://docs.datachecks.io/dcs-oss/validations/distribution)** | Numeric Distribution functions detect changes in the numeric distributions i.e. of values, variance, skew and more |\n| **[Uniqueness](https://docs.datachecks.io/dcs-oss/validations/uniqueness)**                     | Uniqueness functions detect when data constraints are breached like duplicates, number of distinct values etc      |\n| **[Completeness](https://docs.datachecks.io/dcs-oss/validations/completeness)**                 | Completeness functions detect when there are missing values in datasets i.e. Null, empty value                     |\n| **[Validity](https://docs.datachecks.io/dcs-oss/validations/pattern-matching)**                         | Validity functions detect whether data is formatted correctly and represents a valid value                         |\n\n## Overview\n\n\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"datacheck_architecture\" src=\"https://raw.githubusercontent.com/datachecks/dcs-core/main/docs/assets/data_check_architecture.svg\" width=\"800\"\u003e\n\u003c/p\u003e\n\n## What Datacheck does not do?\n\n\u003cp align=\"middle\"\u003e\n  \u003cimg alt=\"\" src=\"https://raw.githubusercontent.com/datachecks/dcs-core/main/docs/assets/datachecks_does_not_do.svg\" width=\"800\"/\u003e\n\u003c/p\u003e\n\n## Community \u0026 Support\n\nFor additional information and help, you can use one of these channels:\n\n- [Slack](https://join.slack.com/t/datachecks/shared_invite/zt-1zqsigy4i-s5aadIh2mjhdpVWU0PstPg) \\(Live chat with the team, support, discussions, etc.\\)\n- [GitHub issues](https://github.com/waterdipai/datachecks/issues) \\(Bug reports, feature requests)\n\n## **Contributions**\n\n:raised_hands: We greatly appreciate contributions - be it a bug fix, new feature, or documentation!\n\nCheck out the [contributions guide](https://github.com/waterdipai/datachecks/blob/main/CONTRIBUTING.md) and [open issues](https://github.com/waterdipai/datachecks/issues).\n\n**Datachecks contributors: :blue_heart:**\n\n\u003ca href=\"https://github.com/subhankarb\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/2178361?v=4\" width=\"50\" height=\"50\" alt=\"\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/niyasrad\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/84234554?v=4\" width=\"50\" height=\"50\" alt=\"\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/WeryZebra-Yue\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/75676675?v=4\" width=\"50\" height=\"50\" alt=\"\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/gaurav-wdi\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/82873511?v=4\" width=\"50\" height=\"50\" alt=\"\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/PULAK0717\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/101057457?v=4\" width=\"50\" height=\"50\" alt=\"\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/fabriciodadosbr\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/96063978?v=4\" width=\"50\" height=\"50\" alt=\"\"/\u003e\u003c/a\u003e\n\n## Telemetry\n\n[Usage Analytics \u0026 Data Privacy](https://github.com/waterdipai/datachecks/blob/main/docs/support/usage_analytics.md)\n\n## License\n\nThis project is licensed under the terms of the [APACHE 2 License](https://github.com/waterdipai/datachecks/blob/main/LICENSE).\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatachecks%2Fdcs-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatachecks%2Fdcs-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatachecks%2Fdcs-core/lists"}