Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/great-expectations/great_expectations
Always know what to expect from your data.
https://github.com/great-expectations/great_expectations
cleandata data-engineering data-profilers data-profiling data-quality data-science data-unit-tests datacleaner datacleaning dataquality dataunittest eda exploratory-analysis exploratory-data-analysis exploratorydataanalysis mlops pipeline pipeline-debt pipeline-testing pipeline-tests
Last synced: 11 days ago
JSON representation
Always know what to expect from your data.
- Host: GitHub
- URL: https://github.com/great-expectations/great_expectations
- Owner: great-expectations
- License: apache-2.0
- Created: 2017-09-11T00:18:46.000Z (about 7 years ago)
- Default Branch: develop
- Last Pushed: 2024-04-12T20:28:09.000Z (7 months ago)
- Last Synced: 2024-04-13T21:52:06.232Z (7 months ago)
- Topics: cleandata, data-engineering, data-profilers, data-profiling, data-quality, data-science, data-unit-tests, datacleaner, datacleaning, dataquality, dataunittest, eda, exploratory-analysis, exploratory-data-analysis, exploratorydataanalysis, mlops, pipeline, pipeline-debt, pipeline-testing, pipeline-tests
- Language: Python
- Homepage: https://docs.greatexpectations.io/
- Size: 189 MB
- Stars: 9,420
- Watchers: 81
- Forks: 1,464
- Open Issues: 189
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING_CODE.md
- License: LICENSE
- Citation: CITATION.cff
- Codeowners: CODEOWNERS
Awesome Lists containing this project
- best-of-python - GitHub - 11% open · ⏱️ 06.06.2024): (Data Pipelines & Streaming)
- awesome-llmops - Great Expectations - expectations/great_expectations.svg?style=flat-square) | (Security / Observability)
- awesome-python-machine-learning - Great Expectations - Great Expectations is a framework that helps teams save time and promote analytic integrity with a new twist on automated testing: pipeline tests. (Uncategorized / Uncategorized)
- awesome-data-quality - great-expectations - tool for data testing, documentation, and profiling. (Table of Contents / Frameworks and Libraries)
- awesome-starred - great-expectations/great_expectations - Always know what to expect from your data. (data-science)
- awesome-list - Great Expectations - Helps data teams eliminate pipeline debt, through data testing, documentation, and profiling. (Data Management & Processing / Database & Cloud Management)
- StarryDivineSky - great-expectations/great_expectations
- awesome-python-machine-learning-resources - GitHub - 12% open · ⏱️ 26.08.2022): (数据管道和流处理)
- project-awesome - great-expectations/great_expectations - Always know what to expect from your data. (Python)
- jimsghstars - great-expectations/great_expectations - Always know what to expect from your data. (Python)
README
[](https://pypi.python.org/pypi/great_expectations)
[](https://pypi.org/project/great-expectations/#history)
[](https://pypistats.org/packages/great-expectations)
[](https://dev.azure.com/great-expectations/great_expectations/_build/latest?definitionId=1&branchName=develop)
[](https://results.pre-commit.ci/latest/github/great-expectations/great_expectations/develop)
[](https://codecov.io/gh/great-expectations/great_expectations)
[](https://doi.org/10.5281/zenodo.5683574)
[](https://twitter.com/expectgreatdata)
[](https://greatexpectations.io/slack)
[](https://github.com/great-expectations/great_expectations/graphs/contributors)
[](https://github.com/charliermarsh/ruff)
## About GX Core
GX Core is the engine of the GX platform. It combines the collective wisdom of thousands of community members with a proven track record in data quality deployments worldwide, wrapped into a super-simple package for data teams.
Its powerful technical tools start with Expectations: expressive and extensible unit tests for your data. Expectations foster collaboration by giving teams a common language to express data quality tests in an intuitive way. You can automatically generate documentation for each set of validation results, making it easy for everyone to stay on the same page. This not only simplifies your data quality processes, but helps preserve your organization’s institutional knowledge about its data.
Learn more about how data teams are using GX Core in our featured [case studies](https://greatexpectations.io/case-studies/).
## Integration support policy
GX Core supports Python `3.9` through `3.12`.
Experimental support for Python `3.13` and later can be enabled by setting a `GX_PYTHON_EXPERIMENTAL` environment variable when installing `great_expectations`.
For data sources and other integrations that GX supports, see [GX integration support policy](https://docs.greatexpectations.io/docs/application_integration_support) for additional information.
## Get started
GX recommends deploying GX Core within a virtual environment. For more information about getting started with GX Core, see [Introduction to GX Core](https://docs.greatexpectations.io/docs/core/introduction/).
1. Run the following command in an empty base directory inside a Python virtual environment to install GX Core:
```bash title="Terminal input"
pip install great_expectations
```
2. Run the following command to import the `great_expectations module` and create a Data Context:
```python
import great_expectations as gx
context = gx.get_context()
```
## Get support from GX and the community
They are listed in the order in which GX is prioritizing the support issues:
1. Issues and PRs in the [GX GitHub repository](https://github.com/great-expectations)
2. Questions posted to the [GX Core Discourse forum](https://discourse.greatexpectations.io/c/oss-support/11)
3. Questions posted to the [GX Slack community channel](https://greatexpectationstalk.slack.com/archives/CUTCNHN82)
## Contribute
We deeply value the contributions of our community. We're now accepting PRs for bug fixes.
To ensure the long-term quality of the GX Core codebase, we're not yet ready to accept feature contributions to the parts of the codebase that don't have clear APIs for extensions. We're actively working to increase the surface area for contributions. Thank you being a crucial part of GX's data quality platform!
### Levels of contribution readiness
🟢 Ready. Have a clear and public API for extensions.
🟡 Partially ready. Case-by-case.
🔴 Not ready. Will accept contributions that fix existing bugs or workflows.
| GX Component | Readiness | Notes |
| -------------------- | ------------------ | ----- |
| Action | 🟢 Ready | |
| CredentialStore | 🟢 Ready | |
| BatchDefinition | 🟡 Partially ready | Formerly known as splitters |
| DataSource | 🔴 Not ready | Includes MetricProvider and ExecutionEngine |
| DataContext | 🔴 Not ready | Also known as Configuration Stores |
| DataAsset | 🔴 Not ready | |
| Expectation | 🔴 Not ready | |
| ValidationDefinition | 🔴 Not ready | |
| Checkpoint | 🔴 Not ready | |
| CustomExpectations | 🔴 Not ready | |
| Data Docs | 🔴 Not ready | Also known as Renderers |
## Code of conduct
Everyone interacting in GX Core project codebases, Discourse forums, Slack channels, and email communications is expected to adhere to the [GX Community Code of Conduct](https://discourse.greatexpectations.io/t/gx-community-code-of-conduct/1199).