{"id":13575676,"url":"https://github.com/Renumics/awesome-open-data-centric-ai","last_synced_at":"2025-04-04T22:31:44.162Z","repository":{"id":94640231,"uuid":"607380349","full_name":"Renumics/awesome-open-data-centric-ai","owner":"Renumics","description":"Curated list of open source tooling for data-centric AI on unstructured data. ","archived":false,"fork":false,"pushed_at":"2023-11-15T11:02:12.000Z","size":586,"stargazers_count":679,"open_issues_count":1,"forks_count":36,"subscribers_count":18,"default_branch":"main","last_synced_at":"2024-05-20T12:03:30.378Z","etag":null,"topics":["active-learning","awesome-list","bias-detection","computer-vision","data-centric-ai","data-curation","data-drift","data-versioning","data-visualization","deep-learning","explainable-ai","feature-vector","machine-learning","nlp","noisy-labels","outlier-detection","robust-machine-learning","synthetic-data","uncertainty-estimation"],"latest_commit_sha":null,"homepage":"https://renumics.com","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc-by-4.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Renumics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-02-27T21:33:59.000Z","updated_at":"2024-05-12T15:45:35.000Z","dependencies_parsed_at":"2024-01-18T04:28:33.524Z","dependency_job_id":null,"html_url":"https://github.com/Renumics/awesome-open-data-centric-ai","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Renumics%2Fawesome-open-data-centric-ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Renumics%2Fawesome-open-data-centric-ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Renumics%2Fawesome-open-data-centric-ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Renumics%2Fawesome-open-data-centric-ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Renumics","download_url":"https://codeload.github.com/Renumics/awesome-open-data-centric-ai/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247260779,"owners_count":20910072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["active-learning","awesome-list","bias-detection","computer-vision","data-centric-ai","data-curation","data-drift","data-versioning","data-visualization","deep-learning","explainable-ai","feature-vector","machine-learning","nlp","noisy-labels","outlier-detection","robust-machine-learning","synthetic-data","uncertainty-estimation"],"created_at":"2024-08-01T15:01:03.205Z","updated_at":"2025-04-04T22:31:39.145Z","avatar_url":"https://github.com/Renumics.png","language":null,"funding_links":[],"categories":["Others","Other Lists","Core Machine Learning Research"],"sub_categories":["TeX Lists","Data-Centric AI and Data Curation"],"readme":"\n\u003ch1 align=\"center\"\u003eAwesome open data-centric AI\u003c/h1\u003e\n\u003cp align=\"center\"\u003eOpen source tooling for data-centric AI on unstructured data\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg src=\"https://camo.githubusercontent.com/64f8905651212a80869afbecbf0a9c52a5d1e70beab750dea40a994fa9a9f3c6/68747470733a2f2f617765736f6d652e72652f62616467652e737667\" alt=\"Awesome\" data-canonical-src=\"https://awesome.re/badge.svg\" style=\"max-width: 100%;\"\u003e\t     \n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003ca href=\"https://github.com/Renumics/awesome-open-data-centric-ai\"\u003e\u003cimg src=\"static/img/dcai_landscape_s.png\" width=\"100%\"/\u003e\u003c/a\u003e\u003c/p\u003e\n\n\n**Data-centric AI (DCAI)** is a development paradigm for ML-based solutions. The term was coined by Andrew Ng who gave the following definition:\n\n\u003e Data-centric AI is the practice of systematically engineering the data used to build AI systems. \n\nAt [Renumics](https://renumics.com), we believe DCAI is an important puzzle piece for building real-world AI systems that generate value. We like the following definition:\n\u003e Data-centric AI means to improve training datasets systematically and iteratively by leveraging information from trained ML models.\n\n**Tools that can be efficiently used in day-to-day applications** are the most important ingredient for the DCAI paradigm. This curated link collection is intended to help you discover useful open source tools for your data-centric AI workflows.\n\n## 🔎 Scope\n\nWe include useful tools that have an **open-source license** and are **actively maintained** in this collection. All tools mentioned are useful for building DCAI workflows on **unstructured data** (e.g. images, audio, video, time-series, text). \n\nWe also collect **workflow snippets** into a **data-centric AI playbook** that show how typical tasks can be solved with open source tooling.\n\nIn order to keep a useful focus and to prevent duplicate work, we exclude some topics from this list such as tooling for tabular data, dedicated labeling tools, MLOps tooling as well as research papers. Please check out the [further reading](#further-reading) section to find awesome lists for these topics.\n\n## :open_hands: Contributing\nDo you think something is missing? Please help contribute to this list by contacting us or adding a pull request.\n\n\n# 🧰 Tooling\n\n## 📒 Categories\n\n\n- [Data versioning](#data-versioning)\n- [Embeddings and pre-trained models](#embeddings-and-pre-trained-models)\n- [Visualization and interaction](#visualization-and-interaction)\n- [Outlier and noise detection](#outlier-and-noise-detection)\n- [Explainability](#explainability)\n- [Active learning](#active-learning)\n- [Uncertainty quantification](#uncertainty-quantification)\n- [Bias and fairness](#bias-and-fairness)\n- [Observability and monitoring](#observability-and-monitoring)\n- [Augmentation and synthetic data](#augmentation-and-synthetic-data)\n- [Security and robustness](#security-and-robustness)\n\n\n## Data versioning\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003cimg src=\"https://camo.githubusercontent.com/7a3bb42ff937bf609d1ce4b4626357cd7b9f0208a46c531b7df8d8636d1aa895/68747470733a2f2f7374617469632e6974657261746976652e61692f6c6f676f2f6476632e737667\" width=\"100\"/\u003e | [Data version control (DVC)](https://github.com/iterative/dvc) | Data Version Control or DVC is a command line tool and VS Code Extension to help you develop reproducible machine learning projects. | ![GitHub stars](https://img.shields.io/github/stars/iterative/dvc?style=social) | \u003ca href=\"https://github.com/iterative/dvc/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/iterative/dvc\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/activeloopai/deeplake\"\u003e\u003cimg src=\"https://camo.githubusercontent.com/d0c805affb06f5ea9ba767de06b77a04de54a7ef433fad08b2729d5e6b11112c/68747470733a2f2f692e706f7374696d672e63632f72736a63576333532f646565706c616b652d6c6f676f2e706e67\"  width=\"100\"/\u003e\u003c/a\u003e | [deeplake](https://github.com/activeloopai/deeplake) | Data Lake for Deep Learning. Build, manage, query, version, \u0026 visualize datasets.  | ![GitHub stars](https://img.shields.io/github/stars/activeloopai/deeplake?style=social) | \u003ca href=\"https://github.com/activeloopai/deeplake/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/activeloopai/deeplake\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/pachyderm/pachyderm\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/10432478?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Pachyderm](https://github.com/pachyderm/pachyderm) | Pachyderm – Automate data transformations with data versioning and lineage. | ![GitHub stars](https://img.shields.io/github/stars/pachyderm/pachyderm?style=social) | \u003ca href=\"https://github.com/pachyderm/pachyderm/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/pachyderm/pachyderm\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/delta-io/delta\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/49767398?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Delta Lake](https://github.com/delta-io/delta) | An open-source storage framework that enables building a Lakehouse architecture.  | ![GitHub stars](https://img.shields.io/github/stars/delta-io/delta?style=social) | \u003ca href=\"https://github.com/delta-io/delta/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/delta-io/delta\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/treeverse/lakeFS\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/treeverse/lakeFS/master/docs/assets/img/logo_large.png\"  width=\"100\"/\u003e\u003c/a\u003e | [lakeFS](https://github.com/treeverse/lakeFS) | lakeFS is an open-source tool that transforms your object storage into a Git-like repository. | ![GitHub stars](https://img.shields.io/github/stars/treeverse/lakeFS?style=social) | \u003ca href=\"https://github.com/treeverse/lakeFS/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/treeverse/lakeFS\" height=\"15\"/\u003e\u003c/a\u003e |\n\n## Embeddings and pre-trained models\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/towhee-io/towhee\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/87362374?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [towhee](https://github.com/towhee-io/towhee) | Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.   | ![GitHub stars](https://img.shields.io/github/stars/towhee-io/towhee?style=social) | \u003ca href=\"https://github.com/towhee-io/towhee/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/towhee-io/towhee\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/tensorflow/hub\"\u003e\u003cimg src=\"https://www.tensorflow.org/static/site-assets/images/project-logos/tensorflow-hub-logo-social.png\"  width=\"100\"/\u003e\u003c/a\u003e | [Tensorflow Hub](https://github.com/tensorflow/hub) | TensorFlow Hub is a repository of reusable assets for machine learning with TensorFlow.   | ![GitHub stars](https://img.shields.io/github/stars/tensorflow/hub?style=social) | \u003ca href=\"https://github.com/tensorflow/hub/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/tensorflow/hub\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/huggingface/transformers\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/25720743?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Huggingface transformers](https://github.com/huggingface/transformers) | State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.   | ![GitHub stars](https://img.shields.io/github/stars/huggingface/transformers?style=social) | \u003ca href=\"https://github.com/huggingface/transformers/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/huggingface/transformers\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/lightly-ai/lightly\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/50146475?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Lightly](https://github.com/lightly-ai/lightly) | Lightly is a computer vision framework for self-supervised learning.    | ![GitHub stars](https://img.shields.io/github/stars/lightly-ai/lightly?style=social) | \u003ca href=\"https://github.com/lightly-ai/lightly/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/lightly-ai/lightly\" height=\"15\"/\u003e\u003c/a\u003e |\n\n## Visualization and Interaction\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/renumics/spotlight\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/Renumics/spotlight/main/static/img/spotlight.svg\"  width=\"100\"/\u003e\u003c/a\u003e | [Renumics Spotlight](https://github.com/renumics/spotlight) | Curation tool for unstructured data that connects your stack to the data-centric AI ecosystem.    | ![GitHub stars](https://img.shields.io/github/stars/renumics/spotlight?style=social) | \u003ca href=\"https://github.com/renumics/spotlight/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/renumics/spotlight\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/voxel51/fiftyone\"\u003e\u003cimg src=\"https://user-images.githubusercontent.com/25985824/106288517-2422e000-6216-11eb-871d-26ad2e7b1e59.png\"  width=\"100\"/\u003e\u003c/a\u003e | [FiftyOne ](https://github.com/voxel51/fiftyone) | The open-source tool for building high-quality datasets and computer vision models.   | ![GitHub stars](https://img.shields.io/github/stars/voxel51/fiftyone?style=social) | \u003ca href=\"https://github.com/voxel51/fiftyone/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/voxel51/fiftyone\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/code-kern-ai/refinery\"\u003e\u003cimg src=\"https://developers.kern.ai/refinery-bicon.png\"  width=\"100\"/\u003e\u003c/a\u003e | [refinery](https://github.com/code-kern-ai/refinery) | The data scientist's open-source choice to scale, assess and maintain natural language data.   | ![GitHub stars](https://img.shields.io/github/stars/code-kern-ai/refinery?style=social) | \u003ca href=\"https://github.com/code-kern-ai/refinery/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/code-kern-ai/refinery\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/argilla-io/argilla\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/dvsrepo/imgs/main/rg.svg\"  width=\"100\"/\u003e\u003c/a\u003e | [Argilla](https://github.com/argilla-io/argilla) | Argilla helps domain experts and data teams to build better NLP datasets in less time.    | ![GitHub stars](https://img.shields.io/github/stars/argilla-io/argilla?style=social) | \u003ca href=\"https://github.com/argilla-io/argilla/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/argilla-io/argilla\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/xtreme1-io/xtreme1\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/120552187?s=280\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Xtreme1](https://github.com/xtreme1-io/xtreme1) | Xtreme1 is the world's first open-source platform for multisensory training data.     | ![GitHub stars](https://img.shields.io/github/stars/xtreme1-io/xtreme1?style=social) | \u003ca href=\"https://github.com/xtreme1-io/xtreme1/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/xtreme1-io/xtreme1\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/ydataai/ydata-profiling\"\u003e\u003cimg src=\"https://assets.ydata.ai/oss/ydata-profiling_red.png\"  width=\"100\"/\u003e\u003c/a\u003e | [YData Profiling](https://github.com/ydataai/ydata-profiling) | YData Profiling is a python package to perform Exploratory Data Analysis (EDA) for tabular and time-series data. | ![GitHub stars](https://img.shields.io/github/stars/ydataai/ydata-profiling?style=social) | \u003ca href=\"https://github.com/ydataai/ydata-profiling/blob/master/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/ydataai/ydata-profiling\" height=\"15\"/\u003e\u003c/a\u003e |\n\n\n## Outlier and noise detection\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/cleanlab/cleanlab\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/90712480?s=280\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Cleanlab](https://github.com/cleanlab/cleanlab) | Cleanlab facilitates machine learning with messy, real-world data by providing clean labels for robust training and flagging errors in your data.  | ![GitHub stars](https://img.shields.io/github/stars/cleanlab/cleanlab?style=social) | \u003ca href=\"https://github.com/cleanlab/cleanlab/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/cleanlab/cleanlab\" height=\"15\"/\u003e\u003c/a\u003e |\n| **PyOD** | [PyOD](https://github.com/yzhao062/pyod) | A Comprehensive and Scalable Python Library for Outlier Detection (Anomaly Detection)     | ![GitHub stars](https://img.shields.io/github/stars/yzhao062/pyod?style=social) | \u003ca href=\"https://github.com/yzhao062/pyod/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/yzhao062/pyod\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/datamllab/tods\"\u003e\u003cimg src=\"https://github.com/datamllab/tods/blob/master/docs/source/img/tods_logo.png\"  width=\"100\"/\u003e\u003c/a\u003e | [TODS](https://github.com/datamllab/tods) | An full-stack automated time-series outlier detection system.     | ![GitHub stars](https://img.shields.io/github/stars/datamllab/tods?style=social) | \u003ca href=\"https://github.com/datamllab/tods/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/datamllab/tods\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/SeldonIO/alibi-detect\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/SeldonIO/alibi-detect/master/doc/source/_static/Alibi_Detect_Logo_rgb.png\"  width=\"100\"/\u003e\u003c/a\u003e | [Alibi Detect](https://github.com/SeldonIO/alibi-detect) | Algorithms for outlier, adversarial and drift detection.    | ![GitHub stars](https://img.shields.io/github/stars/SeldonIO/alibi-detect?style=social) | \u003ca href=\"https://github.com/SeldonIO/alibi-detect/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/SeldonIO/alibi-detect\" height=\"15\"/\u003e\u003c/a\u003e |\n\n## Explainability\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/slundberg/shap\"\u003e\u003cimg src=\"https://shap-lrjball.readthedocs.io/en/latest/_static/shap_logo.png\"  width=\"100\"/\u003e\u003c/a\u003e | [SHAP](https://github.com/slundberg/shap) | A game theoretic approach to explain the output of any machine learning model.     | ![GitHub stars](https://img.shields.io/github/stars/slundberg/shap?style=social) | \u003ca href=\"https://github.com/slundberg/shap/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/slundberg/shap\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/SeldonIO/alibi\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/SeldonIO/alibi/master/doc/source/_static/Alibi_Explain_Logo_rgb.png\"  width=\"100\"/\u003e\u003c/a\u003e | [Alibi](https://github.com/SeldonIO/alibi) | Alibi is an open source Python library aimed at machine learning model inspection and interpretation.     | ![GitHub stars](https://img.shields.io/github/stars/SeldonIO/alibi?style=social) | \u003ca href=\"https://github.com/SeldonIO/alibi/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/SeldonIO/alibi\" height=\"15\"/\u003e\u003c/a\u003e |\n| **LIME** | [LIME](https://github.com/marcotcr/lime) | Explaining the predictions of any machine learning classifier.     | ![GitHub stars](https://img.shields.io/github/stars/marcotcr/lime?style=social) | \u003ca href=\"https://github.com/marcotcr/lime/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/marcotcr/lime\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/pytorch/captum\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/pytorch/captum/master/website/static/img/captum_logo.png\"  width=\"100\"/\u003e\u003c/a\u003e | [Captum](https://github.com/pytorch/captum) | Model interpretability and understanding for PyTorch.     | ![GitHub stars](https://img.shields.io/github/stars/pytorch/captum?style=social) | \u003ca href=\"https://github.com/pytorch/captum/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/pytorch/captum\" height=\"15\"/\u003e\u003c/a\u003e |\n\n## Active learning\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/modAL-python/modAL\"\u003e\u003cimg src=\"https://camo.githubusercontent.com/1e424af39166bfa1bb4b49ba00bd39c617f16e4fc0931d78185413fdf720e015/68747470733a2f2f6d6f64616c2d707974686f6e2e72656164746865646f63732e696f2f656e2f6c61746573742f5f7374617469632f6d6f64414c5f622e706e67\"  width=\"100\"/\u003e\u003c/a\u003e | [modAL](https://github.com/modAL-python/modAL) | A modular active learning framework for Python.   | ![GitHub stars](https://img.shields.io/github/stars/modAL-python/modAL?style=social) | \u003ca href=\"https://github.com/modAL-python/modAL/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/modAL-python/modAL\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/baal-org/baal\"\u003e\u003cimg src=\"https://camo.githubusercontent.com/a29cf47a24993d08cfdae02dabcaa04602e7fd3cd5bbd8ddf053238373409625/68747470733a2f2f692e696d6775722e636f6d2f5a647a6232515a2e706e67\"  width=\"100\"/\u003e\u003c/a\u003e | [Bayesian Active Learning (Baal)](https://github.com/baal-org/baal) | Library to enable Bayesian active learning in your research or labeling work.     | ![GitHub stars](https://img.shields.io/github/stars/baal-org/baal?style=social) | \u003ca href=\"https://github.com/baal-org/baal/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/baal-org/baal\" height=\"15\"/\u003e\u003c/a\u003e |\n\n\n## Uncertainty quantification\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/uncertainty-toolbox /\"\u003e\u003cimg src=\"https://repository-images.githubusercontent.com/293174301/60be5300-463c-11eb-9b69-70a31a38e5ee\"  width=\"100\"/\u003e\u003c/a\u003e | [Uncertainty Toolbox](https://github.com/uncertainty-toolbox/uncertainty-toolbox/) | A Python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization.    | ![GitHub stars](https://img.shields.io/github/stars/uncertainty-toolbox/uncertainty-toolbox?style=social) | \u003ca href=\"https://github.com/scikit-learn-contrib/MAPIE/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/scikit-learn-contrib/MAPIE\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/scikit-learn-contrib/MAPIE\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/scikit-learn-contrib/MAPIE/master/doc/images/mapie_logo_nobg_cut.png\"  width=\"100\"/\u003e\u003c/a\u003e | [MAPIE](https://github.com/scikit-learn-contrib/MAPIE) | A scikit-learn-compatible module for estimating prediction intervals.     | ![GitHub stars](https://img.shields.io/github/stars/scikit-learn-contrib/MAPIE?style=social) | \u003ca href=\"https://github.com/scikit-learn-contrib/MAPIE/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/scikit-learn-contrib/MAPIE\" height=\"15\"/\u003e\u003c/a\u003e |\n\n## Bias and fairness\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/Trusted-AI/AIF360\"\u003e\u003cimg src=\"https://artwork.lfaidata.foundation/projects/ai-fairness-360/icon/color/ai-fairness-360-icon-color.png\"  width=\"100\"/\u003e\u003c/a\u003e | [AIF360](https://github.com/Trusted-AI/AIF360) | The AI Fairness 360 toolkit helps to detect and mitigate bias in machine learning models throughout the AI application lifecycle.    | ![GitHub stars](https://img.shields.io/github/stars/Trusted-AI/AIF360?style=social) | \u003ca href=\"https://github.com/Trusted-AI/AIF360/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/Trusted-AI/AIF360\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/fairlearn/fairlearn\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/54074260?s=60\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Fairlearn](https://github.com/fairlearn/fairlearn) | A Python package to assess and improve fairness of machine learning models.     | ![GitHub stars](https://img.shields.io/github/stars/fairlearn/fairlearn?style=social) | \u003ca href=\"https://github.com/fairlearn/fairlearn/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/fairlearn/fairlearn\" height=\"15\"/\u003e\u003c/a\u003e |\n\n## Observability and Monitoring\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/Arize-ai/phoenix\"\u003e\u003cimg src=\"https://storage.googleapis.com/arize-assets/phoenix/assets/phoenix-logo-light-no-text.svg\"  width=\"100\"/\u003e\u003c/a\u003e | [Arize-Phoenix](https://github.com/Arize-ai/phoenix) | Arize-Phoenix is a Python library for ML observability (monitoring + root-cause analysis) for tabular, CV, NLP, and LLM models.     | ![GitHub stars](https://img.shields.io/github/stars/Arize-AI/phoenix?style=social) | \u003ca href=\"https://github.com/Arize-ai/phoenix/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/Arize-AI/phoenix\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/deepchecks/deepchecks\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/92298186?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Deepchecks](https://github.com/deepchecks/deepchecks) | Deepchecks is a Python package for comprehensively validating your machine learning models and data with minimal effort.     | ![GitHub stars](https://img.shields.io/github/stars/deepchecks/deepchecks?style=social) | \u003ca href=\"https://github.com/deepchecks/deepchecks/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/deepchecks/deepchecks\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/evidentlyai/evidently\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/75031056?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Evidently](https://github.com/evidentlyai/evidently) | An open-source framework to evaluate, test and monitor ML models in production.    | ![GitHub stars](https://img.shields.io/github/stars/evidentlyai/evidently?style=social) | \u003ca href=\"https://github.com/evidentlyai/evidently/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/evidentlyai/evidently\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/langfuse/langfuse\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/134601687?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [langfuse](https://github.com/langfuse/langfuse) | Open source observability and analytics for LLM applications.   | ![GitHub stars](https://img.shields.io/github/stars/langfuse/langfuse?style=social) | \u003ca href=\"https://github.com/langfuse/langfuse/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/langfuse/langfuse\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/whylabs/langkit\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/56651354?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [langkit](https://github.com/whylabs/langkit) | An open-source toolkit for monitoring Large Language Models (LLMs).   | ![GitHub stars](https://img.shields.io/github/stars/whylabs/langkit?style=social) | \u003ca href=\"https://github.com/whylabs/langkit/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/whylabs/langkit\" height=\"15\"/\u003e\u003c/a\u003e |\n\n## Augmentation and synthetic data\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/albumentations-team/albumentations\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/57894582?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Albumentations](https://github.com/albumentations-team/albumentations) | Fast image augmentation library and an easy-to-use wrapper around other libraries.    | ![GitHub stars](https://img.shields.io/github/stars/albumentations-team/albumentations?style=social) | \u003ca href=\"https://github.com/albumentations-team/albumentations/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/albumentations-team/albumentations\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/gretelai/gretel-synthetics\"\u003e\u003cimg src=\"https://camo.githubusercontent.com/d9af85480f0c8d7eef05a377d31b2a4471e83371a92c2fca462943db8dc68bf6/68747470733a2f2f67726574656c2d7075626c69632d776562736974652e73332e616d617a6f6e6177732e636f6d2f6173736574732f676f62735f7468655f6361745f4031782e706e67\"  width=\"100\"/\u003e\u003c/a\u003e | [Gretel Synthetics](https://github.com/gretelai/gretel-synthetics) | Synthetic data generators for structured and unstructured text, featuring differentially private learning.     | ![GitHub stars](https://img.shields.io/github/stars/gretelai/gretel-synthetics?style=social) | \u003ca href=\"https://github.com/gretelai/gretel-synthetics/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/gretelai/gretel-synthetics\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/sdv-dev/SDV\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/sdv-dev/SDV/master/docs/images/SDV-DataCebo.png\"  width=\"100\"/\u003e\u003c/a\u003e | [SDV](https://github.com/sdv-dev/SDV) | Synthetic Data Generation for tabular, relational and time series data.     | ![GitHub stars](https://img.shields.io/github/stars/sdv-dev/SDV?style=social) | \u003ca href=\"https://github.com/sdv-dev/SDV/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/sdv-dev/SDV\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/ydataai/ydata-synthetic\"\u003e\u003cimg src=\"https://assets.ydata.ai/oss/ydata-synthetic-_red.png\"  width=\"100\"/\u003e\u003c/a\u003e | [YData Synthetic](https://github.com/ydataai/ydata-synthetic) | YData Synthetic is a python package to generate synthetic tabular and time-series data by leveraging state-of-the-art generative models. | ![GitHub stars](https://img.shields.io/github/stars/ydataai/ydata-synthetic?style=social) | \u003ca href=\"https://github.com/ydataai/ydata-synthetic/blob/dev/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/ydataai/ydata-synthetic\" height=\"15\"/\u003e\u003c/a\u003e |\n\n## Security and robustness\n\n| Logo | Name | Description | Popularity | License |\n| ------- | ---- | ----------- | ---------- | -------- |\n| \u003ca href=\"https://github.com/cleverhans-lab/cleverhans\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/cleverhans-lab/cleverhans/master/assets/logo.png\"  width=\"100\"/\u003e\u003c/a\u003e | [CleverHans](https://github.com/cleverhans-lab/cleverhans) | An adversarial example library for constructing attacks, building defenses, and benchmarking both.    | ![GitHub stars](https://img.shields.io/github/stars/cleverhans-lab/cleverhans?style=social) | \u003ca href=\"https://github.com/cleverhans-lab/cleverhans/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/cleverhans-lab/cleverhans\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/Trusted-AI/adversarial-robustness-toolbox\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/Trusted-AI/adversarial-robustness-toolbox/main/docs/images/art_lfai.png\"  width=\"100\"/\u003e\u003c/a\u003e | [Adversarial Robustness Toolbox](https://github.com/Trusted-AI/adversarial-robustness-toolbox) | Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams.    | ![GitHub stars](https://img.shields.io/github/stars/Trusted-AI/adversarial-robustness-toolbox?style=social) | \u003ca href=\"https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/Trusted-AI/adversarial-robustness-toolbox\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/bethgelab/foolbox\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/bethgelab/foolbox/master/guide/.vuepress/public/logo_small.png\"  width=\"100\"/\u003e\u003c/a\u003e | [Foolbox](https://github.com/bethgelab/foolbox) | Foolbox is a Python library that lets you easily run adversarial attacks against machine learning models like deep neural networks.    | ![GitHub stars](https://img.shields.io/github/stars/bethgelab/foolbox?style=social) | \u003ca href=\"https://github.com/bethgelab/foolbox/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/bethgelab/foolbox\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/Giskard-AI/giskard\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/71782571?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [Giskard](https://github.com/Giskard-AI/giskard) | The testing framework for ML models, from tabular to LLMs.   | ![GitHub stars](https://img.shields.io/github/stars/Giskard-AI/giskard?style=social) | \u003ca href=\"https://github.com/Giskard-AI/giskard/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/Giskard-AI/giskard\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/laiyer-ai/llm-guard\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/140750944?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [LLM-Guard](https://github.com/laiyer-ai/llm-guard) | The Security Toolkit for LLM Interactions.    | ![GitHub stars](https://img.shields.io/github/stars/laiyer-ai/llm-guard?style=social) | \u003ca href=\"https://github.com/laiyer-ai/llm-guard/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/laiyer-ai/llm-guard\" height=\"15\"/\u003e\u003c/a\u003e |\n| \u003ca href=\"https://github.com/guardrails-ai/guardrails\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/140440022?s=200\u0026v=4\"  width=\"100\"/\u003e\u003c/a\u003e | [guardrails](https://github.com/guardrails-ai/guardrails) | Adding guardrails to large language models.    | ![GitHub stars](https://img.shields.io/github/stars/guardrails-ai/guardrails?style=social) | \u003ca href=\"https://github.com/guardrails-ai/guardrails/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/guardrails-ai/guardrails\" height=\"15\"/\u003e\u003c/a\u003e |\n\n# 🏀 Data-centric AI playbook\n\n\u003cp align=\"center\"\u003e\u003ca href=\"https://github.com/Renumics/awesome-open-data-centric-ai\"\u003e\u003cimg src=\"static/img/dcai_workflow.svg\" width=\"100%\"/\u003e\u003c/a\u003e\u003c/p\u003e\n\n## Exploratory data analysis (EDA)\n\n| Name | Data type | Description | Notebook | \n| ---- | ---- | ---- | ----------- | \n| [Understand distributions](https://renumics.com/docs/playbook/huggingface-embedding) | image  | Use the Huggingface transformers library to compute image embeddings and explore the dataset based on the similarity map and additional metdata. | \u003ca href=\"https://colab.research.google.com/github/Renumics/spotlight/blob/main/playbook/rookie/huggingface_embedding.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e |\n\n## Cleaning\n\n| Name | Data type | Description | Notebook | \n| ---- | ---- | ---- | ----------- | \n| [Detect duplicates](https://renumics.com/docs/playbook/duplicates-annoy/) | agnostic  | Use the Annoy library to detect nearest neighbors in the embedding space and inspect data points that are duplicates / near duplicates. | \u003ca href=\"https://colab.research.google.com/github/Renumics/spotlight/blob/main/playbook/rookie/detect_duplicates.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e |\n| [Detect outliers](https://renumics.com/docs/playbook/outliers-cleanlab/) | agnostic  | Use the Cleanlab library to compute outlier scores based on model output (embeddings, probabilities) and inspect outlier candidates. | \u003ca href=\"https://colab.research.google.com/github/Renumics/spotlight/blob/main/playbook/rookie/outlier_cleanlab.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e |\n| [Detect image issues](https://renumics.com/docs/playbook/cv-issues/) | image  | Use the Cleanvision library to extrapact typical image issues (brightness, blurr, aspect ratio, SNR and duplicates) and identify critical segments through manual inspection.| \u003ca href=\"https://colab.research.google.com/github/Renumics/spotlight/blob/playbook_initial_draft/playbook/rookie/cv_issue_detection.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e |\n\n## Annotation\n\n| Name | Data type | Description | Notebook | \n| ---- | ---- | ---- | ----------- | \n| [Find label inconsistencies](https://renumics.com/docs/playbook/label-errors-cleanlab/) | agnostic | Use the Cleanlab library to compute label error flags based on model probabilities and manually inspect critical data segments. | \u003ca href=\"https://colab.research.google.com/github/Renumics/spotlight/blob/main/playbook/rookie/label_errors_cleanlab.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e |\n\n## Modeling\n| Name | Data type | Description | Notebook | \n| ---- | ---- | ---- | ----------- | \n| [Detect leakage](https://renumics.com/docs/playbook/leakage-annoy/) | agnostic | Use nearest neighbor distances to identify candidates for data leakage and manual inspect them | \u003ca href=\"https://colab.research.google.com/github/Renumics/spotlight/blob/main/playbook/rookie/leakage_annoy.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e |\n\n## Validation\n| Name | Data type | Description | Notebook | \n| ---- | ---- | ---- | ----------- | \n| [Inspect decision boundaries](https://renumics.com/docs/playbook/decision-boundary/) | agnostic | Compute a decision boundary score based on certainty ratios and inspect the results in a scatter plot. | \u003ca href=\"https://colab.research.google.com/github/Renumics/spotlight/blob/main/playbook/rookie/decision_boundary_detection.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e |\n\n## Monitoring\n| Name | Data type | Description | Notebook | \n| ---- | ---- | ---- | ----------- | \n| [Detect data drift ](https://renumics.com/docs/playbook/label-errors-cleanlab/) | agnostic | Compute the cosine distance of the k-nearest neighbor in the embedding space as the drift distance and inspect critical segments. | \u003ca href=\"https://colab.research.google.com/github/Renumics/spotlight/blob/main/playbook/rookie/drift_kcore.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/\u003e\u003c/a\u003e |\n\n# 📖 Further reading\n\nIn order to keep a useful focus and to prevent duplicate work, we excluded some topics from this list. Read more about them here:\n\n1. DCAI tools for tabular data. There is an [awesome list](https://github.com/Data-Centric-AI-Community/awesome-data-centric-ai) for that maintained by the [Ydata team](https://github.com/Data-Centric-AI-Community).\n2. Labeling tools. Although labeling is part of the DCAI workflow, we refer to the [awesome list](https://github.com/zenml-io/awesome-open-data-annotation) of the [ZenML team](https://github.com/zenml-io) on that topic. \n3. MLOps tooling. We exclude all topics that are clearly out of the DCAI scope and refer to established [MLOps awesome lists](https://github.com/EthicalML/awesome-production-machine-learning) for these tools.\n4. Research papers. We focus on industrial-ready open source tooling, check out [this list](https://github.com/HazyResearch/data-centric-ai) for a research-oriented view on DCAI.\n\n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRenumics%2Fawesome-open-data-centric-ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRenumics%2Fawesome-open-data-centric-ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRenumics%2Fawesome-open-data-centric-ai/lists"}