Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

awesome-data-centric-ai

Open-Source Software, Tutorials, and Research on Data-Centric AI 🤖
https://github.com/Data-Centric-AI-Community/awesome-data-centric-ai

Last synced: 1 day ago
JSON representation

  • 📊 Data Profiling

    • D-Tale - Is an open-source python auto-visualization library that brings you an easy way to view & analyze Pandas data structures. It integrates seamlessly with ipython notebooks & python/ipython terminals.
    • YData Profiling - YData Profiling supports both Pandas and Spark DataFrames, providing a fast and straightforward visual data understanding.
    • AutoViz - Automatically Visualize any dataset, any size with a single line of code.
    • SweetViz - SweetViz is an open-source Python library that generates beautiful, high-density visualizations to kickstart EDA (Exploratory Data Analysis) with just two lines of code.
    • DataPrep.EDA - DataPrep.EDA is an EDA (Exploratory Data Analysis) tool in Python that allows you to understand a Pandas/Dask DataFrame with a few lines of code in seconds.
    • Lux - Lux is a Python library that facilitates fast and easy data exploration by automating the visualization and data analysis process.
    • Great Expectations - Great Expectations helps data teams eliminate pipeline debt, through data testing, documentation, and profiling.
    • D-Tale - Is an open-source python auto-visualization library that brings you an easy way to view & analyze Pandas data structures. It integrates seamlessly with ipython notebooks & python/ipython terminals.
    • Data Profiler - The DataProfiler is a Python library designed to make data analysis, monitoring, and sensitive data detection easy.
    • whylogs - whylogs is the open source standard for data logging. It uses data profiling techniques to create *whylogs profiles*, which can be used as logs to enable monitoring and observability for data pipelines and ML models

  • 🏷 Data Labelling

    • LabelImg - LabelImg is a graphical image annotation tool. It is written in Python and uses Qt for its graphical interface.
    • LabelMe - LabelMe is an image polygonal annotation tool that uses Python and Qt.
    • LabelStudio - Label Studio is an open-source data labeling tool. It lets you label data types like audio, text, images, videos, and time series with a simple and straightforward UI and export to various model formats.
    • LabelImg - LabelImg is a graphical image annotation tool. It is written in Python and uses Qt for its graphical interface.
    • LabelStudio - Label Studio is an open-source data labeling tool. It lets you label data types like audio, text, images, videos, and time series with a simple and straightforward UI and export to various model formats.
    • LabelMe - LabelMe is an image polygonal annotation tool that uses Python and Qt.
    • EchoML - Play, visualize and annotate your audio files
    • Awesome Open Source Data Annotation & Labeling Tools - A list of the open-source tools available (sorted by task type) for anyone who wants to label data. Only actively maintained tools are listed.
    • TagAnamoly - Anomaly detection labeling tool, specifically for multiple time series (one time series per category).

  • 👾 An Open Invitation

  • 📚 Tutorials and Resources

  • 🎓 Courses

    • MIT Introduction to Data-Centric AI - This class covers algorithms to find and fix common issues in ML data and to construct better datasets, concentrating on data used in supervised learning tasks like classification. All material taught in this course is highly practical, focused on impactful aspects of real-world ML applications, rather than mathematical details of how particular models work. You can take this course to learn practical techniques not covered in most ML classes, which will help mitigate the “garbage in, garbage out” problem that plagues many real-world ML applications. [Website](https://dcai.csail.mit.edu/) | [Lecture Videos](https://www.youtube.com/watch?v=ayzOzZGHZy4&list=PLnSYPjg2dHQKdig0vVbN-ZnEU0yNJ1mo5) | [Lab Assignments](https://github.com/dcai-course/dcai-lab)

  • 🔐 Synthetic Data

    • DataSynthesizer - DataSynthesizer generates synthetic data that simulates a given dataset. It applies Differential Privacy techniques to achieve a strong privacy guarantee.
    • YData Synthetic - Structured Synthetic Data Generators using Generative Adversarial Networks specifically for tabular and time-series data.
    • SDV - The Synthetic Data Vault (SDV) is a Synthetic Data Generation ecosystem of libraries that allows users to easily learn single-table, multi-table, and time-series datasets to, later on, generate new Synthetic Data that has the same format and statistical properties as the original dataset.
    • Pomegranate - Pomegranate is a package for building probabilistic models in Python that is implemented in Cython for speed. Most of these models can sample data.
    • Gretel Synthetics - Gretel Synthetics package allows developers to quickly get immersed with synthetic data generation through the use of neural networks.
    • Time-Series-Generator - Time-Series-Generator allows developers to create synthetic timeseries datasets in a generic way that follows different trends.The goal here is to have non-sensitive data available to demo solutions and test the effectiveness of those solutions and/or algorithms.
    • Zpy - Zpy solves the problem of lack of business specific datasets for computer vision based applications. Zpy uses Python and Blender (an open source 3D graphics toolset) to create synthetic datasets suitable for unique business cases.
    • Synthpop - A tool for producing synthetic versions of microdata containing confidential information so that they are safe to be released to users for exploratory analysis.

  • 🛠️ Data Preparation

    • DataFix - DataFix is a Python tool for detecting and correcting distribution shifts between a reference and a query dataset. It detects shifts, localizes the specific features responsible for the shift, and corrects them efficiently.
Programming Languages
Python 14 Jupyter Notebook 3 JavaScript 2 TypeScript 1 R 1 HTML 1
Categories
📊 Data Profiling 10 🏷 Data Labelling 9 🔐 Synthetic Data 8 📚 Tutorials and Resources 2 👾 An Open Invitation 1 🛠️ Data Preparation 1 🎓 Courses 1
Sub Categories
Keywords
python 10 machine-learning 10 data-science 8 deep-learning 7 pandas 5 exploratory-data-analysis 5 time-series 4 synthetic-data 4 python3 4 eda 4 data-analysis 4 mlops 4 visualization 3 data-quality 3 data-profiling 3 data-exploration 3 annotations 3 dataset 3 computer-vision 3 pandas-dataframe 2 jupyter-notebook 2 jupyter 2 statistics 2 pytorch 2 generative-adversarial-network 2 exploration 2 annotation 2 gans 2 data-visualization 2 datacleaning 2 gan 2 ai 2 semantic-segmentation 2 labeling-tool 2 labeling 2 image-classification 2 privacy 2 image-annotation 2 gan-architectures 1 datagenerator 1 detection 1 datageneration 1 tensorflow2 1 timeseries 1 training-data 1 data-generation 1 generative-ai 1 generative-model 1 generativeai 1 multi-table 1