awesome-seml
A curated list of articles that cover the software engineering best practices for building machine learning applications.
https://github.com/SE-ML/awesome-seml
Last synced: 1 day ago
JSON representation
Broad Overviews
- Engineering Best Practices for Machine Learning - β
- Rules of Machine Learning: Best Practices for ML Engineering - β
- Software Engineering for Machine Learning: A Case Study - πβ
- AI Engineering: 11 Foundational Practices - β
- Hidden Technical Debt in Machine Learning Systems - πβ
- Best Practices for Machine Learning Applications
Data Management
- A Survey on Data Collection for Machine Learning A Big Data - AI Integration Perspective_2019 - π
- Data management challenges in production machine learning
- Data Validation for Machine Learning - π
- The curse of big data labeling and three ways to solve it
- The ultimate guide to data labeling for ML
- How to organize data labelling for ML
- Automating Large-Scale Data Quality Verification - π
- The Data Linter: Lightweight, Automated Sanity Checking for ML Data Sets - π
Deployment and Operation
- Continuous Delivery for Machine Learning - β
- Continuous Training for Production ML in the TensorFlow Extended (TFX) Platform - π
- Machine Learning Logistics
- Model Governance Reducing the Anarchy of Production - π
- Operational Machine Learning
- Underspecification Presents Challenges for Credibility in Modern Machine Learning - π
- ModelOps: Cloud-based lifecycle management for reliable and trusted AI
- Scaling Machine Learning as a Service - π
- TFX: A tensorflow-based Production-Scale ML Platform - π
- Versioning for end-to-end machine learning pipelines - π
- Building Continuous Integration Services for Machine Learning - π
- Best Practices in Machine Learning Infrastructure
- Fairness Indicators: Scalable Infrastructure for Fair ML Systems - π
- Machine learning: Moving from experiments to production
- ML Ops: Machine Learning as an engineered disciplined
Governance
- An Architectural Risk Analysis Of Machine Learning Systems
- Beyond Debiasing
- Inherent trade-offs in the fair determination of risk scores - π
- Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
- Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing - π
- Responsible AI practices - β
- Understanding Software-2.0 - π
- A Human-Centered Interpretability Framework Based on Weight of Evidence - π
Model Training
- 10 Best Practices for Deep Learning
- Fairness On The Ground: Applying Algorithmic FairnessApproaches To Production Systems - π
- Nitpicking Machine Learning Technical Debt
- Preliminary Systematic Literature Review of Machine Learning System Development Process - π
- What Went Wrong and Why? Diagnosing Situated Interaction Failures in the Wild - π
- Preparing and Architecting for Machine Learning
- Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement - π
- Pitfalls of supervised feature selection - π
- How do you manage your Machine Learning Experiments?
- Machine Learning Testing: Survey, Landscapes and Horizons - π
- On human intellect and machine failures: Troubleshooting integrative machine learning systems - π
- Pitfalls and Best Practices in Algorithm Configuration - π
- On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach - πβ
- Software development best practices in a deep learning environment
- Testing and Debugging in Machine Learning
Social Aspects
Tooling
- Airflow - Programmatically author, schedule and monitor workflows.
- Data Version Control (DVC) - DVC is a data and ML experiments management tool.
- Kubeflow - A platform for data scientists who want to build and experiment with ML pipelines.
- MLFlow - Manage the ML lifecycle, including experimentation, deployment, and a central model registry.
- Neptune.ai - Experiment tracking tool bringing organization and collaboration to data science projects.
- Spark Machine Learning - Sparkβs ML library consisting of common learning algorithms and utilities.
- Alibi Detect - Python library focused on outlier, adversarial and drift detection.
- Archai - Neural architecture search.
- Facets Overview / Facets Dive - Robust visualizations to aid in understanding machine learning datasets.
- Great Expectations - Data validation and testing with integration in pipelines.
- HParams - A thoughtful approach to configuration management for machine learning projects.
- Label Studio - A multi-type data labeling and annotation tool with standardized output format.
- LiFT - Linkedin fairness toolkit.
- Model Card Toolkit - Streamlines and automates the generation of model cards; for model documentation.
- Neuraxle - Sklearn-like framework for hyperparameter tuning and AutoML in deep learning projects.
- PyTorch Lightning - The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.
- REVISE: REvealing VIsual biaSEs - Automatically detect bias in visual data sets.
- Robustness Metrics - Lightweight modules to evaluate the robustness of classification models.
- Seldon Core - An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models on Kubernetes.
- Tensorflow Extended (TFX) - An end-to-end platform for deploying production ML pipelines.
- Tensorflow Data Validation (TFDV) - Library for exploring and validating machine learning data. Similar to Great Expectations, but for Tensorflow data.
- Aim - Aim is an open source experiment tracking tool.
- OpenML - An inclusive movement to build an open, organized, online ecosystem for machine learning.
- FairLearn - A toolkit to assess and improve the fairness of machine learning models.