Articles

• Continuous Delivery for Machine Learning
• Machine Learning Operations (MLOps): Overview, Definition, and Architecture
• Rules of Machine Learning: Best Practices for ML Engineering
• The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction
• MLOps Roadmap: A Complete MLOps Career Guide
• Practitioners guide to MLOps: A framework for continuous delivery and automation of machine learning
• What Is MLOps?
• MLOps: Machine Learning as an Engineering Discipline

AutoML

• AutoKeras - AutoKeras goal is to make machine learning accessible for everyone.
• AutoPyTorch - Automatic architecture search and hyperparameter optimization for PyTorch.
• AutoSKLearn - Automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator.
• EvalML - A library that builds, optimizes, and evaluates ML pipelines using domain-specific functions.
• FLAML - Finds accurate ML models automatically, efficiently and economically.
• MindsDB - AI layer for databases that allows you to effortlessly develop, train and deploy ML models.
• MLBox - MLBox is a powerful Automated Machine Learning python library.
• Model Search - Framework that implements AutoML algorithms for model architecture search at scale.
• NNI - An open source AutoML toolkit for automate machine learning lifecycle.
• AutoGluon - Automated machine learning for image, text, tabular, time-series, and multi-modal data.
• H2O AutoML - Automates ML workflow, which includes automatic training and tuning of models.

Books

• Beginning MLOps with MLFlow
• Deep Learning in Production
• Machine Learning Engineering in Action
• MLOps Engineering at Scale
• Building Machine Learning Pipelines
• Building Machine Learning Powered Applications
• Designing Machine Learning Systems
• Implementing MLOps in the Enterprise
• Introducing MLOps
• Kubeflow for Machine Learning
• Kubeflow Operations Guide
• Machine Learning Design Patterns
• ML Ops: Operationalizing Data Science
• Practical MLOps
• Reliable Machine Learning
• AI Governance
• AI Model Evaluation
• MLOps Lifecycle Toolkit
• Engineering MLOps
• Practical Deep Learning at Scale with MLflow
• Production-Ready Applied Deep Learning
• The Machine Learning Solutions Architect Handbook

CI/CD for Machine Learning

• CML - Open-source library for implementing CI/CD in machine learning projects.
• KitOps
• ClearML - Auto-Magical CI/CD to streamline your ML workflow.

Cron Job Monitoring

• Cronitor - Monitor any cron job or scheduled task.
• HealthchecksIO - Simple and effective cron job monitoring.
• Heartbeat.pm - Monitoring aliveness of any sensor/cron job.

Data Catalog

• Amundsen - Data discovery and metadata engine for improving productivity when interacting with data.
• OpenMetadata - A Single place to discover, collaborate and get your data right.
• CKAN - Open-source DMS (data management system) for powering data hubs and data portals.
• Magda - A federated, open-source data catalog for all your big data and small data.
• Metacat - Unified metadata exploration API service for Hive, RDS, Teradata, Redshift, S3 and Cassandra.
• DataHub - LinkedIn's generalized metadata search & discovery tool.
• Apache Atlas - Provides open metadata management and governance capabilities to build a data catalog.

Data Enrichment

• Snorkel - A system for quickly generating training data with weak supervision.
• Upgini - Enriches training datasets with features from public and community shared data sources.

Data Exploration

• Apache Zeppelin - Enables data-driven, interactive data analytics and collaborative documents.
• Jupyter Notebook - Web-based notebook environment for interactive computing.
• Polynote - The polyglot notebook with first-class Scala support.
• BambooLib - An intuitive GUI for Pandas DataFrames.
• DataPrep - Collect, clean and visualize your data in Python.
• Jupytext - Jupyter Notebooks as Markdown Documents, Julia, Python or R scripts.
• Pandas Profiling - Create HTML profiling reports from pandas DataFrame objects.
• Deepnote - Drop-in replacement for Jupyter and an AI-native workspace for modern data teams.
• Google Colab - Hosted Jupyter notebook service that requires no setup to use.

Data Management

• Arrikto - Dead simple, ultra fast storage for the hybrid Kubernetes world.
• DVC - Management and versioning of datasets and machine learning models.
• BlazingSQL - A lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF.
• Delta Lake - Storage layer that brings scalable, ACID transactions to Apache Spark and other engines.
• Dolt - SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.
• Dud - A lightweight CLI tool for versioning data alongside source code and building data pipelines.
• Intake - A lightweight set of tools for loading and sharing data in data science projects.
• lakeFS - Repeatable, atomic and versioned data lake on top of object storage.
• Marquez - Collect, aggregate, and visualize a data ecosystem's metadata.
• Qdrant - An open source vector similarity search engine with extended filtering support.
• Quilt - A self-organizing data hub with S3 support.
• Hub - A dataset format for creating, storing, and collaborating on AI datasets of any size.
• Milvus - An open source embedding vector similarity search engine powered by Faiss, NMSLIB and Annoy.
• Potato - Portable annotation tool for creating labeled datasets.
• Git LFS - An open source Git extension for versioning large files.
• Pinecone - Managed and distributed vector similarity search used with a lightweight SDK.

Data Processing

• Airflow - Platform to programmatically author, schedule, and monitor workflows.
• Hadoop - Framework that allows for the distributed processing of large data sets across clusters.
• Spark - Unified analytics engine for large-scale data processing.
• Azkaban - Batch workflow job scheduler created at LinkedIn to run Hadoop jobs.
• Dagster - A data orchestrator for machine learning, analytics, and ETL.
• OpenRefine - Power tool for working with messy data and improving it.

Data Validation

• JSON Schema - A vocabulary that allows you to annotate and validate JSON documents.
• Cerberus - Lightweight, extensible data validation library for Python.
• Cleanlab - Python library for data-centric AI and machine learning with messy, real-world data and labels.
• TFDV - An library for exploring and validating machine learning data.
• Great Expectations - A Python data validation framework that allows to test your data against datasets.

Data Visualization

• Grafana - Multi-platform open source analytics and interactive visualization web application.
• Metabase - The simplest, fastest way to get business intelligence and analytics to everyone.
• Redash - Connect to any data source, easily visualize, dashboard and share your data.
• Dash - Analytical Web Apps for Python, R, Julia, and Jupyter.
• Facets - Visualizations for understanding and analyzing machine learning datasets.
• Lux - Fast and easy data exploration by automating the visualization and data analysis process.
• SolidUI - AI-generated visualization prototyping and editing platform, support 2D and 3D models.
• Count - SQL/drag-and-drop querying and visualisation tool based on notebooks.
• Tableau - Powerful and fastest growing data visualization tool used in the business intelligence industry.

Drift Detection

• Alibi Detect - An open source Python library focused on outlier, adversarial and drift detection.
• Frouros - An open source Python library for drift detection in machine learning systems.
• TorchDrift - A data and concept drift library for PyTorch.

Events

• MLOps World: Machine Learning in Production Conference
• NormConf - The Normcore Tech Conference
• Stanford MLSys Seminar Series
• AI Conference Deadline
• MLOps Conference - Keynotes and Panels

Feature Engineering

• Feature Engine - Feature engineering package with SKlearn like functionality.
• Featuretools - Python library for automated feature engineering.
• TSFresh - Python library for automatic extraction of relevant features from time series.

Feature Store

• Feast - End-to-end open source feature store for machine learning.
• Tecton - A fully-managed feature platform built to orchestrate the complete lifecycle of features.
• Butterfree - A tool for building feature stores. Transform your raw data into beautiful features.
• ByteHub - An easy-to-use feature store. Optimized for time-series data.
• Featureform - A Virtual Feature Store. Turn your existing data infrastructure into a feature store.
• Feathr - An enterprise-grade, high performance feature store.

Hyperparameter Tuning

• Optuna - Open source hyperparameter optimization framework to automate hyperparameter search.
• Advisor - Open-source implementation of Google Vizier for hyper parameters tuning.
• Hyperas - A very simple wrapper for convenient hyperparameter optimization.
• Hyperopt - Distributed Asynchronous Hyperparameter Optimization in Python.
• Katib - Kubernetes-based system for hyperparameter tuning and neural architecture search.
• KerasTuner - Easy-to-use, scalable hyperparameter optimization framework.
• Scikit Optimize - Simple and efficient library to minimize expensive and noisy black-box functions.
• Talos - Hyperparameter Optimization for TensorFlow, Keras and PyTorch.
• Tune - Python library for experiment execution and hyperparameter tuning at any scale.

Knowledge Sharing

• Knowledge Repo - Knowledge sharing platform for data scientists and other technical professions.
• Kyso - One place for data insights so your entire team can learn from your data.

Machine Learning Platform

• Bodywork - Deploys machine learning projects developed in Python, to Kubernetes.
• CNVRG - An end-to-end machine learning platform to build and deploy AI models at scale.
• DAGsHub - A platform built on open source tools for data, model and pipeline management.
• Dataiku - Platform democratizing access to data and enabling enterprises to build their own path to AI.
• DataRobot - AI platform that democratizes data science and automates the end-to-end ML at scale.
• Edge Impulse - Platform for creating, optimizing, and deploying AI/ML algorithms for edge devices.
• FedML - Simplifies the workflow of federated learning anywhere at any scale.
• Gradient - Multicloud CI/CD and MLOps platform for machine learning teams.
• Hopsworks - Open-source platform for developing and operating machine learning models at scale.
• Iguazio - Data science platform that automates MLOps with end-to-end machine learning pipelines.
• Knime - Create and productionize data science using one easy and intuitive environment.
• Kubeflow - Making deployments of ML workflows on Kubernetes simple, portable and scalable.
• LynxKite - A complete graph data science platform for very large graphs and other datasets.
• Modzy - Deploy, connect, run, and monitor machine learning (ML) models in the enterprise and at the edge.
• Pachyderm - Combines data lineage with end-to-end pipelines on Kubernetes, engineered for the enterprise.
• Sagemaker - Fully managed service that provides the ability to build, train, and deploy ML models quickly.
• SAS Viya - Cloud native AI, analytic and data management platform that supports the analytics life cycle.
• Valohai - MLOps platform for reproducible ML and LLM workflows from experimentation to production.
• envd - Machine learning development environment for data science and AI/ML engineering teams.
• ML Workspace - All-in-one web-based IDE specialized for machine learning and data science.
• MLReef - Open source MLOps platform that helps you collaborate, reproduce and share your ML work.
• Omnimizer - Simplifies and accelerates MLOps by bridging the gap between ML models and edge hardware.
• Neurolink - TypeScript-first multi-provider AI agent framework with workflow orchestration and MCP support.
• Katonic - Automate your cycle of intelligence with Katonic MLOps Platform.
• Sematic - An open-source end-to-end pipelining tool to go from laptop prototype to cloud in no time.
• aiWARE - aiWARE helps MLOps teams evaluate, deploy, integrate, scale & monitor ML models.
• Allegro AI - Transform ML/DL research into products. Faster.
• Polyaxon - A platform for reproducible and scalable machine learning and deep learning on kubernetes.
• SigOpt - A platform that makes it easy to track runs, visualize training, and scale hyperparameter tuning.
• TrueFoundry - A Cloud-native MLOps Platform over Kubernetes to simplify training and serving of ML Models.

Model Fairness and Privacy

• AIF360 - A comprehensive set of fairness metrics for datasets and machine learning models.
• Fairlearn - A Python package to assess and improve fairness of machine learning models.
• Opacus - A library that enables training PyTorch models with differential privacy.
• TensorFlow Privacy - Library for training machine learning models with privacy for training data.

Model Interpretability

• Alibi - Open-source Python library enabling ML model inspection and interpretation.
• Captum - Model interpretability and understanding library for PyTorch.
• ELI5 - Python package which helps to debug machine learning classifiers and explain their predictions.
• InterpretML - A toolkit to help understand models and enable responsible machine learning.
• LIME - Explaining the predictions of any machine learning classifier.
• Lucid - Collection of infrastructure and tools for research in neural network interpretability.
• SAGE - For calculating global feature importance using Shapley values.
• SHAP - A game theoretic approach to explain the output of any machine learning model.

Model Lifecycle

• Guild AI - Open source experiment tracking, pipeline automation, and hyperparameter tuning.
• MLflow - Open source platform for the machine learning lifecycle.
• Neptune AI - The most lightweight experiment management tool that fits any workflow.
• Aeromancy - A framework for performing reproducible AI and ML for Weights and Biases.
• Aim - A super-easy way to record, search and compare 1000s of ML training runs.
• Cascade - Library of ML-Engineering tools for rapid prototyping and experiment management.
• Keepsake - Version control for machine learning with support to Amazon S3 and Google Cloud Storage.
• Sacred - A tool to help you configure, organize, log and reproduce experiments.
• Comet - Track your datasets, code changes, experimentation history, and models.
• ModelDB - Open source ML model versioning, metadata, and experiment management.
• Weights and Biases - A tool for visualizing and tracking your machine learning experiments.
• Losswise - Makes it easy to track the progress of a machine learning project.

Model Serving

• Cortex - Machine learning model serving infrastructure.
• TensorFlow Serving - Flexible, high-performance serving system for ML models, designed for production.
• Wallaroo.AI - A platform for deploying, serving, and optimizing ML models in both cloud and edge environments.
• BentoML - Open-source platform for high-performance ML model serving.
• BudgetML - Deploy a ML inference service on a budget in less than 10 lines of code.
• Cog - Open-source tool that lets you package ML models in a standard, production-ready container.
• Gradio - Create customizable UI components around your models.
• Hydrosphere - Platform for deploying your Machine Learning to production.
• LocalAI - Drop-in replacement REST API that’s compatible with OpenAI API specifications for inferencing.
• MLEM - Version and deploy your ML models following GitOps principles.
• Opyrator - Turns your ML code into microservices with web API, interactive GUI, and more.
• PredictionIO - Event collection, deployment of algorithms, evaluation, querying predictive results via APIs.
• Rune - Provides containers to encapsulate and deploy EdgeML pipelines and applications.
• Streamlit - Lets you create apps for your ML projects with deceptively simple Python scripts.
• TorchServe - A flexible and easy to use tool for serving PyTorch models.
• Triton Inference Server - Provides an optimized cloud and edge inferencing solution.
• Vespa - Store, search, organize and make machine-learned inferences over big data at serving time.