awesome-mlops
:sunglasses: A curated list of awesome MLOps tools
https://github.com/kelvins/awesome-mlops
Last synced: 4 days ago
JSON representation
-
Data Exploration
- Pandas Profiling - Create HTML profiling reports from pandas DataFrame objects.
- Apache Zeppelin - Enables data-driven, interactive data analytics and collaborative documents.
- BambooLib - An intuitive GUI for Pandas DataFrames.
- DataPrep - Collect, clean and visualize your data in Python.
- Jupyter Notebook - Web-based notebook environment for interactive computing.
- Jupytext - Jupyter Notebooks as Markdown Documents, Julia, Python or R scripts.
- Polynote - The polyglot notebook with first-class Scala support.
-
Data Management
- Hub - A dataset format for creating, storing, and collaborating on AI datasets of any size.
- Milvus - An open source embedding vector similarity search engine powered by Faiss, NMSLIB and Annoy.
- Marquez - Collect, aggregate, and visualize a data ecosystem's metadata.
- Arrikto - Dead simple, ultra fast storage for the hybrid Kubernetes world.
- BlazingSQL - A lightweight, GPU accelerated, SQL engine for Python. Built on RAPIDS cuDF.
- Delta Lake - Storage layer that brings scalable, ACID transactions to Apache Spark and other engines.
- Dolt - SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.
- Dud - A lightweight CLI tool for versioning data alongside source code and building data pipelines.
- DVC - Management and versioning of datasets and machine learning models.
- Intake - A lightweight set of tools for loading and sharing data in data science projects.
- lakeFS - Repeatable, atomic and versioned data lake on top of object storage.
- Qdrant - An open source vector similarity search engine with extended filtering support.
- Quilt - A self-organizing data hub with S3 support.
-
Drift Detection
- Frouros - An open source Python library for drift detection in machine learning systems.
- Frouros - An open source Python library for drift detection in machine learning systems.
- Alibi Detect - An open source Python library focused on outlier, adversarial and drift detection.
- TorchDrift - A data and concept drift library for PyTorch.
-
AutoML
- H2O AutoML - Automates ML workflow, which includes automatic training and tuning of models.
- AutoPyTorch - Automatic architecture search and hyperparameter optimization for PyTorch.
- MindsDB - AI layer for databases that allows you to effortlessly develop, train and deploy ML models.
- MLBox - MLBox is a powerful Automated Machine Learning python library.
- Model Search - Framework that implements AutoML algorithms for model architecture search at scale.
- NNI - An open source AutoML toolkit for automate machine learning lifecycle.
- AutoGluon - Automated machine learning for image, text, tabular, time-series, and multi-modal data.
- AutoKeras - AutoKeras goal is to make machine learning accessible for everyone.
- AutoSKLearn - Automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator.
- EvalML - A library that builds, optimizes, and evaluates ML pipelines using domain-specific functions.
- FLAML - Finds accurate ML models automatically, efficiently and economically.
-
Data Catalog
- Apache Atlas - Provides open metadata management and governance capabilities to build a data catalog.
- Amundsen - Data discovery and metadata engine for improving the productivity when interacting with data.
- CKAN - Open-source DMS (data management system) for powering data hubs and data portals.
- Magda - A federated, open-source data catalog for all your big data and small data.
- Metacat - Unified metadata exploration API service for Hive, RDS, Teradata, Redshift, S3 and Cassandra.
- OpenMetadata - A Single place to discover, collaborate and get your data right.
- DataHub - LinkedIn's generalized metadata search & discovery tool.
-
Articles
- What Is MLOps?
- A Tour of End-to-End Machine Learning Platforms
- MLOps: Continuous delivery and automation pipelines in machine learning
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- Continuous Delivery for Machine Learning
- Machine Learning Operations (MLOps): Overview, Definition, and Architecture
- MLOps Roadmap: A Complete MLOps Career Guide
- Rules of Machine Learning: Best Practices for ML Engineering
- The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
- MLOps: Machine Learning as an Engineering Discipline
-
Books
- Building Machine Learning Pipelines
- Building Machine Learning Powered Applications
- Designing Machine Learning Systems
- Engineering MLOps
- Implementing MLOps in the Enterprise
- Introducing MLOps
- Kubeflow for Machine Learning
- Kubeflow Operations Guide
- Machine Learning Design Patterns
- ML Ops: Operationalizing Data Science
- Practical Deep Learning at Scale with MLflow
- Practical MLOps
- Production-Ready Applied Deep Learning
- Reliable Machine Learning
- The Machine Learning Solutions Architect Handbook
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- Beginning MLOps with MLFlow
- Deep Learning in Production
- Machine Learning Engineering in Action
- MLOps Engineering at Scale
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
- MLOps Lifecycle Toolkit
-
Events
-
Other Lists
-
Podcasts
-
Slack
-
Websites
-
Data Processing
- Airflow - Platform to programmatically author, schedule, and monitor workflows.
- Azkaban - Batch workflow job scheduler created at LinkedIn to run Hadoop jobs.
- Dagster - A data orchestrator for machine learning, analytics, and ETL.
- Hadoop - Framework that allows for the distributed processing of large data sets across clusters.
- OpenRefine - Power tool for working with messy data and improving it.
- Spark - Unified analytics engine for large-scale data processing.
-
Data Visualization
- Data Studio - Reporting solution for power users who want to go beyond the data and dashboards of GA.
- Dash - Analytical Web Apps for Python, R, Julia, and Jupyter.
- Facets - Visualizations for understanding and analyzing machine learning datasets.
- Grafana - Multi-platform open source analytics and interactive visualization web application.
- Lux - Fast and easy data exploration by automating the visualization and data analysis process.
- Metabase - The simplest, fastest way to get business intelligence and analytics to everyone.
- Redash - Connect to any data source, easily visualize, dashboard and share your data.
- SolidUI - AI-generated visualization prototyping and editing platform, support 2D and 3D models.
-
Feature Store
- Feast - End-to-end open source feature store for machine learning.
- Tecton - A fully-managed feature platform built to orchestrate the complete lifecycle of features.
- Butterfree - A tool for building feature stores. Transform your raw data into beautiful features.
- ByteHub - An easy-to-use feature store. Optimized for time-series data.
- Feathr - An enterprise-grade, high performance feature store.
- Featureform - A Virtual Feature Store. Turn your existing data infrastructure into a feature store.
-
Hyperparameter Tuning
- Optuna - Open source hyperparameter optimization framework to automate hyperparameter search.
- Tune - Python library for experiment execution and hyperparameter tuning at any scale.
- Scikit Optimize - Simple and efficient library to minimize expensive and noisy black-box functions.
- Talos - Hyperparameter Optimization for TensorFlow, Keras and PyTorch.
- Advisor - Open-source implementation of Google Vizier for hyper parameters tuning.
- Hyperas - A very simple wrapper for convenient hyperparameter optimization.
- Hyperopt - Distributed Asynchronous Hyperparameter Optimization in Python.
- Katib - Kubernetes-based system for hyperparameter tuning and neural architecture search.
- KerasTuner - Easy-to-use, scalable hyperparameter optimization framework.
-
Machine Learning Platform
- aiWARE - aiWARE helps MLOps teams evaluate, deploy, integrate, scale & monitor ML models.
- Algorithmia - Securely govern your machine learning operations with a healthy ML lifecycle.
- Allegro AI - Transform ML/DL research into products. Faster.
- Iguazio - Data science platform that automates MLOps with end-to-end machine learning pipelines.
- Katonic - Automate your cycle of intelligence with Katonic MLOps Platform.
- Kubeflow - Making deployments of ML workflows on Kubernetes simple, portable and scalable.
- Modzy - Deploy, connect, run, and monitor machine learning (ML) models in the enterprise and at the edge.
- Omnimizer - Simplifies and accelerates MLOps by bridging the gap between ML models and edge hardware.
- Pachyderm - Combines data lineage with end-to-end pipelines on Kubernetes, engineered for the enterprise.
- Sagemaker - Fully managed service that provides the ability to build, train, and deploy ML models quickly.
- Sematic - An open-source end-to-end pipelining tool to go from laptop prototype to cloud in no time.
- SigOpt - A platform that makes it easy to track runs, visualize training, and scale hyperparameter tuning.
- TrueFoundry - A Cloud-native MLOps Platform over Kubernetes to simplify training and serving of ML Models.
- Bodywork - Deploys machine learning projects developed in Python, to Kubernetes.
- CNVRG - An end-to-end machine learning platform to build and deploy AI models at scale.
- DAGsHub - A platform built on open source tools for data, model and pipeline management.
- Dataiku - Platform democratizing access to data and enabling enterprises to build their own path to AI.
- DataRobot - AI platform that democratizes data science and automates the end-to-end ML at scale.
- Edge Impulse - Platform for creating, optimizing, and deploying AI/ML algorithms for edge devices.
- envd - Machine learning development environment for data science and AI/ML engineering teams.
- FedML - Simplifies the workflow of federated learning anywhere at any scale.
- Gradient - Multicloud CI/CD and MLOps platform for machine learning teams.
- Hopsworks - Open-source platform for developing and operating machine learning models at scale.
- Knime - Create and productionize data science using one easy and intuitive environment.
- LynxKite - A complete graph data science platform for very large graphs and other datasets.
- ML Workspace - All-in-one web-based IDE specialized for machine learning and data science.
- MLReef - Open source MLOps platform that helps you collaborate, reproduce and share your ML work.
- Omnimizer - Simplifies and accelerates MLOps by bridging the gap between ML models and edge hardware.
- SAS Viya - Cloud native AI, analytic and data management platform that supports the analytics life cycle.
- Valohai - Takes you from POC to production while managing the whole model lifecycle.
- Neu.ro - MLOps platform that integrates open-source and proprietary tools into client-oriented systems.
- SigOpt - A platform that makes it easy to track runs, visualize training, and scale hyperparameter tuning.
- Omnimizer - Simplifies and accelerates MLOps by bridging the gap between ML models and edge hardware.
-
Model Lifecycle
- Comet - Track your datasets, code changes, experimentation history, and models.
- Neptune AI - The most lightweight experiment management tool that fits any workflow.
- Aeromancy - A framework for performing reproducible AI and ML for Weights and Biases.
- Aim - A super-easy way to record, search and compare 1000s of ML training runs.
- Cascade - Library of ML-Engineering tools for rapid prototyping and experiment management.
- Guild AI - Open source experiment tracking, pipeline automation, and hyperparameter tuning.
- Keepsake - Version control for machine learning with support to Amazon S3 and Google Cloud Storage.
- MLflow - Open source platform for the machine learning lifecycle.
- Sacred - A tool to help you configure, organize, log and reproduce experiments.
- Comet - Track your datasets, code changes, experimentation history, and models.
- ModelDB - Open source ML model versioning, metadata, and experiment management.
- Weights and Biases - A tool for visualizing and tracking your machine learning experiments.
-
Model Serving
- Banana - Host your ML inference code on serverless GPUs and integrate it into your app with one line of code.
- Beam - Develop on serverless GPUs, deploy highly performant APIs, and rapidly prototype ML models.
- Quix - Serverless platform for processing data streams in real-time with machine learning models.
- GraphPipe - Machine learning model deployment made simple.
- Merlin - A platform for deploying and serving machine learning models.
- Geniusrise - Host inference APIs, bulk inference and fine tune text, vision, audio and multi-modal models.
- BentoML - Open-source platform for high-performance ML model serving.
- BudgetML - Deploy a ML inference service on a budget in less than 10 lines of code.
- Cog - Open-source tool that lets you package ML models in a standard, production-ready container.
- Cortex - Machine learning model serving infrastructure.
- Gradio - Create customizable UI components around your models.
- Hydrosphere - Platform for deploying your Machine Learning to production.
- LocalAI - Drop-in replacement REST API that’s compatible with OpenAI API specifications for inferencing.
- MLEM - Version and deploy your ML models following GitOps principles.
- Opyrator - Turns your ML code into microservices with web API, interactive GUI, and more.
- PredictionIO - Event collection, deployment of algorithms, evaluation, querying predictive results via APIs.
- Rune - Provides containers to encapsulate and deploy EdgeML pipelines and applications.
- Seldon - Take your ML projects from POC to production with maximum efficiency and minimal risk.
- Streamlit - Lets you create apps for your ML projects with deceptively simple Python scripts.
- TensorFlow Serving - Flexible, high-performance serving system for ML models, designed for production.
- TorchServe - A flexible and easy to use tool for serving PyTorch models.
- Triton Inference Server - Provides an optimized cloud and edge inferencing solution.
- Vespa - Store, search, organize and make machine-learned inferences over big data at serving time.
- Wallaroo.AI - A platform for deploying, serving, and optimizing ML models in both cloud and edge environments.
-
Optimization Tools
- Dask - Provides advanced parallelism for analytics, enabling performance at scale for the tools you love.
- Singa - Apache top level project, focusing on distributed training of DL and ML models.
- Accelerate - A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision.
- Mahout - Distributed linear algebra framework and mathematically expressive Scala DSL.
- MLlib - Apache Spark's scalable machine learning library.
- Modin - Speed up your Pandas workflows by changing a single line of code.
- Nos - Open-source module for running AI workloads on Kubernetes in an optimized way.
- Petastorm - Enables single machine or distributed training and evaluation of deep learning models.
- Rapids - Gives the ability to execute end-to-end data science and analytics pipelines entirely on GPUs.
- Ray - Fast and simple framework for building and running distributed applications.
- Tpot - Automated ML tool that optimizes machine learning pipelines using genetic programming.
- Nebullvm - Easy-to-use library to boost AI inference.
- DeepSpeed - Deep learning optimization library that makes distributed training easy, efficient, and effective.
- Fiber - Python distributed computing library for modern computer clusters.
- Horovod - Distributed deep learning training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
-
Simplification Tools
- Chassis - Turns models into ML-friendly containers that run just about anywhere.
- Hermione - Help Data Scientists on setting up more organized codes, in a quicker and simpler way.
- Hydra - A framework for elegantly configuring complex applications.
- Koalas - Pandas API on Apache Spark. Makes data scientists more productive when interacting with big data.
- Ludwig - Allows users to train and test deep learning models without the need to write code.
- MLNotify - No need to keep checking your training, just one import line and you'll know the second it's done.
- PyCaret - Open source, low-code machine learning library in Python.
- Sagify - A CLI utility to train and deploy ML/DL models on AWS SageMaker.
- Soopervisor - Export ML projects to Kubernetes (Argo workflows), Airflow, AWS Batch, and SLURM.
- Soorgeon - Convert monolithic Jupyter notebooks into maintainable pipelines.
- TrainGenerator - A web app to generate template code for machine learning.
- Turi Create - Simplifies the development of custom machine learning models.
-
Visual Analysis and Debugging
- Superwise - Fully automated, enterprise-grade model observability in a self-service SaaS platform.
- Aporia - Observability with customized monitoring and explainability for ML models.
- Evidently - Interactive reports to analyze ML models during validation or production monitoring.
- Fiddler - Monitor, explain, and analyze your AI in production.
- Manifold - A model-agnostic visual debugging tool for machine learning.
- NannyML - Algorithm capable of fully capturing the impact of data drift on performance.
- Netron - Visualizer for neural network, deep learning, and machine learning models.
- Opik - Evaluate, test, and ship LLM applications with a suite of observability tools.
- Radicalbit - The open source solution for monitoring your AI models in production.
- Whylogs - The open source standard for data logging. Enables ML monitoring and observability.
- Yellowbrick - Visual analysis and diagnostic tools to facilitate machine learning model selection.
-
CI/CD for Machine Learning
-
Cron Job Monitoring
- Cronitor - Monitor any cron job or scheduled task.
- HealthchecksIO - Simple and effective cron job monitoring.
-
Data Enrichment
-
Data Validation
- Cerberus - Lightweight, extensible data validation library for Python.
- Cleanlab - Python library for data-centric AI and machine learning with messy, real-world data and labels.
- JSON Schema - A vocabulary that allows you to annotate and validate JSON documents.
- TFDV - An library for exploring and validating machine learning data.
-
Feature Engineering
- Feature Engine - Feature engineering package with SKlearn like functionality.
- Featuretools - Python library for automated feature engineering.
- TSFresh - Python library for automatic extraction of relevant features from time series.
-
Knowledge Sharing
- Knowledge Repo - Knowledge sharing platform for data scientists and other technical professions.
-
Model Fairness and Privacy
- AIF360 - A comprehensive set of fairness metrics for datasets and machine learning models.
- Fairlearn - A Python package to assess and improve fairness of machine learning models.
- Opacus - A library that enables training PyTorch models with differential privacy.
- TensorFlow Privacy - Library for training machine learning models with privacy for training data.
-
Model Interpretability
- Alibi - Open-source Python library enabling ML model inspection and interpretation.
- Captum - Model interpretability and understanding library for PyTorch.
- ELI5 - Python package which helps to debug machine learning classifiers and explain their predictions.
- InterpretML - A toolkit to help understand models and enable responsible machine learning.
- LIME - Explaining the predictions of any machine learning classifier.
- Lucid - Collection of infrastructure and tools for research in neural network interpretability.
- SAGE - For calculating global feature importance using Shapley values.
- SHAP - A game theoretic approach to explain the output of any machine learning model.
-
Model Testing & Validation
- Deepchecks - Open-source package for validating ML models & data, with various checks and suites.
- Starwhale - An MLOps/LLMOps platform for model building, evaluation, and fine-tuning.
-
Workflow Tools
- Automate Studio - Rapidly build & deploy AI-powered workflows.
- dstack - An open-core tool to automate data and training workflows.
- Flyte - Easy to create concurrent, scalable, and maintainable workflows for machine learning.
- Hamilton - A scalable general purpose micro-framework for defining dataflows.
- Kale - Aims at simplifying the Data Science experience of deploying Kubeflow Pipelines workflows.
- Kedro - Library that implements software engineering best-practice for data and ML pipelines.
- Luigi - Python module that helps you build complex pipelines of batch jobs.
- Metaflow - Human-friendly lib that helps scientists and engineers build and manage data science projects.
- MLRun - Generic mechanism for data scientists to build, run, and monitor ML tasks and pipelines.
- Ploomber - Write maintainable, production-ready pipelines. Develop locally, deploy to the cloud.
- Argo - Open source container-native workflow engine for orchestrating parallel jobs on Kubernetes.
- Couler - Unified interface for constructing and managing workflows on different workflow engines.
Programming Languages
Categories
Books
58
Articles
38
Machine Learning Platform
33
Model Serving
24
Optimization Tools
15
Data Management
13
Simplification Tools
12
Model Lifecycle
12
Workflow Tools
12
Other Lists
12
Visual Analysis and Debugging
11
AutoML
11
Hyperparameter Tuning
9
Data Visualization
8
Websites
8
Model Interpretability
8
Events
7
Data Catalog
7
Podcasts
7
Data Exploration
7
Feature Store
6
Data Processing
6
Model Fairness and Privacy
4
Drift Detection
4
Data Validation
4
CI/CD for Machine Learning
3
Feature Engineering
3
Data Enrichment
2
Cron Job Monitoring
2
Slack
2
Model Testing & Validation
2
Knowledge Sharing
1
Sub Categories
Keywords
machine-learning
81
data-science
50
python
49
deep-learning
37
mlops
28
pytorch
21
tensorflow
18
automl
15
ai
15
scikit-learn
14
data-engineering
13
kubernetes
12
ml
11
hyperparameter-optimization
11
llm
10
feature-engineering
9
jupyter-notebook
9
keras
8
automated-machine-learning
8
artificial-intelligence
8
interpretability
7
optimization
7
gpu
7
data-analysis
7
analytics
7
data-visualization
7
data-quality
7
visualization
7
deeplearning
6
machinelearning
6
pandas
6
neural-network
6
llmops
6
hyperparameter-tuning
5
distributed
5
workflow
5
docker
5
model-serving
5
jupyter
5
serving
5
llms
5
awesome-list
5
awesome
5
natural-language-processing
5
data-drift
5
feature-selection
4
developer-tools
4
deployment
4
dataset
4
inference
4