Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

awesome-open-source-mlops

An awesome & curated list of best open source MLOps tools for data scientists.
https://github.com/gaocegege/awesome-open-source-mlops

  • code server - server.svg?style=social) - Run VS Code on any machine anywhere and access it in the browser.
  • conda - OS-agnostic, system-level binary package manager and ecosystem.
  • Docker - Moby is an open-source project created by Docker to enable and accelerate software containerization.
  • Jupyter Notebooks - The Jupyter notebook is a web-based notebook environment for interactive computing.
  • Caffe - A fast open framework for deep learning.
  • ColossalAI - An integrated large-scale model training system with efficient parallelization techniques.
  • DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
  • Horovod - Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
  • Jax - Autograd and XLA for high-performance machine learning research.
  • Kedro - org/kedro.svg?style=social) - Kedro is an open-source Python framework for creating reproducible, maintainable and modular data science code.
  • Keras - team/keras.svg?style=social) - Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow.
  • LightGBM - A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
  • MegEngine - MegEngine is a fast, scalable and easy-to-use deep learning framework, with auto-differentiation.
  • MindSpore - ai/mindspore.svg?style=social) - MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
  • MXNet - mxnet.svg?style=social) - Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler.
  • Oneflow - Inc/oneflow.svg?style=social) - OneFlow is a performance-centered and open-source deep learning framework.
  • PaddlePaddle - Machine Learning Framework from Industrial Practice.
  • PyTorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration.
  • PyTorchLightning - lightning.svg?style=social) - The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.
  • XGBoost - Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library.
  • TensorFlow - An Open Source Machine Learning Framework for Everyone.
  • VectorFlow - A minimalist neural network library optimized for sparse data and single machine environments.
  • Aim - an easy-to-use and performant open-source experiment tracker.
  • Guild AI - Experiment tracking, ML developer tools.
  • MLRun - Machine Learning automation and tracking.
  • Kedro-Viz - Kedro-Viz is an interactive development tool for building data science pipelines with Kedro. Kedro-Viz also allows users to view and compare different runs in the Kedro project.
  • LabNotebook - LabNotebook is a tool that allows you to flexibly monitor, record, save, and query all your machine learning experiments.
  • Sacred - Sacred is a tool to help you configure, organize, log and reproduce experiments.
  • Maniford - A model-agnostic visual debugging tool for machine learning.
  • netron - Visualizer for neural network, deep learning, and machine learning models.
  • OpenOps - Bring multiple data streams into one dashboard.
  • TensorBoard - TensorFlow's Visualization Toolkit.
  • TensorSpace - team/tensorspace.svg?style=social) - Neural network 3D visualization framework, build interactive and intuitive model in browsers, support pre-trained deep learning models from TensorFlow, Keras, TensorFlow.js.
  • dtreeviz - A python library for decision tree visualization and model interpretation.
  • Zetane Viewer - ML models and internal tensors 3D visualizer.
  • dvc - Data Version Control | Git for Data & Models | ML Experiments Management
  • ModelDB - Open Source ML Model Versioning, Metadata, and Experiment Management
  • ormb - Docker for Your ML/DL Models Based on OCI Artifacts
  • HuggingFace - State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
  • PaddleNLP - Easy-to-use and Fast NLP library with awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications.
  • PyTorch Image Models - image-models.svg?style=social) - PyTorch image models, scripts, pretrained weights.
  • BentoML - The Unified Model Serving Framework
  • ForestFlow - Policy-driven Machine Learning Model Server.
  • MOSEC - A machine learning model serving framework with dynamic batching and pipelined stages, provides an easy-to-use Python interface.
  • Multi Model Server - model-server.svg?style=social) - Multi Model Server is a tool for serving neural net models for inference.
  • Neuropod - A uniform interface to run deep learning models from multiple frameworks
  • Pinferencia - Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
  • Service Streamer - streamer.svg?style=social) - Boosting your Web Services of Deep Learning Applications.
  • TFServing - A flexible, high-performance serving system for machine learning models.
  • Torchserve - Serve, optimize and scale PyTorch models in production
  • Triton Server (TRTIS) - inference-server/server.svg?style=social) - The Triton Inference Server provides an optimized cloud and edge inferencing solution.
  • FeatherCNN - FeatherCNN is a high performance inference engine for convolutional neural networks.
  • Forward - A library for high performance deep learning inference on NVIDIA GPUs.
  • NCNN - ncnn is a high-performance neural network inference framework optimized for the mobile platform.
  • PocketFlow - use AutoML to do model compression.
  • TNN - A uniform deep learning inference framework for mobile, desktop and server.
  • ClearML - Auto-Magical CI/CD to streamline your ML workflow. Experiment Manager, MLOps and Data-Management.
  • MLflow - Open source platform for the machine learning lifecycle.
  • Kserve - Standardized Serverless ML Inference Platform on Kubernetes
  • Kubeflow - Machine Learning Toolkit for Kubernetes.
  • PAI - Resource scheduling and cluster management for AI.
  • Polyaxon - Machine Learning Management & Orchestration Platform.
  • Seldon-core - core.svg?style=social) - An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
  • Argo - workflows.svg?style=social) - Workflow engine for Kubernetes.
  • Flyte - Kubernetes-native workflow automation platform for complex, mission-critical data and ML processes at scale.
  • Kubeflow - Machine Learning Toolkit for Kubernetes.
  • Metaflow - Build and manage real-life data science projects with ease!
  • ZenML - io/zenml.svg?style=social) - MLOps framework to create reproducible pipelines.
  • Kueue - sigs/kueue.svg?style=social) - Kubernetes-native Job Queueing.
  • PAI - Resource scheduling and cluster management for AI (Open-sourced by Microsoft).
  • Slurm - A Highly Scalable Workload Manager.
  • Volcano - sh/volcano.svg?style=social) - A Cloud Native Batch System (Project under CNCF).
  • Yunikorn - core.svg?style=social) - Light-weight, universal resource scheduler for container orchestrator systems.
  • Adanet - Tensorflow package for AdaNet.
  • Advisor - open-source implementation of Google Vizier for hyper parameters tuning.
  • Archai - a platform for Neural Network Search (NAS) that allows you to generate efficient deep networks for your applications.
  • auptimizer - ARC-AdvancedAI/auptimizer.svg?style=social) - An automatic ML model optimization tool.
  • autoai - A framework to find the best performing AI/ML model for any AI problem.
  • AutoGL - An autoML framework & toolkit for machine learning on graphs
  • AutoGluon - AutoML for Image, Text, and Tabular Data.
  • automl-gs - gs.svg?style=social) - Provide an input CSV and a target field to predict, generate a model + code to run it.
  • autokeras - team/autokeras.svg?style=social) - AutoML library for deep learning.
  • Auto-PyTorch - PyTorch.svg?style=social) - Automatic architecture search and hyperparameter optimization for PyTorch.
  • auto-sklearn - sklearn.svg?style=social) - an automated machine learning toolkit and a drop-in replacement for a scikit-learn estimator.
  • AutoWeka - hyperparameter search for Weka.
  • Chocolate - Labs/chocolate.svg?style=social) - A fully decentralized hyperparameter optimization framework.
  • Dragonfly - An open source python library for scalable Bayesian optimisation.
  • Determined - ai/determined.svg?style=social) - scalable deep learning training platform with integrated hyperparameter tuning support; includes Hyperband, PBT, and other search methods.
  • DEvol (DeepEvolution) - a basic proof of concept for genetic architecture search in Keras.
  • EvalML - An open source python library for AutoML.
  • FEDOT - itmo/FEDOT.svg?style=social) - AutoML framework for the design of composite pipelines.
  • FLAML - Fast and lightweight AutoML ([paper](https://www.microsoft.com/en-us/research/publication/flaml-a-fast-and-lightweight-automl-library/)).
  • Goptuna - bata/goptuna.svg?style=social) - A hyperparameter optimization framework, inspired by Optuna.
  • HpBandSter - a framework for distributed hyperparameter optimization.
  • HPOlib2 - a library for hyperparameter optimization and black box optimization benchmarks.
  • Hyperband - open source code for tuning hyperparams with Hyperband.
  • Hypernets - A General Automated Machine Learning Framework.
  • Hyperopt - Distributed Asynchronous Hyperparameter Optimization in Python.
  • hyperunity - A toolset for black-box hyperparameter optimisation.
  • Katib - Katib is a Kubernetes-native project for automated machine learning (AutoML).
  • Keras Tuner - team/keras-tuner.svg?style=social) - Hyperparameter tuning for humans.
  • learn2learn - PyTorch Meta-learning Framework for Researchers.
  • Ludwig - a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code.
  • MOE - a global, black box optimization engine for real world metric optimization by Yelp.
  • Model Search - a framework that implements AutoML algorithms for model architecture search at scale.
  • NASGym - env.svg?style=social) - a proof-of-concept OpenAI Gym environment for Neural Architecture Search (NAS).
  • NNI - An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.
  • Optuna - A hyperparameter optimization framework.
  • REMBO - Bayesian optimization in high-dimensions via random embedding.
  • RoBO - a Robust Bayesian Optimization framework.
  • scikit-optimize(skopt) - optimize/scikit-optimize.svg?style=social) - Sequential model-based optimization with a `scipy.optimize` interface.
  • Spearmint - a software package to perform Bayesian optimization.
  • TPOT - one of the very first AutoML methods and open-source software packages.
  • Torchmeta - meta.svg?style=social) - A Meta-Learning library for PyTorch.
  • Vegas - noah/vega.svg?style=social) - an AutoML algorithm tool chain by Huawei Noah's Arb Lab.
  • Dolt - Git for Data.
  • DVC - Data Version Control | Git for Data & Models | ML Experiments Management.
  • Hub - Hub is a dataset format with a simple API for creating, storing, and collaborating on AI datasets of any size.
  • Quilt - A self-organizing data hub for S3.
  • LakeFS - Git-like capabilities for your object storage.
  • FeatureTools - An open source python framework for automated feature engineering
  • Upgini - Free automated data & feature enrichment library for machine learning: automatically searches through thousands of ready-to-use features from public and community shared data sources and enriches your training dataset with only the accuracy improving features
Programming Languages
Python 55 C++ 13 Go 12 Jupyter Notebook 7 Java 4 TypeScript 3 JavaScript 3 Matlab 1 C 1 HTML 1
Keywords
machine-learning 57 deep-learning 37 python 27 pytorch 23 tensorflow 23 automl 20 data-science 16 mlops 16 kubernetes 15 hyperparameter-optimization 14 keras 12 ai 11 ml 10 inference 9 neural-architecture-search 9 gpu 8 hyperparameter-tuning 8 neural-network 7 scikit-learn 6 llm 6 automated-machine-learning 6 bayesian-optimization 6 golang 5 visualization 5 k8s 5 optimization 5 data-engineering 5 workflow 5 mxnet 5 jax 5 data-version-control 4 model-management 4 experiment-tracking 4 model-serving 4 kubeflow 4 artificial-intelligence 4 xgboost 4 distributed-training 4 jupyter 3 caffe 3 bert 3 serving 3 hyperparameter-search 3 notebook 3 docker 3 reproducibility 3 nlp 3 reproducible-research 3 computer-vision 3 distributed 3