{"id":13411673,"url":"https://github.com/ahkarami/Deep-Learning-in-Production","last_synced_at":"2025-03-14T17:31:01.133Z","repository":{"id":32232969,"uuid":"131979161","full_name":"ahkarami/Deep-Learning-in-Production","owner":"ahkarami","description":"In this repository, I will share some useful notes and references about deploying deep learning-based models in production.","archived":false,"fork":false,"pushed_at":"2024-10-02T14:18:23.000Z","size":311,"stargazers_count":4304,"open_issues_count":7,"forks_count":684,"subscribers_count":148,"default_branch":"master","last_synced_at":"2024-10-29T14:50:30.932Z","etag":null,"topics":["angularjs","c-plus-plus","caffe2","convert-pytorch-models","deep-learning","deep-neural-networks","flask","keras","model-serving","mxnet","production","python","pytorch","react","rest-api","serving","serving-pytorch-models","tensorflow-models","tesnorflow","tutorial"],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ahkarami.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-03T10:24:24.000Z","updated_at":"2024-10-27T19:26:30.000Z","dependencies_parsed_at":"2023-02-14T18:16:18.015Z","dependency_job_id":"667e2618-d67c-4e95-914e-f363deee7f30","html_url":"https://github.com/ahkarami/Deep-Learning-in-Production","commit_stats":{"total_commits":217,"total_committers":3,"mean_commits":72.33333333333333,"dds":0.009216589861751112,"last_synced_commit":"e27639cc96e599144cd673e52bc6256fc785aa59"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahkarami%2FDeep-Learning-in-Production","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahkarami%2FDeep-Learning-in-Production/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahkarami%2FDeep-Learning-in-Production/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahkarami%2FDeep-Learning-in-Production/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ahkarami","download_url":"https://codeload.github.com/ahkarami/Deep-Learning-in-Production/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243159178,"owners_count":20245674,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["angularjs","c-plus-plus","caffe2","convert-pytorch-models","deep-learning","deep-neural-networks","flask","keras","model-serving","mxnet","production","python","pytorch","react","rest-api","serving","serving-pytorch-models","tensorflow-models","tesnorflow","tutorial"],"created_at":"2024-07-30T20:01:15.632Z","updated_at":"2025-03-14T17:31:01.120Z","avatar_url":"https://github.com/ahkarami.png","language":null,"readme":"# Deep-Learning-in-Production\nIn this repository, I will share some useful notes and references about deploying deep learning-based models in production.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./Final-Logo.jpg?raw=true\" alt=\"Logo\"/\u003e\n\u003c/p\u003e\n\n## Convert PyTorch Models in Production:\n- [PyTorch Production Level Tutorials](https://pytorch.org/tutorials/#production-usage) [_Fantastic_]  \n- [The road to 1.0: production ready PyTorch](https://pytorch.org/2018/05/02/road-to-1.0.html)\n- [PyTorch 1.0 tracing JIT and LibTorch C++ API to integrate PyTorch into NodeJS](http://blog.christianperone.com/2018/10/pytorch-1-0-tracing-jit-and-libtorch-c-api-to-integrate-pytorch-into-nodejs/) [_Good Article_]\n- [Model Serving in PyTorch](https://pytorch.org/blog/model-serving-in-pyorch/)\n- [PyTorch Summer Hackathon](https://pytorch.devpost.com/) [_Very Important_]\n- [Deploying PyTorch and Building a REST API using Flask](https://pytorch.org/tutorials/intermediate/flask_rest_api_tutorial.html) [_Important_]\n- [PyTorch model recognizing hotdogs and not-hotdogs deployed on flask](https://github.com/jaroslaw-weber/hotdog-not-hotdog)\n- [Serving PyTorch 1.0 Models as a Web Server in C++ ](https://github.com/Wizaron/pytorch-cpp-inference) [_Useful Example_]\n- [PyTorch Internals](http://blog.ezyang.com/2019/05/pytorch-internals/)  [_Interesting \u0026 Useful Article_]  \n- [Flask application to support pytorch model prediction](https://github.com/craigsidcarlson/PytorchFlaskApp)\n- [Serving PyTorch Model on Flask Thread-Safety](https://discuss.pytorch.org/t/serving-pytorch-model-on-flask-thread-safety/13921)\n- [Serving PyTorch Models on AWS Lambda with Caffe2 \u0026 ONNX](https://machinelearnings.co/serving-pytorch-models-on-aws-lambda-with-caffe2-onnx-7b096806cfac)\n- [Serving PyTorch Models on AWS Lambda with Caffe2 \u0026 ONNX (Another Version)](https://blog.waya.ai/deploy-deep-machine-learning-in-production-the-pythonic-way-a17105f1540e)\n- [Deep Dive into ONNX Runtime](https://medium.com/@mohsen.mahmoodzadeh/a-deep-dive-into-onnx-onnx-runtime-part-1-874517c66ffc)  \n- [EuclidesDB - _multi-model machine learning feature database with PyTorch_](https://euclidesdb.readthedocs.io/en/latest/)\n- [EuclidesDB - GitHub](https://github.com/perone/euclidesdb/)\n- [WebDNN: Fastest DNN Execution Framework on Web Browser](https://github.com/mil-tokyo/webdnn)\n- [FastAI PyTorch Serverless API (with AWS Lambda)](https://github.com/alecrubin/pytorch-serverless/)\n- [FastAI PyTorch in Production (discussion)](http://forums.fast.ai/t/fastai-pytorch-in-production/16928)   \n- [OpenMMLab Model Deployment Framework](https://github.com/open-mmlab/mmdeploy)\n- [PyTorch Native Architecture Optimization: torchao](https://pytorch.org/blog/pytorch-native-architecture-optimization/?utm_content=309679619\u0026utm_medium=social\u0026utm_source=twitter\u0026hss_channel=tw-776585502606721024)  \n- [TorchServe](https://github.com/pytorch/serve) [Great Tool]    \n- [TorchServe Video Tutorial](https://www.youtube.com/watch?v=XlO7iQMV3Ik)\n- [LitServe: Easily serve AI models Lightning fast](https://github.com/Lightning-AI/litserve)  \n\n## Convert PyTorch Models to C++:\n- [**Loading a PyTorch Model in C++**](https://pytorch.org/tutorials/advanced/cpp_export.html) [_**Fantastic**_]\n- [**PyTorch C++ API**](https://pytorch.org/cppdocs/index.html) [_Bravo_]\n- [An Introduction To Torch (Pytorch) C++ Front-End](https://radicalrafi.github.io/posts/pytorch-cpp-intro/) [_Very Good_]\n- [Blogs on using PyTorch C++ API](https://discuss.pytorch.org/t/a-series-of-blogs-on-pytorch-c-api-transfer-learning-jupyter-notebook-with-libtorch-xeus-cling-and-more/54628) [_Good_]\n- [ATen: A TENsor library](https://github.com/pytorch/pytorch/tree/master/aten)\n- [Important Issue about PyTorch-like C++ interface](https://github.com/pytorch/pytorch/issues/3335)\n- [PyTorch C++ API Test](https://github.com/pytorch/pytorch/tree/master/test/cpp/api)\n- [PyTorch via C++](https://discuss.pytorch.org/t/pytorch-via-c/19234) [_Useful Notes_]\n- [AUTOGRADPP](https://github.com/pytorch/pytorch/tree/master/torch/csrc/api)\n- [PyTorch C++ Library](https://github.com/warmspringwinds/pytorch-cpp)\n- [Direct C++ Interface to PyTorch](https://github.com/ebetica/autogradpp)\n- [A Python module for compiling PyTorch graphs to C](https://github.com/lantiga/pytorch2c)\n\n## Deploy TensorFlow Models in Production:\n- [How to deploy Machine Learning models with TensorFlow - _Part1_](https://towardsdatascience.com/how-to-deploy-machine-learning-models-with-tensorflow-part-1-make-your-model-ready-for-serving-776a14ec3198)\n- [How to deploy Machine Learning models with TensorFlow - _Part2_](https://towardsdatascience.com/how-to-deploy-machine-learning-models-with-tensorflow-part-2-containerize-it-db0ad7ca35a7)\n- [How to deploy Machine Learning models with TensorFlow - _Part3_](https://towardsdatascience.com/how-to-deploy-machine-learning-models-with-tensorflow-part-3-into-the-cloud-7115ff774bb6)\n- [Neural Structured Learning (NSL) in TensorFlow](https://medium.com/tensorflow/introducing-neural-structured-learning-in-tensorflow-5a802efd7afd) [_Great_]\n- [Building Robust Production-Ready Deep Learning Vision Models](https://medium.com/google-developer-experts/building-robust-production-ready-deep-learning-vision-models-in-minutes-acd716f6450a)  \n- [Creating REST API for TensorFlow models](https://becominghuman.ai/creating-restful-api-to-tensorflow-models-c5c57b692c10)\n- [\"How to Deploy a Tensorflow Model in Production\" by _Siraj Raval_ on YouTube](https://www.youtube.com/watch?v=T_afaArR0E8)\n- [Code for the \"How to Deploy a Tensorflow Model in Production\" by _Siraj Raval_ on YouTube](https://github.com/llSourcell/How-to-Deploy-a-Tensorflow-Model-in-Production)\n- [How to deploy an Object Detection Model with TensorFlow serving](https://medium.freecodecamp.org/how-to-deploy-an-object-detection-model-with-tensorflow-serving-d6436e65d1d9) [_Very Good Tutorial_]\n- [Freeze Tensorflow models and serve on web](http://cv-tricks.com/how-to/freeze-tensorflow-models/) [_Very Good Tutorial_]\n- [How to deploy TensorFlow models to production using TF Serving](https://medium.freecodecamp.org/how-to-deploy-tensorflow-models-to-production-using-tf-serving-4b4b78d41700) [_Good_]\n- [How Zendesk Serves TensorFlow Models in Production](https://medium.com/zendesk-engineering/how-zendesk-serves-tensorflow-models-in-production-751ee22f0f4b)\n- [TensorFlow Serving Example Projects](https://github.com/Vetal1977/tf_serving_example)\n- [Serving Models in Production with TensorFlow Serving](https://www.youtube.com/watch?v=q_IkJcPyNl0) [_TensorFlow Dev Summit 2017 Video_]\n- [Building TensorFlow as a Standalone Project](https://tuatini.me/building-tensorflow-as-a-standalone-project/)\n- [TensorFlow C++ API Example](https://github.com/jhjin/tensorflow-cpp)\n- [TensorFlow.js](https://js.tensorflow.org/)\n- [Introducing TensorFlow.js: Machine Learning in Javascript](https://medium.com/tensorflow/introducing-tensorflow-js-machine-learning-in-javascript-bf3eab376db)\n\n## Convert Keras Models in Production:\n- [Deep learning in production with Keras, Redis, Flask, and Apache](https://www.pyimagesearch.com/2018/02/05/deep-learning-production-keras-redis-flask-apache/) [_Rank: 1st \u0026 General Usefult Tutorial_]\n- [Deploying a Keras Deep Learning Model as a Web Application in Python](https://towardsdatascience.com/deploying-a-keras-deep-learning-model-as-a-web-application-in-p-fc0f2354a7ff) [_Very Good_]\n- [Deploying a Python Web App on AWS](https://towardsdatascience.com/deploying-a-python-web-app-on-aws-57ed772b2319) [_Very Good_]\n- [Deploying Deep Learning Models Part 1: Preparing the Model](https://blog.paperspace.com/deploying-deep-learning-models-flask-web-python/)  \n- [Deploying your Keras model](https://medium.com/@burgalon/deploying-your-keras-model-35648f9dc5fb)\n- [Deploying your Keras model using Keras.JS](https://becominghuman.ai/deploying-your-keras-model-using-keras-js-2e5a29589ad8)\n- [\"How to Deploy a Keras Model to Production\" by _Siraj Raval_ on Youtube](https://github.com/llSourcell/how_to_deploy_a_keras_model_to_production)\n- [Deploy Keras Model with Flask as Web App in 10 Minutes](https://github.com/mtobeiyf/keras-flask-deploy-webapp) [Good Repository]\n- [Deploying Keras Deep Learning Models with Flask](https://towardsdatascience.com/deploying-keras-deep-learning-models-with-flask-5da4181436a2)\n- [keras2cpp](https://github.com/pplonski/keras2cpp)\n\n## Deploy MXNet Models in Production:\n- [Model Server for Apache MXNet](https://github.com/awslabs/mxnet-model-server)\n- [Running the Model Server](https://github.com/awslabs/mxnet-model-server/blob/master/docs/server.md)\n- [Multi Model Server (MMS) Documentation](https://github.com/awslabs/multi-model-server/tree/master/docs)  \n- [Introducing Model Server for Apache MXNet](https://aws.amazon.com/blogs/machine-learning/introducing-model-server-for-apache-mxnet/)  \n- [Single Shot Multi Object Detection Inference Service](https://github.com/awslabs/mxnet-model-server/blob/master/examples/ssd/README.md)\n- [Amazon SageMaker](https://aws.amazon.com/sagemaker/)\n- [How can we serve MXNet models built with gluon api](https://discuss.mxnet.io/t/how-can-we-serve-mxnet-models-built-with-gluon-api/684)\n- [MXNet C++ Package](https://github.com/apache/incubator-mxnet/tree/master/cpp-package)\n- [MXNet C++ Package Examples](https://github.com/apache/incubator-mxnet/tree/master/cpp-package/example)\n- [MXNet Image Classification Example of C++](https://github.com/apache/incubator-mxnet/tree/master/example/image-classification/predict-cpp)\n- [MXNet C++ Tutorial](http://mxnet.incubator.apache.org/tutorials/c%2B%2B/basics.html)\n- [An introduction to the MXNet API](https://becominghuman.ai/an-introduction-to-the-mxnet-api-part-1-848febdcf8ab) [Very Good Tutorial for Learning MXNet]\n- [GluonCV](https://gluon-cv.mxnet.io/)\n- [GluonNLP](http://gluon-nlp.mxnet.io/)\n- [Model Quantization for Production-Level Neural Network Inference](https://medium.com/apache-mxnet/model-quantization-for-production-level-neural-network-inference-f54462ebba05) [_Excellent_]\n\n## Deploy Machine Learning Models with Go:\n- [Cortex: Deploy machine learning models in production](https://github.com/cortexlabs/cortex)  \n- [Cortex - Main Page](https://www.cortex.dev/)  \n- [Why we deploy machine learning models with Go — not Python](https://towardsdatascience.com/why-we-deploy-machine-learning-models-with-go-not-python-a4e35ec16deb)  \n- [Go-Torch](https://github.com/orktes/go-torch)  \n- [Gotch - Go API for PyTorch](https://github.com/sugarme/gotch)  \n- [TensorFlow Go Lang](https://www.tensorflow.org/install/lang_go)  \n- [Go-onnx](https://github.com/dhdanie/goonnx)  \n\n## General Deep Learning Deployment Toolkits:\n- [OpenVINO Toolkit - Deep Learning Deployment Toolkit repository](https://github.com/openvinotoolkit/openvino) [_Great_]   \n- [ClearML - ML/DL development and production suite](https://github.com/allegroai/clearml)  \n- [Model Deployment Using Heroku: A Complete Guide on Heroku](https://www.analyticsvidhya.com/blog/2021/10/a-complete-guide-on-machine-learning-model-deployment-using-heroku/) [Good]   \n- [NVIDIA Triton Inference Server](https://developer.nvidia.com/nvidia-triton-inference-server) [**Great**]      \n- [NVIDIA Triton Inference Server - GitHub](https://github.com/triton-inference-server/server) [**Great**]   \n- [Cohere Boosts Inference Speed With NVIDIA Triton Inference Server](https://txt.cohere.ai/nvidia-boosts-inference-speed-with-cohere/)  \n- [NVIDIA Deep Learning Examples for Tensor Cores](https://github.com/NVIDIA/DeepLearningExamples) [Interesting]  \n- [Deploying the Jasper Inference model using Triton Inference Server](https://github.com/NVIDIA/DeepLearningExamples/tree/master/PyTorch/SpeechRecognition/Jasper/triton) [Useful]   \n- [Nvidia MLOPs Course via Triton](https://analyticsindiamag.com/nvidia-is-offering-a-four-hour-self-paced-course-on-mlops/)  \n- [Awesome Production Machine Learning](https://github.com/EthicalML/awesome-production-machine-learning) [Great]\n- [BentoML Model Serving](https://www.linkedin.com/posts/axelmdz_mlops-machinelearning-datascience-activity-7194637600861618176-iQaF?utm_source=share\u0026utm_medium=member_desktop)  \n\n## Huawei Deep Learning Framework:\n- [MindSpore - Huawei Deep Learning Framework](https://github.com/mindspore-ai/mindspore)  \n- [MindSpore - Tutorial](https://www.mindspore.cn/tutorial/en/0.1.0-alpha/quick_start/quick_start.html)  \n\n## General Deep Learning Compiler Stack:\n- [TVM Stack](https://tvm.ai/)\n\n## Model Conversion between Deep Learning Frameworks:\n- [ONNX (Open Neural Network Exchange)](https://onnx.ai/)\n- [Tutorials for using ONNX](https://github.com/onnx/tutorials)\n- [MMdnn](https://github.com/Microsoft/MMdnn) [_Fantastic_]  \n- [Convert Full ImageNet Pre-trained Model from MXNet to PyTorch](https://blog.paperspace.com/convert-full-imagenet-pre-trained-model-from-mxnet-to-pytorch/) [_Fantastic_, \u0026 Full ImageNet model means the model trained on ~ 14M images] \n\n## Some Caffe2 Tutorials:\n- [Mnist using caffe2](http://vast.uccs.edu/~adhamija/blog/MNIST_singleGPU.html)\n- [Caffe2 C++ Tutorials and Examples](https://github.com/leonardvandriel/caffe2_cpp_tutorial)\n- [Make Transfer Learning of SqueezeNet on Caffe2](https://medium.com/@KazamiXHayato/make-transfer-learning-in-caffe2-21d96c47ba0e)\n- [Build Basic program by using Caffe2 framework in C++](https://medium.com/@KazamiXHayato/write-caffe2-program-in-c-5519e2646382)\n\n## Some Useful Resources for Designing UI (Front-End Development):\n- [ReactJS vs Angular5 vs Vue.js](https://medium.com/@TechMagic/reactjs-vs-angular5-vs-vue-js-what-to-choose-in-2018-b91e028fa91d)\n- [A comparison between Angular and React and their core languages](https://medium.freecodecamp.org/a-comparison-between-angular-and-react-and-their-core-languages-9de52f485a76)\n- [A Guide to Becoming a Full-Stack Developer](https://medium.com/coderbyte/a-guide-to-becoming-a-full-stack-developer-in-2017-5c3c08a1600c) [_Very Good Tutorial_]\n- [Roadmap to becoming a web developer in 2018](https://github.com/kamranahmedse/developer-roadmap) [_Very Good Repository_]\n- [Modern Frontend Developer in 2018](https://medium.com/tech-tajawal/modern-frontend-developer-in-2018-4c2072fa2b9c)\n- [Roadmap to becoming a React developer in 2018](https://github.com/adam-golab/react-developer-roadmap)\n- [2019 UI and UX Design Trends](https://uxplanet.org/2019-ui-and-ux-design-trends-92dfa8323225) [_Good_]\n- [Streamlit](https://streamlit.io/) [_The fastest way to build custom ML tools_]  \n- [Gradio](https://www.gradio.app/) [**Good**]   \n- [Web Developer Monthly](https://medium.com/@andreineagoie/web-developer-monthly-july-2018-513e02f15fb6)\n- [23 Best React UI Component Frameworks](https://hackernoon.com/23-best-react-ui-component-libraries-and-frameworks-250a81b2ac42)\n- [9 React Styled-Components UI Libraries for 2018](https://blog.bitsrc.io/9-react-styled-components-ui-libraries-for-2018-4e1a0bd3e179)\n- [35 New Tools for UI Design](https://blog.prototypr.io/35-new-tools-for-ui-design-412cf1d701fd)\n- [5 Tools To Speed Up Your App Development](https://medium.com/swlh/5-tools-to-speed-up-your-app-development-6979d0e49e34) [_Very Good_]\n- [How to use ReactJS with Webpack 4, Babel 7, and Material Design](https://medium.freecodecamp.org/how-to-use-reactjs-with-webpack-4-babel-7-and-material-design-ff754586f618)\n- [Adobe Typekit](https://typekit.com/) [_Great fonts, where you need them_]\n- [Build A Real World Beautiful Web APP with Angular 6](https://medium.com/@hamedbaatour/build-a-real-world-beautiful-web-app-with-angular-6-a-to-z-ultimate-guide-2018-part-i-e121dd1d55e)\n- [You Don't Know JS](https://github.com/getify/You-Dont-Know-JS)\n- [JavaScript Top 10 Articles](https://medium.mybridge.co/javascript-top-10-articles-for-the-past-month-v-sep-2018-8f27a300d6c5)\n- [Web Design with Adobe XD](https://medium.freecodecamp.org/a-developers-guide-to-web-design-for-non-designers-1f64ce28c38d)\n- [INSPINIA Bootstrap Web Theme](https://wrapbootstrap.com/theme/inspinia-responsive-admin-theme-WB0R5L90S)\n- [A Learning Tracker for Front-End Developers](https://github.com/Syknapse/My-Learning-Tracker-first-ten-months)\n- [The best front-end hacking cheatsheets — all in one place](https://medium.freecodecamp.org/modern-frontend-hacking-cheatsheets-df9c2566c72a) [_Useful \u0026 Interesting_]\n- [GUI-fying the Machine Learning Workflow (Machine Flow)](https://towardsdatascience.com/gui-fying-the-machine-learning-workflow-towards-rapid-discovery-of-viable-pipelines-cab2552c909f)\n- [Electron - Build cross platform desktop apps with JavaScript](https://electronjs.org/) [_Very Good_]\n- [Opyrator - Turns Python functions into microservices with web API](https://github.com/ml-tooling/opyrator) [**Great**]    \n- [A First Look at PyScript: Python in the Web Browser](https://realpython.com/pyscript-python-in-browser/) [**Interesting**]  \n\n## Mobile \u0026 Embedded Devices Development:\n- [PyTorch Mobile](https://pytorch.org/mobile/home/) [_Excellent_]  \n- [Mobile UI Design Trends In 2018](https://uxplanet.org/mobile-ui-design-trends-in-2018-ccd26031dfd8)  \n- [ncnn - high-performance neural network inference framework optimized for the mobile platform](https://github.com/Tencent/ncnn) [_Useful_]  \n- [Alibaba - MNN](https://github.com/alibaba/MNN)  \n- [Awesome Mobile Machine Learning](https://github.com/fritzlabs/Awesome-Mobile-Machine-Learning)  \n- [EMDL - Embedded and Mobile Deep Learning](https://github.com/EMDL/awesome-emdl)  \n- [Fritz - machine learning platform for iOS and Android](https://www.fritz.ai/)  \n- [TensorFlow Lite](https://www.tensorflow.org/mobile/tflite/)  \n- [Tiny Machine Learning: The Next AI Revolution](https://medium.com/@matthew_stewart/tiny-machine-learning-the-next-ai-revolution-495c26463868)  \n- [TLT - NVIDIA Transfer Learning Toolkit](https://developer.nvidia.com/transfer-learning-toolkit)  \n- [NVIDIA Jetson Inference](https://github.com/dusty-nv/jetson-inference)  [_Great_]\n- [EfficientML.ai Lecture (MIT Course)](https://www.youtube.com/playlist?list=PL80kAHvQbh-pT4lCkDT53zT8DKmhE0idB)\n- [PyTorch Edge: Enabling On-Device Inference Across Mobile and Edge Devices with ExecuTorch](https://pytorch.org/blog/pytorch-edge/?utm_content=268296369\u0026utm_medium=social\u0026utm_source=linkedin\u0026hss_channel=lcp-78618366)  \n\n## Back-End Development Part:\n- [Modern Backend Developer in 2018](https://medium.com/tech-tajawal/modern-backend-developer-in-2018-6b3f7b5f8b9)\n- [Deploying frontend applications — the fun way](https://hackernoon.com/deploying-frontend-applications-the-fun-way-bc3f69e15331) [_Very Good_]\n- [RabbitMQ](https://www.rabbitmq.com/) [_Message Broker Software_]\n- [Celery](http://www.celeryproject.org/) [_Distributed Task Queue_]\n- [Kafka](https://kafka.apache.org/) [_Distributed Streaming Platform_]\n- [Docker training with DockerMe](https://github.com/AhmadRafiee/Docker_training_with_DockerMe)  \n- [Kubernetes - GitHub](https://github.com/kubernetes/kubernetes)\n- [Deploy Machine Learning Pipeline on Google Kubernetes Engine](https://towardsdatascience.com/deploy-machine-learning-model-on-google-kubernetes-engine-94daac85108b)  \n- [An introduction to Kubernetes for Data Scientists](https://www.jeremyjordan.me/kubernetes/)  \n- [Jenkins and Kubernetes with Docker Desktop](https://medium.com/@garunski/jenkins-and-kubernetes-with-docker-desktop-53a853486f7c)\n- [Helm: The package manager for Kubernetes](https://helm.sh/)  \n- [Create Cluster using docker swarm](https://medium.com/tech-tajawal/create-cluster-using-docker-swarm-94d7b2a10c43)  \n- [deepo - Docker Image for all DL Framewors](https://github.com/ufoym/deepo)  \n- [Kubeflow](https://www.kubeflow.org/)  [_deployments of ML workflows on Kubernetes_]  \n- [kubespray - Deploy a Production Ready Kubernetes Cluster](https://github.com/kubernetes-sigs/kubespray)  \n- [KFServing - Kubernetes for Serving ML Models](https://github.com/kubeflow/kfserving)  \n- [Deploying a HuggingFace NLP Model with KFServing](http://www.pattersonconsultingtn.com/blog/deploying_huggingface_with_kfserving.html) [_Interesting_]  \n- [Seldon Core - Deploying Machine Learning Models on Kubernetes](https://www.seldon.io/tech/products/core/)  \n- [Seldon Core - GitHub](https://github.com/SeldonIO/seldon-core)  \n- [Machine Learning: serving models with Kubeflow on Ubuntu, Part 1](https://ubuntu.com/blog/ml-serving-models-with-kubeflow-on-ubuntu-part-1)  \n- [CoreWeave Kubernetes Cloud](https://github.com/coreweave/kubernetes-cloud/tree/master/online-inference/)  \n- [MLOps References](https://github.com/visenger/mlops-references)  [_DevOps for ML_]\n- [Data Version Control - DVC](https://dvc.org/)  [_Great_]  \n- [MLEM: package and deploy machine learning models](https://github.com/iterative/mlem)  \n- [PySyft - A library for encrypted, privacy preserving deep learning](https://github.com/OpenMined/PySyft)  \n- [LocalStack - A fully functional local AWS cloud stack](https://github.com/localstack/localstack)  \n- [poetry: Python packaging and dependency management](https://python-poetry.org/)\n- [Netflix's Powerful Database Selections for Streaming at Scale](https://www.linkedin.com/posts/chandreshdesai_cloudcomputing-devops-softwareengineering-activity-7108414221482115072-HGil?utm_source=share\u0026utm_medium=member_desktop)\n- [Ray: a unified framework for scaling AI and Python applications](https://github.com/ray-project/ray)  \n\n## GPU Management Libraries:\n- [GPUtil](https://github.com/anderskm/gputil)\n- [py3nvml](https://github.com/fbcotter/py3nvml) [_Python 3 binding to the NVIDIA Management Library_]\n- [PyCUDA - GitHub](https://github.com/inducer/pycuda)\n- [PyCUDA](https://mathema.tician.de/software/pycuda/)\n- [PyCUDA Tutorial](https://documen.tician.de/pycuda/)\n- [setGPU](https://github.com/bamos/setGPU)\n- [Monitor your GPUs](https://github.com/msalvaris/gpu_monitor) [**Excellent**]  \n- [GPU-Burn - Multi-GPU CUDA stress test](https://github.com/wilicc/gpu-burn) [_Useful_]   \n- [Grafana - Monitoring and Observability](https://github.com/grafana/grafana) [**Excellent**]  \n- [Prometheus](https://prometheus.io/) [_Excellent for monitoring solution \u0026 extract required metrics_]  \n- [OpenAI Triton: Open-Source GPU Programming for Neural Networks](https://openai.com/blog/triton/)  \n\n## Speed-up \u0026 Scalabale Python Codes:\n- [Numba - makes Python code fast](http://numba.pydata.org/)\n- [Dask - natively scales Python](https://dask.org/)\n- [What is Dask](https://medium.com/better-programming/what-is-dask-and-how-can-it-help-you-as-a-data-scientist-72adec7cec57)  \n- [Ray - running distributed applications](https://github.com/ray-project/ray)  \n- [Neural Network Distiller](https://github.com/NervanaSystems/distiller/) [_Distillation \u0026 Quantization of Deep Learning Models in PyTorch_]\n- [Autodistill](https://github.com/autodistill/autodistill)  \n- [PyTorch Pruning Tutorial](https://pytorch.org/tutorials/intermediate/pruning_tutorial.html)  \n- [Can you remove 99% of a neural network without losing accuracy? - An introduction to weight pruning](https://towardsdatascience.com/can-you-remove-99-of-a-neural-network-without-losing-accuracy-915b1fab873b)  \n- [PocketFlow - An Automatic Model Compression (AutoMC) framework](https://github.com/Tencent/PocketFlow) [**Great**]  \n- [Introducing the Model Optimization Toolkit for TensorFlow](https://medium.com/tensorflow/introducing-the-model-optimization-toolkit-for-tensorflow-254aca1ba0a3)  \n- [TensorFlow Model Optimization Toolkit — Post-Training Integer Quantization](https://medium.com/tensorflow/tensorflow-model-optimization-toolkit-post-training-integer-quantization-b4964a1ea9ba)  \n- [TensorFlow Post-training Quantization](https://www.tensorflow.org/lite/performance/post_training_quantization)  \n- [Dynamic Quantization in PyTorch](https://pytorch.org/tutorials/advanced/dynamic_quantization_tutorial.html)  \n- [Static Quantization in PyTorch](https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html)  \n- [NVIDIA DALI - highly optimized data pre-processing in deep learning](https://github.com/NVIDIA/dali)  \n- [Horovod - Distributed training framework](https://github.com/horovod/horovod)  \n- [ONNX Float32 to Float16](https://github.com/onnx/onnx-docker/blob/master/onnx-ecosystem/converter_scripts/float32_float16_onnx.ipynb)  \n- [Speeding Up Deep Learning Inference Using TensorRT](https://devblogs.nvidia.com/speeding-up-deep-learning-inference-using-tensorrt/)  \n- [Speed up Training](https://ai.googleblog.com/2020/05/speeding-up-neural-network-training.html)  \n- [Native PyTorch automatic mixed precision for faster training on NVIDIA GPUs](https://pytorch.org/blog/accelerating-training-on-nvidia-gpus-with-pytorch-automatic-mixed-precision/)  \n- [JAX - Composable transformations of Python+NumPy programs](https://github.com/google/jax)  \n- [TensorRTx - popular DL networks with tensorrt](https://github.com/wang-xinyu/tensorrtx)  \n- [Speeding up Deep Learning Inference Using TensorFlow, ONNX, and TensorRT](https://devblogs.nvidia.com/speeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt/)  \n- [TensorRT Developer Guide](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html)  \n- [How to Convert a Model from PyTorch to TensorRT and Speed Up Inference](https://www.learnopencv.com/how-to-convert-a-model-from-pytorch-to-tensorrt-and-speed-up-inference/) [_Good_]\n- [cuDF: GPU DataFrame Library](https://github.com/rapidsai/cudf)\n- [cuDF example](https://colab.research.google.com/drive/12tCzP94zFG2BRduACucn5Q_OcX1TUKY3)  \n\n## Hardware Notes for Deep Learning:  \n- [Hardware for Deep Learning](https://blog.inten.to/hardware-for-deep-learning-part-3-gpu-8906c1644664)  \n\n## MLOPs Courses \u0026 Resources:  \n- [MLOps-Basics](https://github.com/graviraja/MLOps-Basics) [Great]  \n- [MLOPs-Zoomcamp](https://github.com/DataTalksClub/mlops-zoomcamp) [Great]   \n- [A collection of resources to learn about MLOPs](https://github.com/dair-ai/MLOPs-Primer) [Great]  \n- [Awesome MLOPs](https://github.com/visenger/awesome-mlops) [Great]  \n- [Data Science Topics \u0026 MLOPs](https://github.com/khuyentran1401/Data-science#mlops) [Great]  \n- [MLEM: package and deploy machine learning models](https://github.com/iterative/mlem)  \n- [DevOps Exercises](https://github.com/bregman-arie/devops-exercises)  \n- [MlOPs Sample Project](https://github.com/AntonisCSt/Mlops_project_semicon)  \n- [prefect: Orchestrate and observe all of your workflows](https://www.prefect.io/)  \n- [DataTalks Club: The place to talk about data](https://datatalks.club/)  \n- [OpenNMT CTranslate2: Fast inference engine for Transformer models](https://github.com/OpenNMT/CTranslate2/)  \n\n## Other:\n- [A Guide to Production Level Deep Learning](https://github.com/alirezadir/Production-Level-Deep-Learning)  \n- [Facebook Says Developers Will Love PyTorch 1.0](https://medium.com/syncedreview/facebook-says-developers-will-love-pytorch-1-0-ba2f89ebc9cc)\n- [Some PyTorch Workflow Changes](https://github.com/pytorch/pytorch/issues/6032)\n- [wandb - A tool for visualizing and tracking your machine learning experiments](https://github.com/wandb/client)  \n- [PyTorch and Caffe2 repos getting closer together](https://github.com/caffe2/caffe2/issues/2439#issuecomment-391155017)\n- [PyTorch or TensorFlow?](https://awni.github.io/pytorch-tensorflow/)\n- [Choosing a Deep Learning Framework in 2018: Tensorflow or Pytorch?](http://cv-tricks.com/deep-learning-2/tensorflow-or-pytorch/)\n- [Deep Learning War between PyTorch \u0026 TensorFlow](https://hub.packtpub.com/can-a-production-ready-pytorch-1-0-give-tensorflow-a-tough-time/)\n- [Embedding Machine Learning Models to Web Apps (Part-1)](https://towardsdatascience.com/embedding-machine-learning-models-to-web-apps-part-1-6ab7b55ee428)\n- [Deploying deep learning models: Part 1 an overview](https://towardsdatascience.com/deploying-deep-learning-models-part-1-an-overview-77b4d01dd6f7)\n- [Machine Learning in Production](https://medium.com/contentsquare-engineering-blog/machine-learning-in-production-c53b43283ab1)\n- [how you can get a 2–6x speed-up on your data pre-processing with Python](https://towardsdatascience.com/heres-how-you-can-get-a-2-6x-speed-up-on-your-data-pre-processing-with-python-847887e63be5)\n- [Making your C library callable from Python](https://medium.com/@shamir.stav_83310/making-your-c-library-callable-from-python-by-wrapping-it-with-cython-b09db35012a3)\n- [MIL WebDNN](https://mil-tokyo.github.io/webdnn/)\n- [Multi-GPU Framework Comparisons](https://medium.com/@iliakarmanov/multi-gpu-rosetta-stone-d4fa96162986) [_Great_]  \n\n","funding_links":[],"categories":["Uncategorized","Others","其他_机器学习与深度学习","Other Lists"],"sub_categories":["Uncategorized"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahkarami%2FDeep-Learning-in-Production","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fahkarami%2FDeep-Learning-in-Production","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahkarami%2FDeep-Learning-in-Production/lists"}