awesome-emdl
Embedded and mobile deep learning research resources
https://github.com/csarron/awesome-emdl
Last synced: 6 days ago
JSON representation
-
Courses
-
General
-
Edge / Tiny MLOps
- Tiny-MLOps: a framework for orchestrating ML applications at the far edge of IoT systems
- MLOps for TinyML: Challenges & Directions in Operationalizing TinyML at Scale
- TinyMLOps: Operational Challenges for Widespread Edge AI Adoption
- A TinyMLaaS Ecosystem for Machine Learning in IoT: Overview and Research Challenges - DAT '21]
- SOLIS: The MLOps journey from data acquisition to actionable insights
- Edge MLOps: An Automation Framework for AIoT Applications
- SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devices
- TinyMLOps: Operational Challenges for Widespread Edge AI Adoption
-
OpenCL
-
RenderScript
-
Vulkan
-
Web
-
-
Libraries
-
Inference Framework
- Apple - CoreML - is integrate machine learning models into your app. [BERT and GPT-2 on iPhone](https://github.com/huggingface/swift-coreml-transformers)
- Google - TensorFlow Lite - is an open source deep learning framework for on-device inference.
- xmartlabs - Bender - Easily craft fast Neural Networks on iOS! Use TensorFlow models. Metal under the hood.
- Alibaba - MNN - is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba.
- Arm - ComputeLibrary - is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies. [Intro](https://developer.arm.com/technologies/compute-library)
- Arm - Arm NN - is the most performant machine learning (ML) inference engine for Android and Linux, accelerating ML on Arm Cortex-A CPUs and Arm Mali GPUs.
- Baidu - Paddle Lite - is multi-platform high performance deep learning inference engine.
- DeepLearningKit - is Open Source Deep Learning Framework for Apple's iOS, OS X and tvOS.
- Intel - OpenVINO - Comprehensive toolkit to optimize your processes for faster inference.
- JDAI Computer Vision - dabnn - is an accelerated binary neural networks inference framework for mobile platform.
- Meta - PyTorch Mobile - is a new framework for helping mobile developers and machine learning engineers embed PyTorch ML models on-device.
- Microsoft - DeepSpeed - is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
- Microsoft - ELL - allows you to design and deploy intelligent machine-learned models onto resource constrained platforms and small single-board computers, like Raspberry Pi, Arduino, and micro:bit.
- Microsoft - ONNX RUntime - cross-platform, high performance ML inferencing and training accelerator.
- Nvidia - TensorRT - is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.
- OAID - Tengine - is a lite, high performance, modular inference engine for embedded device
- Tencent - ncnn - is a high-performance neural network inference framework optimized for the mobile platform.
- uTensor - AI inference library based on mbed (an RTOS for ARM chipsets) and TensorFlow.
- XiaoMi - Mace - is a deep learning inference framework optimized for mobile heterogeneous computing platforms.
- Google - TensorFlow Lite - is an open source deep learning framework for on-device inference.
- Edge Impulse - Interactive platform to generate models that can run in microcontrollers. They are also quite active on social netwoks talking about recent news on EdgeAI/TinyML.
- Google - TensorFlow Lite - is an open source deep learning framework for on-device inference.
- Meta - PyTorch Mobile - is a new framework for helping mobile developers and machine learning engineers embed PyTorch ML models on-device.
- Qualcomm - Neural Processing SDK for AI - Libraries to developers run NN models on Snapdragon mobile platforms taking advantage of the CPU, GPU and/or DSP.
-
Optimization Tools
- PocketFlow - An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.
- Neural Network Distiller - Python package for neural network compression research.
-
Research Demos
- RSTensorFlow - GPU Accelerated TensorFlow for Commodity Android Devices.
- RSTensorFlow - GPU Accelerated TensorFlow for Commodity Android Devices.
-
Web
- mil-tokyo/webdnn - Fastest DNN Execution Framework on Web Browser.
-
-
Papers
-
Approximation
- High performance ultra-low-precision convolutions on mobile devices
- Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications
- Efficient and Accurate Approximations of Nonlinear Convolutional Networks
- Accelerating Very Deep Convolutional Networks for Classification and Detection
- Convolutional neural networks with low-rank regularization
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
-
Characterization
- A First Look at Deep Learning Apps on Smartphones
- Machine Learning at Facebook: Understanding Inference at the Edge
- NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
- Latency and Throughput Characterization of Convolutional Neural Networks for Mobile Computer Vision
- Machine Learning at Facebook: Understanding Inference at the Edge
-
Model
- MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
- Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets
- MCUNet: Tiny Deep Learning on IoT Devices
- GhostNet: More Features from Cheap Operations
- MicroNet for Efficient Language Modeling
- Searching for MobileNetV3
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- DeepRebirth: Accelerating Deep Neural Network Execution on Mobile Devices
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- CondenseNet: An Efficient DenseNet using Learned Group Convolutions
- NasNet: Learning Transferable Architectures for Scalable Image Recognition
- EtinyNet: Extremely Tiny Network for TinyML
- SkyNet: a Hardware-Efficient Method for Object Detection and Tracking on Embedded Systems
-
Pruning
- Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration
- To prune, or not to prune: exploring the efficacy of pruning for model compression
- Pruning Filters for Efficient ConvNets
- Pruning Convolutional Neural Networks for Resource Efficient Inference
- Soft Weight-Sharing for Neural Network Compression
- Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning
- ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Dynamic Network Surgery for Efficient DNNs
- Learning both Weights and Connections for Efficient Neural Networks
- Awesome-Pruning
-
Quantization
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- Training and Inference with Integers in Deep Neural Networks
- The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning
- Loss-aware Binarization of Deep Networks
- Towards the Limit of Network Quantization
- Deep Learning with Low Precision by Half-wave Gaussian Quantization
- ShiftCNN: Generalized Low-Precision Architecture for Inference of Convolutional Neural Networks
- Quantized Convolutional Neural Networks for Mobile Devices
- Fixed-Point Performance Analysis of Recurrent Neural Networks
- Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
- Compressing Deep Convolutional Networks using Vector Quantization
- LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
-
Survey
- TinyML Platforms Benchmarking
- TinyML: A Systematic Review and Synthesis of Existing Research
- TinyML Meets IoT: A Comprehensive Survey
- A review on TinyML: State-of-the-art and prospects
- TinyML Benchmark: Executing Fully Connected Neural Networks on Commodity Microcontrollers
- Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
- Benchmarking TinyML Systems: Challenges and Direction
- Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
- The Deep Learning Compiler: A Comprehensive Survey
- Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
- A Survey of Model Compression and Acceleration for Deep Neural Networks
- Awesome ML Model Compression
- TinyML Papers and Projects
- EfficientDNNs
-
System
- BSC: Block-based Stochastic Computing to Enable Accurate and Efficient TinyML - DAC '22]
- CFU Playground: Full-Stack Open-Source Framework for Tiny Machine Learning (tinyML) Acceleration on FPGAs
- UDC: Unified DNAS for Compressible TinyML Models
- AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator
- TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning
- Once for All: Train One Network and Specialize it for Efficient Deployment
- DeepMon: Mobile GPU-based Deep Learning Framework for Continuous Vision Applications
- DeepEye: Resource Efficient Local Execution of Multiple Deep Vision Models using Wearable Commodity Hardware
- MobiRNN: Efficient Recurrent Neural Network Execution on Mobile GPU
- fpgaConvNet: A Toolflow for Mapping Diverse Convolutional Neural Networks on Embedded FPGAs
- DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices
- EIE: Efficient Inference Engine on Compressed Deep Neural Network
- MCDNN: An Approximation-Based Execution Framework for Deep Stream Processing Under Resource Constraints
- DXTK: Enabling Resource-efficient Deep Learning on Mobile and Embedded Devices with the DeepX Toolkit
- Sparsification and Separation of Deep Learning Layers for Constrained Resource Inference on Wearables
- An Early Resource Characterization of Deep Learning on Wearables, Smartphones and Internet-of-Things Devices - App ’15]
- CNNdroid: GPU-Accelerated Execution of Trained Deep Convolutional Neural Networks on Android
- BSC: Block-based Stochastic Computing to Enable Accurate and Efficient TinyML - DAC '22]
- MCDNN: An Approximation-Based Execution Framework for Deep Stream Processing Under Resource Constraints
- BSC: Block-based Stochastic Computing to Enable Accurate and Efficient TinyML - DAC '22]
- DeepEye: Resource Efficient Local Execution of Multiple Deep Vision Models using Wearable Commodity Hardware
- DeepSense: A GPU-based deep convolutional neural network framework on commodity mobile devices
-
-
Related Repos
-
Driver
- Model Compression - marple-dev  
- awesome-AutoML-and-Lightweight-Models - yuan  
- knowledge-distillation-papers - distillation-papers?style=social) 
- Awesome-model-compression-and-acceleration - model-compression-and-acceleration?style=social) 
- Embedded Neural Network - Neural-Network?style=social) 
- Embedded Neural Network - Neural-Network?style=social) 
-
-
Tools
-
Driver
-
GPU
- Midgard GPU Architecture - T880 GPU](https://www.hotchips.org/wp-content/uploads/hc_archives/hc27/HC27.25-Tuesday-Epub/HC27.25.50-GPU-Epub/HC27.25.531-Mali-T880-Bratt-ARM-2015_08_23.pdf)
- Mobile GPU market share
-
-
Tutorials
-
General
- Squeezing Deep Learning Into Mobile Phones
- Deep Learning – Tutorial and Recent Trends
- Tutorial on Hardware Architectures for Deep Neural Networks
- Efficient Convolutional Neural Network Inference on Mobile GPUs
- Tutorial on Hardware Architectures for Deep Neural Networks
- Squeezing Deep Learning Into Mobile Phones
- Efficient Convolutional Neural Network Inference on Mobile GPUs
-
NEON
-
OpenCL
-
Programming Languages
Sub Categories
Keywords
deep-learning
13
machine-learning
12
neural-network
6
model-compression
6
pytorch
6
tensorflow
6
deep-neural-networks
4
computer-vision
4
arm
4
neural-networks
4
inference
4
pruning
3
awesome-list
3
vulkan
3
quantization
3
onnx
3
ios
3
embedded
2
riscv
2
simd
2
neural-architecture-search
2
neon
2
opencl
2
fpga
2
automl
2
knowledge-distillation
2
model-acceleration
2
artificial-intelligence
2
android
2
compression
2
tensorrt
2
metal
2
linux
1
cpp
1
armv8
1
armv7
1
sve
1
baidu
1
mali
1
mdl
1
mobile
1
mobile-deep-learning
1
ai
1
deploy-ai
1
diffusion-models
1
generative-ai
1
good-first-issue
1
llm-inference
1
apple
1
convolutional-neural-networks
1