awesome-emdl
Embedded and mobile deep learning research resources
https://github.com/csarron/awesome-emdl
Last synced: 28 days ago
JSON representation
Courses
OpenCL
General
Edge / Tiny MLOps
- Tiny-MLOps: a framework for orchestrating ML applications at the far edge of IoT systems
- MLOps for TinyML: Challenges & Directions in Operationalizing TinyML at Scale
- TinyMLOps: Operational Challenges for Widespread Edge AI Adoption
- A TinyMLaaS Ecosystem for Machine Learning in IoT: Overview and Research Challenges - DAT '21]
- SOLIS: The MLOps journey from data acquisition to actionable insights
- Edge MLOps: An Automation Framework for AIoT Applications
- SensiX++: Bringing MLOPs and Multi-tenant Model Serving to Sensory Edge Devices
OpenCL
RenderScript
Vulkan
Web
Libraries
Inference Framework
- Apple - CoreML - is integrate machine learning models into your app. BERT and GPT-2 on iPhone
- Google - TensorFlow Lite - is an open source deep learning framework for on-device inference.
- xmartlabs - Bender - Easily craft fast Neural Networks on iOS! Use TensorFlow models. Metal under the hood.
- Alibaba - MNN - is a blazing fast, lightweight deep learning framework, battle-tested by business-critical use cases in Alibaba.
- Arm - ComputeLibrary - is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies. Intro
- Arm - Arm NN - is the most performant machine learning (ML) inference engine for Android and Linux, accelerating ML on Arm Cortex-A CPUs and Arm Mali GPUs.
- Baidu - Paddle Lite - is multi-platform high performance deep learning inference engine.
- DeepLearningKit - is Open Source Deep Learning Framework for Apple's iOS, OS X and tvOS.
- Intel - OpenVINO - Comprehensive toolkit to optimize your processes for faster inference.
- JDAI Computer Vision - dabnn - is an accelerated binary neural networks inference framework for mobile platform.
- Meta - PyTorch Mobile - is a new framework for helping mobile developers and machine learning engineers embed PyTorch ML models on-device.
- Microsoft - DeepSpeed - is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
- Microsoft - ELL - allows you to design and deploy intelligent machine-learned models onto resource constrained platforms and small single-board computers, like Raspberry Pi, Arduino, and micro:bit.
- Microsoft - ONNX RUntime - cross-platform, high performance ML inferencing and training accelerator.
- Nvidia - TensorRT - is a C++ library for high performance inference on NVIDIA GPUs and deep learning accelerators.
- OAID - Tengine - is a lite, high performance, modular inference engine for embedded device
- Tencent - ncnn - is a high-performance neural network inference framework optimized for the mobile platform.
- uTensor - AI inference library based on mbed (an RTOS for ARM chipsets) and TensorFlow.
- XiaoMi - Mace - is a deep learning inference framework optimized for mobile heterogeneous computing platforms.
- Google - TensorFlow Lite - is an open source deep learning framework for on-device inference.
Optimization Tools
- PocketFlow - An Automatic Model Compression (AutoMC) framework for developing smaller and faster AI applications.
- Neural Network Distiller - Python package for neural network compression research.
Research Demos
- RSTensorFlow - GPU Accelerated TensorFlow for Commodity Android Devices.
Web
- mil-tokyo/webdnn - Fastest DNN Execution Framework on Web Browser.
Papers
Approximation
- High performance ultra-low-precision convolutions on mobile devices
- Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications
- Efficient and Accurate Approximations of Nonlinear Convolutional Networks
- Accelerating Very Deep Convolutional Networks for Classification and Detection
- Convolutional neural networks with low-rank regularization
- Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
Characterization
- A First Look at Deep Learning Apps on Smartphones
- Machine Learning at Facebook: Understanding Inference at the Edge
- NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
- Latency and Throughput Characterization of Convolutional Neural Networks for Mobile Computer Vision
Model
- MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
- Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNets
- MCUNet: Tiny Deep Learning on IoT Devices
- GhostNet: More Features from Cheap Operations
- MicroNet for Efficient Language Modeling
- Searching for MobileNetV3
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- DeepRebirth: Accelerating Deep Neural Network Execution on Mobile Devices
- ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
- MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
- CondenseNet: An Efficient DenseNet using Learned Group Convolutions
- NasNet: Learning Transferable Architectures for Scalable Image Recognition
Pruning
- Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration
- To prune, or not to prune: exploring the efficacy of pruning for model compression
- Pruning Filters for Efficient ConvNets
- Pruning Convolutional Neural Networks for Resource Efficient Inference
- Soft Weight-Sharing for Neural Network Compression
- Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning
- ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
- Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
- Dynamic Network Surgery for Efficient DNNs
- Learning both Weights and Connections for Efficient Neural Networks
- Awesome-Pruning
Quantization
- Quantizing deep convolutional networks for efficient inference: A whitepaper
- Training and Inference with Integers in Deep Neural Networks
- The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning
- Loss-aware Binarization of Deep Networks
- Towards the Limit of Network Quantization
- Deep Learning with Low Precision by Half-wave Gaussian Quantization
- ShiftCNN: Generalized Low-Precision Architecture for Inference of Convolutional Neural Networks
- Quantized Convolutional Neural Networks for Mobile Devices
- Fixed-Point Performance Analysis of Recurrent Neural Networks
- Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
- Compressing Deep Convolutional Networks using Vector Quantization
Survey
- TinyML Platforms Benchmarking
- TinyML: A Systematic Review and Synthesis of Existing Research
- TinyML Meets IoT: A Comprehensive Survey
- A review on TinyML: State-of-the-art and prospects
- TinyML Benchmark: Executing Fully Connected Neural Networks on Commodity Microcontrollers
- Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better
- Benchmarking TinyML Systems: Challenges and Direction
- Model Compression and Hardware Acceleration for Neural Networks: A Comprehensive Survey
- The Deep Learning Compiler: A Comprehensive Survey
- Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
- A Survey of Model Compression and Acceleration for Deep Neural Networks
- Awesome ML Model Compression
- TinyML Papers and Projects
- EfficientDNNs
System
- BSC: Block-based Stochastic Computing to Enable Accurate and Efficient TinyML - DAC '22]
- CFU Playground: Full-Stack Open-Source Framework for Tiny Machine Learning (tinyML) Acceleration on FPGAs
- UDC: Unified DNAS for Compressible TinyML Models
- AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator
- TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning
- Once for All: Train One Network and Specialize it for Efficient Deployment
- DeepMon: Mobile GPU-based Deep Learning Framework for Continuous Vision Applications
- DeepEye: Resource Efficient Local Execution of Multiple Deep Vision Models using Wearable Commodity Hardware
- MobiRNN: Efficient Recurrent Neural Network Execution on Mobile GPU
- fpgaConvNet: A Toolflow for Mapping Diverse Convolutional Neural Networks on Embedded FPGAs
- DeepX: A Software Accelerator for Low-Power Deep Learning Inference on Mobile Devices
- EIE: Efficient Inference Engine on Compressed Deep Neural Network
- MCDNN: An Approximation-Based Execution Framework for Deep Stream Processing Under Resource Constraints
- DXTK: Enabling Resource-efficient Deep Learning on Mobile and Embedded Devices with the DeepX Toolkit
- Sparsification and Separation of Deep Learning Layers for Constrained Resource Inference on Wearables
- An Early Resource Characterization of Deep Learning on Wearables, Smartphones and Internet-of-Things Devices - App ’15]
- CNNdroid: GPU-Accelerated Execution of Trained Deep Convolutional Neural Networks on Android
- BSC: Block-based Stochastic Computing to Enable Accurate and Efficient TinyML - DAC '22]
- MCDNN: An Approximation-Based Execution Framework for Deep Stream Processing Under Resource Constraints
Related Repos
Driver
- Model Compression - marple-dev
- awesome-AutoML-and-Lightweight-Models - yuan
- knowledge-distillation-papers - distillation-papers?style=social)
- Awesome-model-compression-and-acceleration - model-compression-and-acceleration?style=social)
- Embedded Neural Network - Neural-Network?style=social)
- Embedded Neural Network - Neural-Network?style=social)
Tools
Driver
GPU
Tutorials
General
- Squeezing Deep Learning Into Mobile Phones
- Deep Learning – Tutorial and Recent Trends
- Tutorial on Hardware Architectures for Deep Neural Networks
- Efficient Convolutional Neural Network Inference on Mobile GPUs
- Tutorial on Hardware Architectures for Deep Neural Networks
- Squeezing Deep Learning Into Mobile Phones
- Efficient Convolutional Neural Network Inference on Mobile GPUs