https://github.com/sayakpaul/adventures-in-tensorflow-lite
This repository contains notebooks that show the usage of TensorFlow Lite for quantizing deep neural networks.
https://github.com/sayakpaul/adventures-in-tensorflow-lite
inference model-optimization model-quantization on-device-ml post-training-quantization pruning quantization-aware-training tensorflow-2 tensorflow-lite tf-hub tf-lite-model
Last synced: 9 months ago
JSON representation
This repository contains notebooks that show the usage of TensorFlow Lite for quantizing deep neural networks.
- Host: GitHub
- URL: https://github.com/sayakpaul/adventures-in-tensorflow-lite
- Owner: sayakpaul
- License: apache-2.0
- Created: 2020-04-29T00:07:54.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2023-01-23T13:21:12.000Z (over 3 years ago)
- Last Synced: 2024-12-28T03:39:03.806Z (over 1 year ago)
- Topics: inference, model-optimization, model-quantization, on-device-ml, post-training-quantization, pruning, quantization-aware-training, tensorflow-2, tensorflow-lite, tf-hub, tf-lite-model
- Language: Jupyter Notebook
- Homepage:
- Size: 49.1 MB
- Stars: 172
- Watchers: 12
- Forks: 34
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Adventures-in-TensorFlow-Lite
This repository contains notebooks that show the usage of TensorFlow Lite (TF Lite) for quantizing deep neural networks in TensorFlow 2. ***It is currently under active development so, there might be some inconsistencies in the description below***.
## About the notebooks
- `A_tale_of_quantization.ipynb`: A comprehensive notebook showing different ways to quantize a model in `tf.keras` (with fine-tuning). It includes both **quantization-aware training** as well as **post-training quantization**. This notebook is accompanied by [this report](https://bit.ly/3dlCRSI) that has some additional details and performance considerations. The notebook is best referred with this report.
- `Custom_Image_Classification_EdgeTPU.ipynb`: It shows how to use post-training quantization with a representative dataset to calibrate the dynamic ranges of activations. It also shows how to quantize the model in a way that is compatible with an Edge TPU USB Accelerator.
- `DeepLabV3/DeepLab_TFLite_*.ipynb`: These notebooks show how to convert several DeepLabV3 based segmentation models (trained on the PASCAL VOC 2012, ADE20k, and CityScapes datasets) to TF Lite and run inference with them.
- `ESRGAN_TFLite.ipynb`: Shows how to run inference with this ESRGAN module from TF Hub, convert it to TF Lite, run inference with the model. A bonus includes running inference with a distilled version of the main model (only 33 KB).
- `Magenta_arbitrary_style_transfer_model_conversion.ipynb`: Shows to how apply apply different post-training quantization schemes to the [arbitrary style transfer model provided by Magenta](https://github.com/magenta/magenta/tree/f3b66aa1354cd933f0e9757a567cc9a3d2d03297/magenta/models/arbitrary_image_stylization). This generates image stylization models that produce higher-quality images than [these ones](https://tfhub.dev/google/magenta/arbitrary-image-stylization-v1-256/2). Inference notebook is available here - `Style_Transfer_Demo_InceptionV3.ipynb`. Additionally, in this notebook you can find how to run inference with the same TF Lite models but with dynamic shape support: `Style_Transfer_Demo_InceptionV3_Dynamic_Shape.ipynb`.
- `Model_Pruning_in_Deep_Learning_with_tfmot.ipynb`: Shows how to use the pruning APIs of TensorFlow Model Optimization toolkit along with TF Lite.
- `Selfie2Anime_TFLite(50_Checkpoints).ipynb`: Shows how to convert the [UGATIT model](https://github.com/taki0112/UGATIT) (also known as Selfie2Anime GAN) to TF Lite. Inference notebook is available [here](https://github.com/margaretmz/selfie2anime-e2e-tutorial/blob/master/ml/Selfie2Anime_Model_Conversion_50_Epochs.ipynb). Here's a multi-part tutorial of this project - [Part I](https://bit.ly/selfie2anime-1), [Part II](https://bit.ly/selfie2anime-2), [Part III](https://bit.ly/selfie2anime-3). This was jointly done with [Margaret](https://twitter.com/margaretmz).
- `Semantic_Segmentation_+_Background_Removal_+_Style_Transfer.ipynb`: Presents a demo on how to use multiple TF Lite models to run segmentation, remove background, and apply stylization.
- `Style_Transfer_Demo.ipynb` & `Style_Transfer_Demo_InceptionV3.ipynb`: Present interactive image stylization demos.
- `TUNIT_Conversion_to_TF_Lite.ipynb`: Shows the PyTorch -> TF Lite model conversion worflow (this is currently buggy).
- `CartoonGAN_TFLite.ipynb`: Shows to how to convert the CartoonGAN model (proposed [here](https://bit.ly/cartoon-gan)) to a TF Lite model. Includes how to run inference as well. Here's the [official TensorFlow blog](https://github.com/SystemErrorWang/White-box-Cartoonization) in case you are interested to know more.
- `EAST_TFLite.ipynb`: Shows how to convert the famous [EAST](https://arxiv.org/abs/1704.03155) model for text detection to TensorFlow Lite.
- `Boundless_TFLite.ipynb`: Shows how to convert the famous [Boundless](https://arxiv.org/pdf/1908.07007.pdf) model for image extrapolation to TensorFlow Lite.
## References
- [Post-training quantization](https://www.tensorflow.org/lite/performance/post_training_quantization)
- [Quantization aware training with TensorFlow Model Optimization Toolkit](https://blog.tensorflow.org/2020/04/quantization-aware-training-with-tensorflow-model-optimization-toolkit.html)
## Accompanying materials
- [Doing more with TF Lite](http://bit.ly/tfl-pune)
- [Model optimization 101](http://bit.ly/mo-101)
- [A few good stuff in TF Lite](http://bit.ly/stuff-tflite)
- [A Tale of Model Quantization in TF Lite](https://bit.ly/3dlCRSI)
- [Plunging into Model Pruning in Deep Learning](https://bit.ly/2AJ67W4)
- [Optimizing MobileDet for Mobile Deployments](https://sayak.dev/mobiledet-optimization/)
- [Peer Bonus Experiences: Building tiny models for the ML community with TensorFlow](https://opensource.googleblog.com/2020/10/peer-bonus-experiences-building-tiny.html)
- [A Battle of Text Detectors for Mobile Deployments: CRAFT vs. EAST](https://sayak.dev/optimizing-text-detectors/)
## Find the models on TF Hub
https://tfhub.dev/s?publisher=sayakpaul
## Other notable repositories covering similar things
* [tulasiram58827/ocr_tflite](https://github.com/tulasiram58827/ocr_tflite)
* [tulasiram58827/TTS_TFLite](https://github.com/tulasiram58827/TTS_TFLite)
## Acknowledgements
- [Arun Venkatesan](https://www.linkedin.com/in/arun-venkatesan-9317796/)
- [Khanh LeViet](https://www.linkedin.com/in/lvgk/)
- [Pulkit Bhuwalka](https://www.linkedin.com/in/pulkitbhuwalka/)