https://github.com/kentaroy47/quantize-huggingface

Quantize Huggingface transformers like BERT :hugs:
https://github.com/kentaroy47/quantize-huggingface

bert quantization

Last synced: 3 months ago
JSON representation

Quantize Huggingface transformers like BERT :hugs:

• Host: GitHub
• URL: https://github.com/kentaroy47/quantize-huggingface
• Owner: kentaroy47
• Created: 2022-12-15T05:51:48.000Z (almost 3 years ago)
• Default Branch: master
• Last Pushed: 2022-12-15T06:18:03.000Z (almost 3 years ago)
• Last Synced: 2025-02-01T01:08:24.709Z (8 months ago)
• Topics: bert, quantization
• Language: Jupyter Notebook
• Homepage:
• Size: 35.2 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Quantize huggingface models

see `huggingface-tweet.ipynb` for the implementation.

we quantize distilbert for test, but you can swap to any models.

| Quantized bit | F1    |

|---------------|-------|

| 4             | 0.71  |

| 5             | 0.73  |

| 6             | 0.745 |

| 7             | 0.75  |

| 8             | 0.75  |

| Full          | 0.76  |