https://github.com/ikomia-dev/onnx-donut
Export Donut model to onnx and run it with onnxruntime
https://github.com/ikomia-dev/onnx-donut
Last synced: 10 months ago
JSON representation
Export Donut model to onnx and run it with onnxruntime
- Host: GitHub
- URL: https://github.com/ikomia-dev/onnx-donut
- Owner: Ikomia-dev
- License: apache-2.0
- Created: 2023-11-08T15:22:06.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-11-21T10:26:13.000Z (over 2 years ago)
- Last Synced: 2025-05-14T18:52:32.086Z (about 1 year ago)
- Language: Python
- Size: 28.3 KB
- Stars: 23
- Watchers: 1
- Forks: 4
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Onnx Donut
Package to export a [Donut](https://github.com/clovaai/donut) model from pytorch to ONNX, then run it with onnxruntime.
## Installation
```shell
pip install onnx-donut
```
## Export to onnx
```python
from onnx_donut.exporter import export_onnx
from onnx_donut.quantizer import quantize
# Hugging Face model card or folder
model_path = "naver-clova-ix/donut-base-finetuned-docvqa"
# Folder where the exported model will be stored
dst_folder = "converted_donut"
# Export from Pytorch to ONNX
export_onnx(model_path, dst_folder, opset_version=16)
# Quantize your model to int8
quantize(dst_folder, dst_folder + "_quant")
```
## Model inference with onnxruntime
```python
from onnx_donut.predictor import OnnxPredictor
import numpy as np
from PIL import Image
# Image path to run on
img_path = "/path/to/your/image.png"
# Folder where the exported model will be stored
onnx_folder = "converted_donut"
# Read image
img = np.array(Image.open(img_path).convert('RGB'))
# Instantiate ONNX predictor
predictor = OnnxPredictor(model_folder=onnx_folder)
# Write your prompt accordingly to the model you use
prompt = f"what is the title?"
# Run prediction
out = predictor.generate(img, prompt)
# Display prediction
print(out)
```