Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/echogarden-project/whisper-onnx-exporter
Exports OpenAI Whisper speech recognition models to ONNX. Mainly intended for use with Echogarden.
https://github.com/echogarden-project/whisper-onnx-exporter
Last synced: about 1 month ago
JSON representation
Exports OpenAI Whisper speech recognition models to ONNX. Mainly intended for use with Echogarden.
- Host: GitHub
- URL: https://github.com/echogarden-project/whisper-onnx-exporter
- Owner: echogarden-project
- License: mit
- Created: 2023-04-20T04:38:10.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2023-11-26T03:41:47.000Z (about 1 year ago)
- Last Synced: 2023-11-26T20:30:57.695Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 15.6 KB
- Stars: 7
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Whisper ONNX exporter
A tool to export OpenAI Whisper speech recognition models to ONNX.
The core model file (`model.py`) has been isolated from the [original Whisper codebase](https://github.com/openai/whisper). Other files are not included or needed.
Taking some of the code in [`whisper-openvino`](https://github.com/zhuzilin/whisper-openvino) as a starting point, the model's key-value structure has been modified to be passed as an input or output, removing the need for hooks.
The `TextDecoder`, `ResidualAttentionBlock` and `MultiHeadAttention` classes have also been further modified to directly output the cross-attention weights, without any hooks.
The exported ONNX models are primarily intended to be used with [Echogarden](https://github.com/echogarden-project/echogarden), which has its own implementation of the higher-level Whisper API, and is written in TypeScript. The code doesn't include a way to use the exported models from Python. However, since it is closely related to the code on [`whisper-openvino`](https://github.com/zhuzilin/whisper-openvino), which adapts the higher-level Python API to use it, it should be possible to make it work with it, with some modifications.
## Downloading pre-exported models
You can download pre-exported models for all sizes, except `large`, `large-v1`, `large-v2`, `large-v3` from the releases section of the [`whisper-onnx-models` repository](https://github.com/echogarden-project/whisper-onnx-models).
## Usage
Ensure you have `torch` and `onnx` Python libraries installed.
Copy the official Whisper model files (`.pt`) to the `pytorch-models` subdirectory.
To get the models you can use the official Whisper CLI, which would auto-download a model as needed. On Windows, the downloaded models should be stored at `%userprofile%\.cache\whisper`.
Alternatively, you can find direct download URLs in the [original Whisper source code](https://github.com/openai/whisper/blob/25639fc17ddc013d56c594bfbf7644f2185fad84/whisper/__init__.py#L17).
Run:
```
python export-whisper-onnx.py whisper-model-name [--export-fp16] [--export-fp16-mixed]
```
For example:
```
python export-whisper-onnx.py tiny
```
The exported encoder and decoder ONNX models would be located at:
```
onnx-models/tiny/encoder.onnx
onnx-models/tiny/decoder.onnx
```
## License
MIT