https://github.com/olilarkin/ort-builder
ONNX Runtime static library builder
https://github.com/olilarkin/ort-builder
machine-learning onnx onnxruntime
Last synced: about 1 year ago
JSON representation
ONNX Runtime static library builder
- Host: GitHub
- URL: https://github.com/olilarkin/ort-builder
- Owner: olilarkin
- License: mit
- Created: 2023-02-09T16:41:00.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2025-04-22T21:44:53.000Z (about 1 year ago)
- Last Synced: 2025-04-22T22:24:42.909Z (about 1 year ago)
- Topics: machine-learning, onnx, onnxruntime
- Language: C++
- Homepage:
- Size: 48.2 MB
- Stars: 54
- Watchers: 6
- Forks: 15
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ONNX Runtime static library builder
Converts an [ONNX](https://onnx.ai) model to ORT format and serializes it to C++ source code, generate custom slimmed ONNX Runtime static libs & xcframework for apple platforms.
The goal here is to create a flexible but tiny inference engine for a specific model e.g. [iPlug2 example](https://github.com/olilarkin/iPlug2OnnxRuntime).
*NOTE: due to risk of ODR violations, global/static variable conflicts, and dependency symbol clashes with DAW Hosts that use ORT themselves - think hard before you use this in an audio plugin!*
The scripts here are configured to create a minimal ORT binary using only the CPU provider. If you want to experiment with GPU inference, Core ML etc, you will have to modify.
## Requirements:
CMake v2.6+
## Instructions:
1. Checkout ONNX Runtime submodule `$ git submodule update --init`
2. Create a [virtual environment](https://packaging.python.org/tutorials/installing-packages/#creating-virtual-environments) and activate it
windows
```bash
$ py -3 -m venv venv
$ source ./venv/Scripts/activate`
```
mac/linux
```bash
$ python3 -m venv venv
$ source ./venv/bin/activate`
```
3. Install dependencies `$ pip install -r requirements.txt`
4. Run `$ ./convert-model-to-ort.sh model.onnx`
This converts the .onnx file to .ort and produces a .config file which slims the onnxruntime library build in the next step.
It also serializes the .ort format model to C++ source code, which can be used to bake the model into your app binary. If the model
is large this might not be a great solution, and it might be better to locate the .ort file at runtime.
6. Build customized onnx runtime static libraries
```mac
$ ./build-mac.sh
```
```ios
$ ./build-ios.sh
$ ./build-ios-simulator.sh
```
```xcframework build
$ ./build-xcframework.sh
```
**Note:** windows static lib builds can get very large due to the LTO/LTCG settings in onnxruntime.
You can turn that off by applying the change in ltcg_patch_for_windows.patch to the onnxruntime repo.
Due to different MSVC runtimes for Debug and Release builds, we need to build two binaries for windows.
```windows
$ ./build-win.sh
```