https://github.com/cbourjau/benchmarks_ort_session_creation

Benchmark for the creation time of onnxruntime.InferenceSession objects
https://github.com/cbourjau/benchmarks_ort_session_creation

Last synced: 25 days ago
JSON representation

Benchmark for the creation time of onnxruntime.InferenceSession objects

• Host: GitHub
• URL: https://github.com/cbourjau/benchmarks_ort_session_creation
• Owner: cbourjau
• Created: 2024-02-28T16:04:01.000Z (10 months ago)
• Default Branch: master
• Last Pushed: 2024-02-28T16:04:33.000Z (10 months ago)
• Last Synced: 2024-12-01T08:54:05.008Z (25 days ago)
• Language: Python
• Size: 1000 Bytes
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Benchmark for instantiating `onnxruntime.InferenceSession`

There has been a large regression in the session creation time between onnxruntime 1.16.3 and 1.17.0. Running `bench.py` with `onnxruntime=1.17.0` on `osx-arm64` yields:

```shell

$ python bench.py 

onnx.__version__='1.15.0'

ort.__version__='1.17.0'

n_parallel_subgraphs=1000

n_stages=1

Constructing: 0.752s

Building: 1.44s

Session creation: 28.8s

```

while `onnxruntime=1.16.3` exhibits much better performance

```shell

$ python models.py 

onnx.__version__='1.15.0'

ort.__version__='1.16.3'

n_parallel_subgraphs=1000

n_stages=1

Constructing: 0.753s

Building: 1.47s

Session creation: 1.0s

```