https://github.com/kojix2/onnxruntime.cr
https://github.com/kojix2/onnxruntime.cr
crystal crystal-language onnxruntime
Last synced: 12 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/kojix2/onnxruntime.cr
- Owner: kojix2
- License: mit
- Created: 2022-09-24T08:20:54.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2025-04-11T22:50:43.000Z (12 months ago)
- Last Synced: 2025-04-12T22:58:44.714Z (12 months ago)
- Topics: crystal, crystal-language, onnxruntime
- Language: C++
- Homepage: https://kojix2.github.io/onnxruntime.cr/
- Size: 313 KB
- Stars: 5
- Watchers: 2
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# onnxruntime.cr
[](https://github.com/kojix2/onnxruntime.cr/actions/workflows/test.yml)
[](https://tokei.kojix2.net/github/kojix2/onnxruntime.cr)
[ONNX Runtime](https://github.com/Microsoft/onnxruntime) bindings for Crystal
## Installation
1. Install ONNX Runtime
Download and install the ONNX Runtime from the [official releases](https://github.com/microsoft/onnxruntime/releases).
For Linux:
```bash
# Example for Linux
wget https://github.com/microsoft/onnxruntime/releases/download/v1.21.0/onnxruntime-linux-x64-1.21.0.tgz
tar -xzf onnxruntime-linux-x64-1.21.0.tgz
export ONNXRUNTIMEDIR=/path/to/onnxruntime-linux-x64-1.21.0
```
For macOS:
```bash
# Example for macOS
wget https://github.com/microsoft/onnxruntime/releases/download/v1.21.0/onnxruntime-osx-x86_64-1.21.0.tgz
tar -xzf onnxruntime-osx-x86_64-1.21.0.tgz
export ONNXRUNTIMEDIR=/path/to/onnxruntime-osx-x86_64-1.21.0
```
For Windows:
```powershell
# Example for Windows
# Download from https://github.com/microsoft/onnxruntime/releases
# Extract and set environment variable
$env:ONNXRUNTIMEDIR = "C:\path\to\onnxruntime-win-x64-1.21.0"
```
2. Add the dependency to your `shard.yml`:
```yaml
dependencies:
onnxruntime:
github: kojix2/onnxruntime.cr
```
3. Run `shards install`
## Usage
```crystal
require "onnxruntime"
# Load a model
model = OnnxRuntime::Model.new("path/to/model.onnx")
# Print model inputs and outputs
puts "Inputs:"
model.inputs.each do |input|
puts " #{input[:name]}: #{input[:type]} #{input[:shape]}"
end
puts "Outputs:"
model.outputs.each do |output|
puts " #{output[:name]}: #{output[:type]} #{output[:shape]}"
end
# Prepare input data
input_data = {
"input_name" => [1.0_f32, 2.0_f32, 3.0_f32]
}
# Run inference
result = model.predict(input_data)
# Process results
result.each do |name, data|
puts "#{name}: #{data}"
end
```
## MNIST Example
Download the MNIST model: [mnist-12.onnx](https://github.com/onnx/models/blob/main/validated/vision/classification/mnist/model/mnist-12.onnx)
```crystal
require "onnxruntime"
# Load the MNIST model
model = OnnxRuntime::Model.new("mnist.onnx")
# Create a dummy input (28x28 image draw 1)
input_data = Array(Float32).new(28 * 28) { |i| (i % 14 == 0 ? 1.0 : 0.0).to_f32 }
# Run inference
result = model.predict({"Input3" => input_data}, ["Plus214_Output_0"], shape: {"Input3" => [1_i64, 1_i64, 28_i64, 28_i64]})
# Get the output probabilities
probabilities = result["Plus214_Output_0"].as(Array(Float32))
# Find the digit with highest probability
predicted_digit = probabilities.index(probabilities.max)
puts "Predicted digit: #{predicted_digit}"
# Explicitly release resources
model.release
OnnxRuntime::InferenceSession.release_env
```
## Memory Management
Currently, this library requires explicit resource management as reference counting is not yet fully implemented. To prevent memory leaks, you must explicitly release resources when you're done with them:
```crystal
# Create and use model
model = OnnxRuntime::Model.new("path/to/model.onnx")
result = model.predict(input_data)
# When finished, explicitly release resources
model.release
OnnxRuntime::InferenceSession.release_env
```
For long-running applications like web servers, consider setting up signal handlers to ensure resources are properly released on shutdown:
```crystal
Signal::INT.trap do
puts "Shutting down..."
model.release
OnnxRuntime::InferenceSession.release_env
exit
end
```
See the examples directory for more detailed implementations.
## Development
The code is generated by AI and may not be perfect.
Please feel free to contribute and improve it.
## Contributing
1. Fork it ()
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request