https://github.com/elixir-nx/ortex

ONNX Runtime bindings for Elixir
https://github.com/elixir-nx/ortex

elixir machine-learning onnx onnxruntime rust

Last synced: about 2 months ago
JSON representation

ONNX Runtime bindings for Elixir

• Host: GitHub
• URL: https://github.com/elixir-nx/ortex
• Owner: elixir-nx
• License: mit
• Created: 2023-05-10T17:53:46.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-06-17T21:25:24.000Z (3 months ago)
• Last Synced: 2024-07-18T04:02:53.871Z (2 months ago)
• Topics: elixir, machine-learning, onnx, onnxruntime, rust
• Language: Elixir
• Homepage:
• Size: 95.7 KB
• Stars: 116
• Watchers: 8
• Forks: 15
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Ortex

`Ortex` is a wrapper around [ONNX Runtime](https://onnxruntime.ai/) (implemented as

bindings to [`ort`](https://github.com/pykeio/ort)). Ortex leverages

[`Nx.Serving`](https://hexdocs.pm/nx/Nx.Serving.html) to easily deploy ONNX models

that run concurrently and distributed in a cluster. Ortex also provides a storage-only

tensor implementation for ease of use.

ONNX models are a standard machine learning model format that can be exported from most ML

libraries like PyTorch and TensorFlow. Ortex allows for easy loading and fast inference of

ONNX models using different backends available to ONNX Runtime such as CUDA, TensorRT, Core

ML, and ARM Compute Library.

## Examples

TL;DR:

```elixir

iex> model = Ortex.load("./models/resnet50.onnx")

#Ortex.Model<

  inputs: [{"input", "Float32", [nil, 3, 224, 224]}]

  outputs: [{"output", "Float32", [nil, 1000]}]>

iex> {output} = Ortex.run(model, Nx.broadcast(0.0, {1, 3, 224, 224}))

iex> output |> Nx.backend_transfer() |> Nx.argmax

#Nx.Tensor<

  s64

  499

>

```

Inspecting a model shows the expected inputs, outputs, data types, and shapes. Axes with

`nil` represent a dynamic size.

To see more real world examples see the `examples` folder.

### Serving

`Ortex` also implements `Nx.Serving` behaviour. To use it in your application's

supervision tree consult the `Nx.Serving` docs.

```elixir

iex> serving = Nx.Serving.new(Ortex.Serving, model)

iex> batch = Nx.Batch.stack([{Nx.broadcast(0.0, {3, 224, 224})}])

iex> {result} = Nx.Serving.run(serving, batch)

iex> result |> Nx.backend_transfer() |> Nx.argmax(axis: 1)

#Nx.Tensor<

  s64[1]

  [499]

>

```

## Installation

`Ortex` can be installed by adding `ortex` to your list of dependencies in `mix.exs`:

```elixir

def deps do

  [

    {:ortex, "~> 0.1.9"}

  ]

end

```

You will need [Rust](https://www.rust-lang.org/tools/install) for compilation to succeed.