{"id":15350127,"url":"https://github.com/emergentorder/onnx-scala","last_synced_at":"2025-04-04T10:09:34.496Z","repository":{"id":34697600,"uuid":"144785298","full_name":"EmergentOrder/onnx-scala","owner":"EmergentOrder","description":"An ONNX (Open Neural Network eXchange) API and backend for typeful, functional deep learning and classical machine learning in Scala 3","archived":false,"fork":false,"pushed_at":"2025-03-22T16:15:54.000Z","size":2904,"stargazers_count":138,"open_issues_count":1,"forks_count":8,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-28T09:09:35.144Z","etag":null,"topics":["deep-learning","deep-neural-networks","dotty","jvm","machine-learning","neural-network","onnx","scala","scala3"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EmergentOrder.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-08-15T00:20:30.000Z","updated_at":"2025-03-22T16:15:58.000Z","dependencies_parsed_at":"2023-02-14T09:16:19.087Z","dependency_job_id":"e83c24cc-002b-4e52-9f8a-937071348342","html_url":"https://github.com/EmergentOrder/onnx-scala","commit_stats":{"total_commits":1058,"total_committers":5,"mean_commits":211.6,"dds":0.4130434782608695,"last_synced_commit":"6c1d340e5e703717a4071375467a1da73d33c14f"},"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmergentOrder%2Fonnx-scala","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmergentOrder%2Fonnx-scala/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmergentOrder%2Fonnx-scala/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EmergentOrder%2Fonnx-scala/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EmergentOrder","download_url":"https://codeload.github.com/EmergentOrder/onnx-scala/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247157283,"owners_count":20893220,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-neural-networks","dotty","jvm","machine-learning","neural-network","onnx","scala","scala3"],"created_at":"2024-10-01T11:57:34.950Z","updated_at":"2025-04-04T10:09:34.472Z","avatar_url":"https://github.com/EmergentOrder.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"Logotype-500px.png\" /\u003e\u003c/p\u003e\n\n--------------------------------------------------------------------------------\n\n[![Build status](https://travis-ci.com/EmergentOrder/onnx-scala.svg?branch=master)](http://travis-ci.com/EmergentOrder/onnx-scala)\n[![Latest version](https://index.scala-lang.org/emergentorder/onnx-scala/onnx-scala/latest.svg?color=orange)](https://index.scala-lang.org/emergentorder/onnx-scala/onnx-scala)\n## Getting Started\nAdd this to the build.sbt in your project:\n\n```scala\nlibraryDependencies += \"org.emergent-order\" %% \"onnx-scala-backends\" % \"0.17.0\"\n```\n\nA short, recent talk I gave about the project: [ONNX-Scala: Typeful, Functional Deep Learning / Dotty Meets an Open AI Standard](https://youtu.be/8HuZTeHi7lg?t=1156)\n\n### Full ONNX model inference - quick start\nFirst, download the [model file](https://media.githubusercontent.com/media/onnx/models/main/Computer_Vision/squeezenet1_1_Opset18_torch_hub/squeezenet1_1_Opset18.onnx) for [SqueezeNet](https://en.wikipedia.org/wiki/SqueezeNet).\nYou can use `get_models.sh`\n\nNote that all code snippets are written in Scala 3 (Dotty).\n\nFirst we create an \"image\" tensor composed entirely of pixel value [42](https://upload.wikimedia.org/wikipedia/commons/0/0e/Answer_to_Life_42.svg):\n\n```scala\nimport java.nio.file.{Files, Paths}\nimport org.emergentorder.onnx.Tensors._\nimport org.emergentorder.onnx.Tensors.Tensor._\nimport org.emergentorder.onnx.backends._\nimport org.emergentorder.compiletime._\nimport org.emergentorder.io.kjaer.compiletime._\n\nval squeezenetBytes = Files.readAllBytes(Paths.get(\"squeezenet1_1_Opset18.onnx\"))\nval squeezenet = new ORTModelBackend(squeezenetBytes)\n\nval data = Array.fill(1*3*224*224){42f}\n\n//In NCHW tensor image format\nval shape =                    1     #:     3      #:    224    #: 224     #: SNil\nval tensorShapeDenotation = \"Batch\" ##: \"Channel\" ##: \"Height\" ##: \"Width\" ##: TSNil\n\nval tensorDenotation: String \u0026 Singleton = \"Image\"\n\nval imageTens = Tensor(data,tensorDenotation,tensorShapeDenotation,shape)\n\n//or as a shorthand if you aren't concerned with enforcing denotations\nval imageTensDefaultDenotations = Tensor(data,shape)\n```\n\nNote that ONNX tensor content is in row-major order.\n\nNext we run SqueezeNet image classification inference on it:\n\n```scala\nval out = squeezenet.fullModel[Float, \n                               \"ImageNetClassification\",\n                               \"Batch\" ##: \"Class\" ##: TSNil,\n                               1 #: 1000 #: SNil](Tuple(imageTens))\n// val out:\n//  Tensor[Float,(\"ImageNetClassification\", \n//                \"Batch\" ##: \"Class\" ##: TSNil,\n//                1 #: 1000 #: 1 #: 1 SNil)] = IO(...)\n// ...\n\n//The output shape\nout.shape.unsafeRunSync()\n// val res0: Array[Int] = Array(1, 1000, 1, 1)\n\nval data = out.data.unsafeRunSync()\n// val data: Array[Float] = Array(1.786191E-4, ...)\n\n//The highest scoring and thus highest probability (predicted) class\ndata.indices.maxBy(data)\n// val res1: Int = 753\n```\n\nReferring to the [ImageNet 1000 class labels](https://gist.github.com/yrevar/942d3a0ac09ec9e5eb3a), we see that the predicted class is \"radiator\".\n\nBased on a simple benchmark of 100000 iterations of SqueezeNet inference, the run time is on par (within 3% of) ONNX Runtime (via Python).\nThe discrepancy can be accounted for by the overhead of shipping data between the JVM and native memory.\n\nWhen using this API, we load the provided ONNX model file and pass it as-is to the underlying ONNX backend, which is able to optimize the full graph.\nThis is the most performant execution mode, and is recommended for off-the-shelf models / performance-critical scenarios.\n\nThis full-model API is untyped in the inputs, so it can fail at runtime. This is inevitable because we load models from disk at runtime.\nAn upside of this is that you are free to use dynamic shapes, for example in the case of differing batch sizes per model call (assuming your model supports this via symbolic dimensions, see [ONNX Shape Inference](https://github.com/onnx/onnx/blob/main/docs/ShapeInference.md) ).\nIf your input shapes are static, feel free to wrap your calls into it in a facade with typed inputs.\n\n## Project Details\n\nONNX-Scala is cross-built against Scala JVM, Scala.js/JavaScript and Scala Native (for Scala 3 / Dotty )\n\nCurrently at ONNX 1.17.0 (Backward compatible to at least 1.2.0 for the full model API, 1.7.0 for the fine-grained API), ONNX Runtime 1.20.0.\n \n### Fine-grained API\nA complete\\*, versioned, numerically generic, type-safe / typeful API to ONNX(Open Neural Network eXchange, an open format to represent deep learning and classical machine learning models), derived from the Protobuf definitions and the operator schemas (defined in C++). \n\nWe also provide implementations for each operator in terms of a generic core operator method to be implemented by the backend.\nFor more details on the low-level fine-grained API see [here](FineGrainedAPI.md)\n\nThe preferred high-level fine-grained API, most suitable for the end user, is [NDScala](https://github.com/SciScala/NDScala)\n\n\\* Up to roughly the set of ops supported by ONNX Runtime Web (WebGL backend)\n\n#### Training\nAutomatic differentiation to enable training is under consideration (ONNX currently provides facilities for training as a tech preview only).\n\n#### Type-safe Tensors\nFeaturing type-level tensor and axis labels/denotations, which along with literal types for dimension sizes allow for tensor/axes/shape/data-typed tensors.\nType constraints, as per the ONNX spec, are implemented at the operation level on inputs and outputs, using union types, match types and compiletime singleton ops (thanks to @MaximeKjaer for getting the latter into dotty).\nUsing ONNX docs for [dimension](https://github.com/onnx/onnx/blob/main/docs/DimensionDenotation.md) and [type](https://github.com/onnx/onnx/blob/main/docs/TypeDenotation.md) denotation, as well as the [operators doc](https://github.com/onnx/onnx/blob/v1.7.0/docs/Operators.md) as a reference,\nand inspired by [Nexus](https://github.com/ctongfei/nexus), [Neurocat](https://github.com/mandubian/neurocat) and [Named Tensors](https://pytorch.org/docs/stable/named_tensor.html).\n\n### Backend\nThere is one backend per Scala platform.\nFor the JVM the backend is based on [ONNX Runtime](https://github.com/microsoft/onnxruntime), via their official Java API.\nFor Scala.js / JavaScript the backend is based on the [ONNX Runtime Web](https://github.com/microsoft/onnxruntime/tree/main/js/web).\n\nSupported ONNX input and output tensor data types:\n* Byte\n* Short\n* Int\n* Long\n* Float\n* Double\n* Boolean\n* String\n\nSupported ONNX ops:\n* ONNX-Scala, Fine-grained API: 87/178 total\n* ONNX-Scala, Full model API: Same as below\n\n* ONNX Runtime Web (using Wasm backend): 165/178 total.\n* ONNX Runtime: 165/178 total\n\nSee the [ONNX backend scoreboard](http://onnx.ai/backend-scoreboard/index.html) \n\n#### Example execution\n\nTODO: T5 example\n\n## Build / Publish\n\nYou'll need sbt.\n\nTo build and publish locally:\n\n```\nsbt publishLocal\n```\n\n### Built With\n\n#### Core\n\n* [ONNX](https://github.com/onnx/onnx) via [ScalaPB](https://github.com/scalapb/ScalaPB) - Open Neural Network Exchange / The missing bridge between Java and native C++ libraries (For access to Protobuf definitions, used in the fine-grained API to create ONNX models in memory to send to the backend)\n\n* [Spire](https://github.com/typelevel/spire) - Powerful new number types and numeric abstractions for Scala.  (For support for unsigned ints, complex numbers and the Numeric type class in the core API)\n\n* [Dotty](https://github.com/lampepfl/dotty) - The Scala 3 compiler, also known as Dotty. (For union types (used here to express ONNX type constraints), match types, compiletime singleton ops, ...)\n\n#### Backends\n\n* [ONNX Runtime via ORT Java API](https://github.com/microsoft/onnxruntime/tree/main/java) - ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator\n\n* [ONNX Runtime Web](https://github.com/microsoft/onnxruntime/tree/main/js/web)\n\n### Inspiration\n\n#### Scala\n\n* [Neurocat](https://github.com/mandubian/neurocat) -  From neural networks to the Category of composable supervised learning algorithms in Scala with compile-time matrix checking based on singleton-types\n\n* [Nexus](https://github.com/ctongfei/nexus) - Experimental typesafe tensors \u0026 deep learning in Scala\n\n* [Lantern](https://github.com/feiwang3311/Lantern) - Machine learning framework prototype in Scala. The design of Lantern is built on two important and well-studied programming language concepts, delimited continuations (for automatic differentiation) and multi-stage programming (staging for short).\n\n* [DeepLearning.scala](https://github.com/ThoughtWorksInc/DeepLearning.scala) - A simple library for creating complex neural networks\n\n* [tf-dotty](https://github.com/MaximeKjaer/tf-dotty) - Shape-safe TensorFlow in Dotty \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femergentorder%2Fonnx-scala","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femergentorder%2Fonnx-scala","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femergentorder%2Fonnx-scala/lists"}