{"id":46932623,"url":"https://github.com/gomlx/onnx-gomlx","last_synced_at":"2026-03-11T05:01:21.936Z","repository":{"id":260056803,"uuid":"878807846","full_name":"gomlx/onnx-gomlx","owner":"gomlx","description":"ONNX / GoMLX Conversion","archived":false,"fork":false,"pushed_at":"2026-03-02T18:25:13.000Z","size":2115,"stargazers_count":51,"open_issues_count":3,"forks_count":8,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-03-02T21:58:02.069Z","etag":null,"topics":["go","golang","gomlx","machine-learning","onnx"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gomlx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-10-26T06:51:05.000Z","updated_at":"2026-03-02T18:25:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"cdb6e760-f9aa-4c48-bcec-7f5fb76baf8b","html_url":"https://github.com/gomlx/onnx-gomlx","commit_stats":null,"previous_names":["gomlx/onnx-gomlx"],"tags_count":19,"template":false,"template_full_name":null,"purl":"pkg:github/gomlx/onnx-gomlx","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gomlx%2Fonnx-gomlx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gomlx%2Fonnx-gomlx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gomlx%2Fonnx-gomlx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gomlx%2Fonnx-gomlx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gomlx","download_url":"https://codeload.github.com/gomlx/onnx-gomlx/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gomlx%2Fonnx-gomlx/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30372125,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-10T21:41:54.280Z","status":"online","status_checked_at":"2026-03-11T02:00:07.027Z","response_time":84,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","gomlx","machine-learning","onnx"],"created_at":"2026-03-11T05:01:21.208Z","updated_at":"2026-03-11T05:01:21.889Z","avatar_url":"https://github.com/gomlx.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ONNX-GoMLX: Inference and fine-tuning ONNX models with GoMLX\n\n[![GoDev](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go\u0026logoColor=white)](https://pkg.go.dev/github.com/gomlx/onnx-gomlx?tab=doc)\n[![Slack](https://img.shields.io/badge/Slack-GoMLX-purple.svg?logo=slack)](https://app.slack.com/client/T029RQSE6/C08TX33BX6U)\n\n## 📖 Overview\nONNX-GoMLX converts [ONNX models](https://onnx.ai/) (`.onnx` suffix) to \n[GoMLX (an accelerated machine learning framework for Go](https://github.com/gomlx/gomlx).\n\nOne can also fine-tune models with GoMLX and then save back its weights to the ONNX model.\n\nFuture plans also include creating an ONNX backend for GoMLX: so one can execute or export GoMLX models using ONNX.\n\nThe main use cases so far are:\n\n1. **Fine-tuning**: import an inference-only ONNX model to GoMLX and use its auto-differentiation and training loop to\n   fine-tune models. It allows saving the fine-tuned model as a GoMLX checkpoint or exporting the fine-tuned weights\n   back to the ONNX model. This can also be used to expand / combine models.\n2. **Inference**: use an ONNX file using Go and not having to include [ONNX Runtime](https://onnxruntime.ai/) (or Python)\n   -- at the cost of including XLA/PJRT (the current only backend for GoMLX). It also allows one to extend the\n   model with extra ML pre-/post-processing using GoMLX (image transformations, normalization, combining models,\n   building ensembles, etc.). This may be interesting for large/expensive models, or large throughput on large\n   batches.\n    * Notice if you want to simply get a pure Go inference of ONNX models, see \n      [github.com/AdvancedClimateSystems/gonnx](https://github.com/AdvancedClimateSystems/gonnx) or\n      [github.com/oramasearch/onnx-go](https://github.com/oramasearch/onnx-go). They will be slower (~8x based on a SentenceEncoder model, BERT based using `gonnx` vs `ONNXRuntime`) than \n      the XLA inference (or `onnxruntime`) for large projects, but for many use cases it doesn't matter, and they\n      are a much smaller pure Go dependency. Only for CPU (no GPU support).\n\n## Coverage of ONNX Ops Set\n\nThere are at least some 20 or so models that are working so far, but the list is growing.\n\n* [Sentence Encoding all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)\nhas been working perfectly, see example below.\n* [ONNX-GoMLX demo/development notebook](https://github.com/gomlx/onnx-gomlx/blob/main/onnx-go.ipynb): both serve as a functional test and to demo what it can do.\n\nBut **not all operations (\"ops\") are converted yet**. \nIf you try it and find some operation that is not converted, please let us know (create an \"issue\") we will be happy to try to convert them.\nGenerally, all the required scaffolding and tooling are already there, and converting ops has been straightforward.\n\n## 🎓 Example\n\nWe download (and cache) the [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)  \nusing [github.com/gomlx/go-huggingface](https://github.com/gomlx/go-huggingface).\n\nThe tokens in the example are hardcoded for simplicity. See [github.com/gomlx/go-huggingface](https://github.com/gomlx/go-huggingface) for tokenization for various models.\n\n```go\nimport (\n\t\"github.com/gomlx/onnx-gomlx/onnx\"\n    \"github.com/gomlx/go-huggingface/hub\"\n)\n\n...\n\n// Download and cache the ONNX model from HuggingFace.\nhfAuthToken := os.Getenv(\"HF_TOKEN\")\nhfModelID := \"sentence-transformers/all-MiniLM-L6-v2\"\nrepo := hub.New(modelID).WithAuth(hfAuthToken)\nmodelPath := must.M1(repo.DownloadFile(\"onnx/model.onnx\"))\n\n// Parse ONNX model.\nmodel := must.M1(onnx.ReadFile(modelPath))\n\n// Convert ONNX variables (model weights) to GoMLX Context -- which stores variables and can be checkpoint (saved):\nctx := context.New()\nmust.M(model.VariablesToContext(ctx))\n\n// Execute it with GoMLX/XLA:\nsentences := []string{\n    \"This is an example sentence\",\n    \"Each sentence is converted\"}\n//... tokenize ...\ninputIDs := [][]int64{\n    {101, 2023, 2003, 2019, 2742, 6251,  102},\n    { 101, 2169, 6251, 2003, 4991,  102,    0}}\ntokenTypeIDs := [][]int64{\n    {0, 0, 0, 0, 0, 0, 0},\n    {0, 0, 0, 0, 0, 0, 0}}\nattentionMask := [][]int64{\n    {1, 1, 1, 1, 1, 1, 1},\n    {1, 1, 1, 1, 1, 1, 0}}\nvar embeddings []*tensors.Tensor\nembeddings = context.MustExecOnceN( // Execute a GoMLX computation graph with a context\n\tbackends.New(),  // GoMLX backend to use (defaults to XLA) \n\tctx, // Context store the model variables/weights and optional hyperparameters.\n\tfunc (ctx *context.Context, inputs []*Node) []*Node {\n\t\t// Convert ONNX model (in `model`) to a GoMLX computation graph. It returns a slice of values (with only one for this model)\n\t\treturn model.CallGraph(ctx, inputs[0].Graph(), map[string]*Node{\n\t\t\t\"input_ids\": inputs[0],\n\t\t\t\"attention_mask\": inputs[1],\n\t\t\t\"token_type_ids\": inputs[2]}, targetOutputs...)\n\t}, \n\tinputIDs, attentionMask, tokenTypeIDs)  // Inputs to the GoMLX function.\nfmt.Printf(\"Embeddings: %s\", embeddings)\n```\n\nThe output looks like:\n\n```\nEmbeddings: [2][7][384]float32{\n {{-0.0886, -0.0368, 0.0180, ..., 0.0261, 0.0912, -0.0152},\n  {-0.0200, -0.0014, -0.0177, ..., 0.0204, 0.0522, 0.1991},\n  {-0.0196, -0.0336, -0.0319, ..., 0.0203, 0.0709, 0.0644},\n  ...,\n  {-0.0253, 0.0408, 0.0125, ..., -0.0270, 0.0377, 0.1133},\n  {-0.0140, -0.0275, 0.0796, ..., -0.0748, 0.0774, -0.0657},\n  {0.0318, -0.0032, -0.0210, ..., 0.0387, 0.0191, -0.0059}},\n {{-0.0886, -0.0368, 0.0180, ..., 0.0261, 0.0912, -0.0152},\n  {0.0304, 0.0531, -0.0238, ..., -0.1011, 0.0218, 0.0473},\n  {-0.0027, -0.0508, 0.0805, ..., -0.0777, 0.0881, -0.0560},\n  ...,\n  {0.0928, 0.0165, -0.0976, ..., 0.0449, 0.0390, -0.0182},\n  {0.0231, 0.0090, -0.0213, ..., 0.0232, 0.0191, -0.0066},\n  {-0.0213, 0.0019, 0.0043, ..., 0.0561, 0.0170, 0.0256}}}\n```\n\n## Fine-Tuning\n\n1. Extract the ONNX model's weight to GoMLX `Context`: see `Model.VariablesToContext()`.\n2. Use `Model.CallGraph()` in your GoMLX model function (see example just above).\n3. Train model as usual in GoMLX.\n4. Depending on how you are going to use the model:\n   1. Save the model as a GoMLX checkpoint, as usual.\n   2. Save the model by updating the ONNX model: after training use `Model.ContextToONNX()` to copy the update variable  \n      values from GoMLX `Context` back to the ONNX model (in-memory), and then use `Model.Write()` or \n      `Model.SaveToFile()` to save the updated ONNX model to disk.\n\n## Benchmarks\n\nWe have some GoMLX/XLA and ONNX Runtime (Microsoft) benchmarks in [this spreadsheet](https://docs.google.com/spreadsheets/d/1ikpJH6rVVHq8ES-IA8U4lkKH4XsTSpRyZewXwGTgits/edit?usp=sharing), \ntested on the sentence encoder model we were interested in. This was used during development and reflects\nhow it improves performance—the numbers on the bottom of the sheets are the currently accurate.\n\nSee [docs/benchmarks.md](docs/benchmarks.md) for more information.\n   \n## 🤝 Collaborating\n\nCollaboration is very welcome: either in the form of code or simply with ideas with real applicability. Don't\nhesitate to start a discussion or issue in the repository.\n\nYou can find the author and other interested parties in the [Slack channel #gomlx](https://app.slack.com/client/T029RQSE6/C08TX33BX6U) (you can [join the slack server here](https://invite.slack.golangbridge.org/)).\n\nIf you are interested, we have two notebooks we use to compare results. They are a good starting point for anyone curious:\n\n* [Go Version](https://github.com/gomlx/onnx-gomlx/blob/main/onnx-go.ipynb)\n* [ONNX Python Version](https://github.com/gomlx/onnx-gomlx)\n\n## 🥳 Thanks\n\n* This project was born from brainstorming with the talented folks at [KnightAnalytics](https://www.knightsanalytics.com/).\n  Without their insight and enthusiasm this wouldn't have gotten off the ground.\n* [ONNX models](https://onnx.ai/) is such a nice open source standard to communicate models across different implementations.\n* [OpenXLA/XLA](https://github.com/openxla/xla) the open-source backend engine by Google that powers this implementation.\n* Sources of inspiration:\n  * https://github.com/knights-analytics/onnx-gomlx\n  * https://github.com/oramasearch/onnx-go\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgomlx%2Fonnx-gomlx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgomlx%2Fonnx-gomlx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgomlx%2Fonnx-gomlx/lists"}