{"id":13626169,"url":"https://github.com/sonos/tract","last_synced_at":"2026-02-27T15:14:58.545Z","repository":{"id":37003269,"uuid":"99561213","full_name":"sonos/tract","owner":"sonos","description":"Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference","archived":false,"fork":false,"pushed_at":"2025-05-09T09:02:45.000Z","size":45860,"stargazers_count":2423,"open_issues_count":75,"forks_count":222,"subscribers_count":43,"default_branch":"main","last_synced_at":"2025-05-09T10:22:25.324Z","etag":null,"topics":["artificial-intelligence","neural-networks","onnx","rust","rust-library","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sonos.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-08-07T09:31:26.000Z","updated_at":"2025-05-09T09:02:49.000Z","dependencies_parsed_at":"2023-09-25T08:32:39.326Z","dependency_job_id":"3eadcf3f-d5a6-4997-8fe5-2a06fc2436df","html_url":"https://github.com/sonos/tract","commit_stats":{"total_commits":6531,"total_committers":81,"mean_commits":80.62962962962963,"dds":"0.16092482008880726","last_synced_commit":"65ef32da9047a92b78de8f03ffaf8ecd6b45fa3d"},"previous_names":["snipsco/tract","kali/tensorflow-deploy-rust"],"tags_count":452,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonos%2Ftract","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonos%2Ftract/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonos%2Ftract/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonos%2Ftract/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sonos","download_url":"https://codeload.github.com/sonos/tract/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253233348,"owners_count":21875395,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","neural-networks","onnx","rust","rust-library","tensorflow"],"created_at":"2024-08-01T21:02:11.714Z","updated_at":"2026-02-18T11:12:08.231Z","avatar_url":"https://github.com/sonos.png","language":"Rust","funding_links":[],"categories":["Rust","Neural Networks","Machine Learning","Lighter and Deployment Frameworks","Model Inference"],"sub_categories":[],"readme":"![tract-logo](assets/tract-logo/PNG/tract-horizontal-blue.png)\n\n![Rust](https://img.shields.io/badge/rust-%23000000.svg?style=for-the-badge\u0026logo=rust\u0026logoColor=white)\n![rustc \u003e= 1.89.0](https://img.shields.io/badge/rustc-%3E%3D1.89.0-brightgreen)\n![MIT/Apache 2](https://img.shields.io/crates/l/tract)\n[![Native Linux test status](https://github.com/snipsco/tract/workflows/Native%20Linux/badge.svg)](https://github.com/snipsco/tract/actions)\n[![Embedded targets status](https://github.com/snipsco/tract/workflows/Embedded%20targets/badge.svg)](https://github.com/snipsco/tract/actions)\n[![Doc](https://docs.rs/tract-core/badge.svg)](https://docs.rs/tract-core)\n\n[![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge\u0026logo=python\u0026logoColor=ffdd54)](https://pypi.org/project/tract/)\n\n\nSonos' Neural Network inference engine.\n\n_This project used to be called tfdeploy, or Tensorflow-deploy-rust._\n\n## What ?\n\n`tract` is a Neural Network inference toolkit. It can read ONNX or NNEF, optimize them and run them.\n\n## Quick start, examples\n\n* [MobileNet v2 with ONNX](examples/onnx-mobilenet-v2)\n* [BERT example with ONNX](examples/pytorch-albert-v2)\n* [MobileNet v2 with TensorFlow](examples/tensorflow-mobilenet-v2)\n* [From Keras and TensorFlow 2 to tract](examples/keras-tract-tf2)\n* [ResNet with PyTorch](examples/pytorch-resnet)\n\nThere is also [some technical documentation](doc/) and [blog](https://tech-blog.sonos.com/posts/optimising-a-neural-network-for-inference/) posts.\n\n## Tract in the landscape\n\n### ONNX\n\nAs of today, `tract` passes successfully about 85% of ONNX backends\ntests. All \"real life\" integration tests in ONNX test suite are passing: \nbvlc_alexnet, densenet121, inception_v1, inception_v2, resnet50, shufflenet,\nsqueezenet, vgg19, zfnet512.\n\nNotable missing parts are operators dealing with Tensor Sequences and Optional Tensors : tract /really/ wants to flow Tensors and nothing else.\nThis is structural. Changing it would be pretty difficult, and it's unclear whether it can be done without impairing performance or maintainability.\nWe are not convinced these features have shown their interest in the wild yet, so we prefer to leave them aside.\n\nOther dark corners are specific operators like \"Resize\" which fit perfectly in the framework but need a complex internal logic that is far\nfrom our core business. In these cases, we are happy to accept contributions and to help. \n\nThe following operators are implemented and tested.\n\nAbs, Acos, Acosh, Add, And, ArgMax, ArgMin, ArrayFeatureExtractor, Asin, Asinh, Atan, Atanh, AveragePool, BatchNormalization, BitShift, BitwiseAnd, BitwiseNot, BitwiseOr, BitwiseXor, BlackmanWindow, Cast, CastLike, CategoryMapper, Ceil, Clip, Compress, Concat, Constant, ConstantLike, ConstantOfShape, Conv, ConvInteger, ConvTranspose, Cos, Cosh, CumSum, DFT, DepthToSpace, DequantizeLinear, Div, Dropout, DynamicQuantizeLinear, Einsum, Elu, Equal, Erf, Exp, Expand, EyeLike, Flatten, Floor, GRU, Gather, GatherElements, GatherND, Gemm, GlobalAveragePool, GlobalLpPool, GlobalMaxPool, Greater, GreaterOrEqual, HammingWindow, HannWindow, HardSigmoid, Hardmax, Identity, If, InstanceNormalization, IsInf, IsNaN, LRN, LSTM, LeakyRelu, Less, LessOrEqual, Log, LogSoftmax, MatMul, MatMulInteger, Max, MaxPool, Mean, MelWeightMatrix, Min, Mod, Mul, Multinomial, Neg, NonMaxSuppression, NonZero, Not, OneHot, Or, PRelu, Pad, ParametricSoftplus, Pow, QLinearConv, QLinearMatMul, QuantizeLinear, RNN, RandomNormal, RandomNormalLike, RandomUniform, RandomUniformLike, Range, Reciprocal, ReduceL1, ReduceL2, ReduceLogSum, ReduceLogSumExp, ReduceMax, ReduceMean, ReduceMin, ReduceProd, ReduceSum, ReduceSumSquare, Relu, Reshape, Resize, Round, Rsqrt, STFT, ScaledTanh, Scan, Scatter, ScatterElements, ScatterND, Selu, Shape, Shrink, Sigmoid, Sign, Sin, Sinh, Size, Slice, Softmax, Softplus, Softsign, SpaceToDepth, Split, Sqrt, Squeeze, Sub, Sum, Tan, Tanh, ThresholdedRelu, Tile, Transpose, TreeEnsembleClassifier, Unsqueeze, Where, Xor\n\nWe test these operators against from ONNX 1.4.1 (operator set 9), up to ONNX 1.13.0 (operator set 18).\n\nWe are using ONNX test suite, but it does not cover everything.\nWe also deliberately ignore some tests, or restricting their scope depending on what we feel is realistic.\nSometimes these decisions are just wrong, and sometimes they become wrong as time goes by and the fields moves in unexpected directions.\nSo if you are puzzled by an ONNX model that does not work in tract, we are happy to take a look.\n\n### NNEF\n\nLong story short, TensorFlow and ONNX formats are good for designing and\ntraining networks. They need to move fast to follow the research field, tend to\nintegrate new features and operators greedily. They also exhibit a high level\nof expressivity to facilitate network design.\n\nOn the other hand, only a subset of operators and network features actually\nreach production, so systems running production network do not have to deal\nwith so many operators. Furthermore, some information required for training can\nbe stripped from the network before going to production for prediction.\n\nNNEF tries to bridge the gap between training frameworks and inference by\nproposing a format dedicated to production and prediction.\n\nTract supports NNEF:\n\n* tract_nnef can load and execute NNEF networks\n* tract supports most of the NNEF specification, the most notable exception\n    being the ROI operators\n* tract introduces tract-OPL, a series of NNEF extensions to support other\n    operators (or extend some operators semantics) in order to represent the\n    full range of tract-core neural network support: any network understood by\n    tract should be serializable to tract-OPL. This is a work in progress.\n* tract command line can translate networks from TensorFlow or ONNX to NNEF/OPL.\n\n### tract-opl version compatibility\n\nA remainder: NNEF is not expressive enough to represent all ONNX. tract-OPL extends\nNNEF using proprietary to support what is missing. Notable extensions are pulse\noperators, recurring operators (as Scan) and symbolic extensions.\n\nThere is no strict check in place here, so... implementation is not bullet proof.\n* NNEF part aims at being very stable. It is strongly constrained with compatibility\nwith NNEF specification.\n* tract-opl is a bit more in flux. Nevertheless we try to maintain the following\ngolden rule:\n\n     `models serialized with tract 0.x.y should work with tract 0.x.z where z \u003e= y`\n\n* in practice, breaking changes have been relatively rare so far. Most models are\nforward and retro compatible from when tract has acquired NNEF support.\n\nNotable breakage occurred:\n* 0.16.3 (forward compatible) on Scan operator\n* 0.17.0 for binary decision tree classifier\n\nStarting with `0.17.0`, a model property is injected in tract-opl files (`tract_nnef_ser_version`)\nto tag which version of tract generated the file. As most models will remain compatible,\ntract will not do any version check. It is up to the application developer to do so.\n\nA softer version tag exists as `tract_nnef_format_version`. pre-0.17.0 version set it to\n`alpha1`, post-0.17.0 set it `beta1`. Don't put too much emphasis into the \"alpha-ness\" naming \nof versions here.\n\n### Note: support for TensorFlow 1.x\n\nEven if `tract` is very far from supporting any arbitrary model, it can run\nGoogle Inception v3 and Snips wake word models. Missing operators are relatively \neasy to add. The lack of easy to reuse test suite, and the wide diversity of \noperators in Tensorflow make it difficult to target a full support.\n\nThe following operators are implemented and tested:\n\nAbs, Add, AddN, AddV2, Assign, AvgPool, BatchToSpaceND, BiasAdd, BlockLSTM, Cast, Ceil, ConcatV2, Const, Conv2D, DepthwiseConv2dNative, Div, Enter, Equal, Exit, ExpandDims, FakeQuantWithMinMaxVars, Fill, FloorMod, FusedBatchNorm, GatherNd, GatherV2, Greater, GreaterEqual, Identity, Less, LessEqual, Log, LogicalAnd, LogicalOr, LoopCond, MatMul, Max, MaxPool, Maximum, Mean, Merge, Min, Minimum, Mul, Neg, NoOp, Pack, Pad, Placeholder, Pow, Prod, RandomUniform, RandomUniformInt, Range, RealDiv, Relu, Relu6, Reshape, Rsqrt, Shape, Sigmoid, Slice, Softmax, SpaceToBatchND, Squeeze, StridedSlice, Sub, Sum, Switch, Tanh, Tile, Transpose, VariableV2\n\nAdditionally, the complexity of TensorFlow 2 make it very unlikely that a direct\nsupport will ever exist in tract. But many TensorFlow 2 models can be\nconverted to ONNX and then loaded in tract.\n\n## Example of supported networks\n\nThese models among others, are used to track tract performance evolution as\npart of the Continuous Integration jobs. See [.travis/README.md](readme) and \n[.travis/bundle-entrypoint.sh](.travis/bundle-entrypoint.sh) for more\ninformation.\n\n### Keyword spotting on Arm Cortex-M Microcontrollers\n\nhttps://github.com/ARM-software/ML-KWS-for-MCU\n\nARM demonstrated the capabilities of the Cortex-M family by providing\ntutorials and pre-trained models for keyword spotting. While the exercise\nis ultimately meant for micro-controllers, `tract` can run the intermediate\nTensorFlow models.\n\nFor instance, on a Raspberry Pi Zero, the \"CNN M\" model runs in about 70\nmicro-seconds, and 11 micro-seconds on a Raspberry Pi 3.\n\n### Snips wake word models\n\nhttps://arxiv.org/abs/1811.07684\n\nSnips uses `tract` to run the wake word detectors. While earlier models were\nclass-based and did not require any special treatment, `tract` pulsing\ncapabilities made it possible to run WaveNet models efficiently enough for a\nRaspberry Pi Zero.\n\n### Inception v3\n\n|      Device         |      Family    |  TensorFlow-lite  |  tract  |\n|---------------------|----------------|-------------------|---------|\n|  Raspberry Pi Zero  |    Armv6 VFP   |        113s       |   39s   |\n|  Raspberry Pi 2     |    Armv7 NEON  |         25s       |    7s   |\n|  Raspberry Pi 3     |  aarch32 NEON  |          5s       |    5s   |\n\nNotes:\n\n * while the Raspberry Pi 3 is an Armv8 device, this bench is running\n     on Raspbian, an armv6 operating system, crippling the performance\n     of both benches\n * there exists other benches on the internet that show better\n     performance results for TensorFlow (not -Lite) on the Pi 3.\n     They use all four cores of the device. Both TensorFlow-Lite and tract\n     here have been made to run on a single-core.\n\n# License\n\nNote: files in the `tensorflow/protos` directory are copied from the\n[TensorFlow](https://github.com/tensorflow/tensorflow) project and are not\ncovered by the following licence statement.\n\nNote: files in the `onnx/protos` directory are copied from the\n[ONNX](https://github.com/onnx/onnx) project and are not\ncovered by the following license statement.\n\n## Apache 2.0/MIT\n\nAll original work licensed under either of\n * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any Contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall\nbe dual licensed as above, without any additional terms or conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsonos%2Ftract","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsonos%2Ftract","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsonos%2Ftract/lists"}