{"id":22923679,"url":"https://github.com/mithril-security/tract-sgx-xargo","last_synced_at":"2025-04-01T14:45:08.811Z","repository":{"id":40793238,"uuid":"438958815","full_name":"mithril-security/tract-sgx-xargo","owner":"mithril-security","description":null,"archived":false,"fork":false,"pushed_at":"2022-10-29T14:22:55.000Z","size":22674,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-07T09:27:05.125Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mithril-security.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-16T11:04:01.000Z","updated_at":"2022-02-14T21:05:38.000Z","dependencies_parsed_at":"2023-01-19T21:32:51.984Z","dependency_job_id":null,"html_url":"https://github.com/mithril-security/tract-sgx-xargo","commit_stats":null,"previous_names":[],"tags_count":100,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mithril-security%2Ftract-sgx-xargo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mithril-security%2Ftract-sgx-xargo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mithril-security%2Ftract-sgx-xargo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mithril-security%2Ftract-sgx-xargo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mithril-security","download_url":"https://codeload.github.com/mithril-security/tract-sgx-xargo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246659618,"owners_count":20813331,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-14T08:16:19.577Z","updated_at":"2025-04-01T14:45:08.496Z","avatar_url":"https://github.com/mithril-security.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"![tract-logo](assets/tract-logo/PNG/tract-horizontal-blue.png)\n\n![rustc \u003e= 1.59.0](https://img.shields.io/badge/rustc-%3E%3D1.59.0-brightgreen)\n![MIT/Apache 2](https://img.shields.io/crates/l/tract)\n[![Native Linux test status](https://github.com/snipsco/tract/workflows/Native%20Linux/badge.svg)](https://github.com/snipsco/tract/actions)\n[![Embedded targets status](https://github.com/snipsco/tract/workflows/Embedded%20targets/badge.svg)](https://github.com/snipsco/tract/actions)\n[![Doc](https://docs.rs/tract-core/badge.svg)](https://docs.rs/tract-core)\n\nSonos' Neural Network inference engine.\n\n_This project used to be called tfdeploy, or Tensorflow-deploy-rust._\n\n## SGX Related\n\nMultiple modules of `tract` are working for Rust SGX, such as: \n* `tract-data`\n* `tract-linalg`\n* `tract-core`\n* `tract-hir`\n* `tract-nnef`\n* `tract-onnx`\n* `tract-onnx-opl`\n\nThose modules are NOT ported yet: \n* `tract-tensorflow`\n* `tract-pulse`\n* `tract-pulse-opl`\n* `tract-kaidi`\n\nIn an `SGX` environment, all functions related to the `filesystem` are DISABLED.\nYou can however enable back the reading from disk functions by passing the feature `untrusted_fs` to Tract.\n\n## What ?\n\n`tract` is a Neural Network inference toolkit. It can read Tensorflow 1, ONNX\nor NNEF, optimize them and run data through them.\n\n## Quick start\n\n* [MobileNet v2 with ONNX](examples/onnx-mobilenet-v2)\n* [MobileNet v2 with ONNX and batch](examples/onnx-mobilenet-v2-batch)\n* [BERT example with ONNX](examples/pytorch-albert-v2)\n* [MobileNet v2 with TensorFlow](examples/tensorflow-mobilenet-v2)\n* [From Keras and TensorFlow 1 in Jupyter to tract](examples/jupyter-keras-tract-tf1)\n* [From Keras and TensorFlow 2 in Jupyter to tract](examples/jupyter-keras-tract-tf2)\n* [ResNet with PyTorch](examples/pytorch-resnet)\n\nThere is also [some technical documentation](doc/) and [blog](https://tech-blog.sonos.com/posts/optimising-a-neural-network-for-inference/) posts.\n\n## Tract in the landscape\n\n### ONNX\n\nAs of today (October 2020), `tract` passes successfully about 85% of ONNX backends\ntests. All \"real life\" integration tests in Onnx test suite are passing: \nbvlc_alexnet, densenet121, inception_v1, inception_v2, resnet50, shufflenet,\nsqueezenet, vgg19, zfnet512.\n\nThe following operators are implemented and tested.\n\nAbs, Acos, Acosh, Add, And, ArgMax, ArgMin, Asin, Asinh, Atan, Atanh, AveragePool, BatchNormalization, BitShift, Cast, CategoryMapper, Ceil, Clip, Compress, Concat, Constant, ConstantLike, ConstantOfShape, Conv, ConvInteger, ConvTranspose, Cos, Cosh, CumSum, DepthToSpace, DequantizeLinear, Div, Dropout, DynamicQuantizeLinear, Elu, Equal, Erf, Exp, Expand, EyeLike, Flatten, Floor, GRU, Gather, GatherElements, GatherND, Gemm, GlobalAveragePool, GlobalLpPool, GlobalMaxPool, Greater, GreaterOrEqual, HardSigmoid, Hardmax, Identity, If, InstanceNormalization, IsInf, IsNaN, LRN, LSTM, LeakyRelu, Less, LessOrEqual, Log, LogSoftmax, MatMul, MatMulInteger, Max, MaxPool, Mean, Min, Mod, Mul, Neg, NonZero, Not, OneHot, Or, PRelu, Pad, ParametricSoftplus, Pow, QLinearConv, QLinearMatMul, QuantizeLinear, RNN, Range, Reciprocal, ReduceL1, ReduceL2, ReduceLogSum, ReduceLogSumExp, ReduceMax, ReduceMean, ReduceMin, ReduceProd, ReduceSum, ReduceSumSquare, Relu, Reshape, Resize, Round, Rsqrt, ScaledTanh, Scan, Scatter, ScatterElements, ScatterND, Selu, Shape, Shrink, Sigmoid, Sign, Sin, Sinh, Size, Slice, Softmax, Softplus, Softsign, SpaceToDepth, Split, Sqrt, Squeeze, Sub, Sum, Tan, Tanh, ThresholdedRelu, Tile, Transpose, TreeEnsembleClassifier, Unsqueeze, Where, Xor\n\n\nWe test these operators against Onnx 1.4.1 (operator set 9), Onnx 1.5.0\n(operator set 10), Onnx 1.6.0 (operator set 11), Onnx 1.7.0 (operator set\n12), Onnx 1.8.1 (operator set 13), Onnx 1.9.0 (operator set 14), and Onnx\n1.10.1 (operator set 15).\nMany networks in operator set 8 are also working.\n\n### TensorFlow 1.x\n\nEven if `tract` is very far from supporting any arbitrary model, it can run\nGoogle Inception v3 and Snips wake word models. Missing operators are relatively \neasy to add. The lack of easy to reuse test suite, and the wide diversity of \noperators in Tensorflow make it difficult to target a full support.\n\nThe following operators are implemented and tested:\n\nAbs, Add, AddN, AddV2, Assign, AvgPool, BatchToSpaceND, BiasAdd, BlockLSTM, Cast, Ceil, ConcatV2, Const, Conv2D, DepthwiseConv2dNative, Div, Enter, Equal, Exit, ExpandDims, FakeQuantWithMinMaxVars, Fill, FloorMod, FusedBatchNorm, GatherNd, GatherV2, Greater, GreaterEqual, Identity, Less, LessEqual, Log, LogicalAnd, LogicalOr, LoopCond, MatMul, Max, MaxPool, Maximum, Mean, Merge, Min, Minimum, Mul, Neg, NoOp, Pack, Pad, Placeholder, Pow, Prod, RandomUniform, RandomUniformInt, Range, RealDiv, Relu, Relu6, Reshape, Rsqrt, Shape, Sigmoid, Slice, Softmax, SpaceToBatchND, Squeeze, StridedSlice, Sub, Sum, Switch, Tanh, Tile, Transpose, VariableV2\n\nAddiotionaly, the complexity of TensorFlow 2 make it very unlikely that a direct\nsupport will ever exist in tract. Many TensorFlow 2 nets can be\nconverted to ONNX and loaded in tract.\n\n### NNEF\n\nLong story short, TensorFlow and Onnx formats are good for designing and\ntraining networks. They need to move fast to follow the research field, tend to\nintegrate new features and operators greedily. They also exhibit a high level\nof expressivity to facilitate network design.\n\nOn the other hand, only a subset of operators and network features actually\nreach production, so systems running production network do not have to deal\nwith so many operators. Furthermore, some information required for training can\nbe stripped from the network before going to production for prediction.\n\nNNEF tries to bridge the gap between training frameworks and inference by\nproposing a format dedicated to production and prediction.\n\nTract supports NNEF:\n\n* tract_nnef can load and execute NNEF networks\n* tract supports most of the NNEF specification, the most notable exception\n    being the ROI operators and deconvolution\n* tract introduces tract-OPL, a series of NNEF extensions to support other\n    operators (or extend some operators semantics) in order to represent the\n    full range of tract-core neural network support: any network understood by\n    tract should be serializable to tract-OPL. This is a work in progress.\n* tract command line can translate networks from TensorFlow or ONNX to NNEF/OPL.\n\n## Example of supported networks\n\nThese models among others, are used to track tract performance evolution as\npart of the Continuous Integration jobs. See [.travis/README.md](readme) and \n[.travis/bundle-entrypoint.sh](.travis/bundle-entrypoint.sh) for more\ninformation.\n\n### Keyword spotting on Arm Cortex-M Microcontrollers\n\nhttps://github.com/ARM-software/ML-KWS-for-MCU\n\nARM demonstrated the capabilited of the Cortex-M family by providing\ntutorials and pre-trained models for keyword spotting. While the exercise\nis ultimately meant for micro-controllers, `tract` can run the intermediate\nTensorFlow models.\n\nFor instance, on a Rasperry Pi Zero, the \"CNN M\" model runs in about 70\nmicro-seconds, and 11 micro-seconds on a Raspberry Pi 3.\n\n### Snips wake word models\n\nhttps://arxiv.org/abs/1811.07684\n\nSnips uses `tract` to run the wake word detectors. While earlier models were\nclass-based and did not require any special treatment, `tract` pulsing\ncapabilities made it possible to run WaveNet models efficiently enough for a\nRaspberry Pi Zero.\n\n### Inception v3\n\n|      Device         |      Family    |  TensorFlow-lite  |  tract  |\n|---------------------|----------------|-------------------|---------|\n|  Raspberry Pi Zero  |    Armv6 VFP   |        113s       |   39s   |\n|  Raspberry Pi 2     |    Armv7 NEON  |         25s       |    7s   |\n|  Raspberry Pi 3     |  aarch32 NEON  |          5s       |    5s   |\n\nNotes:\n\n * while the Raspberry Pi 3 is an Armv8 device, this bench is running\n     on Raspbian, an armv6 operating system, crippling the performance\n     of both benches\n * there exists other benches on the internet that show better\n     performance results for TensorFlow (not -Lite) on the Pi 3.\n     They use all four cores of the device. Both TensorFlow-Lite and tract\n     here have been made to run on a single-core.\n\n# License\n\nNote: files in the `tensorflow/protos` directory are copied from the\n[TensorFlow](https://github.com/tensorflow/tensorflow) project and are not\ncovered by the following licence statement.\n\nNote: files in the `onnx/protos` directory are copied from the\n[ONNX](https://github.com/onnx/onnx) project and are not\ncovered by the following licence statement.\n\n## Apache 2.0/MIT\n\nAll original work licensed under either of\n * Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall\nbe dual licensed as above, without any additional terms or conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmithril-security%2Ftract-sgx-xargo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmithril-security%2Ftract-sgx-xargo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmithril-security%2Ftract-sgx-xargo/lists"}