{"id":13574025,"url":"https://github.com/jiazhihao/TASO","last_synced_at":"2025-04-04T14:31:30.432Z","repository":{"id":35209095,"uuid":"211426129","full_name":"jiazhihao/TASO","owner":"jiazhihao","description":"The Tensor Algebra SuperOptimizer for Deep Learning","archived":false,"fork":false,"pushed_at":"2023-01-26T06:04:39.000Z","size":1266,"stargazers_count":690,"open_issues_count":55,"forks_count":91,"subscribers_count":24,"default_branch":"master","last_synced_at":"2024-10-30T02:36:47.907Z","etag":null,"topics":["deep-learning","deep-neural-networks","inference-optimization"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jiazhihao.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-28T01:24:21.000Z","updated_at":"2024-10-26T09:30:21.000Z","dependencies_parsed_at":"2023-02-14T14:45:52.787Z","dependency_job_id":null,"html_url":"https://github.com/jiazhihao/TASO","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jiazhihao%2FTASO","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jiazhihao%2FTASO/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jiazhihao%2FTASO/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jiazhihao%2FTASO/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jiazhihao","download_url":"https://codeload.github.com/jiazhihao/TASO/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247194058,"owners_count":20899424,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-neural-networks","inference-optimization"],"created_at":"2024-08-01T15:00:45.271Z","updated_at":"2025-04-04T14:31:30.414Z","avatar_url":"https://github.com/jiazhihao.png","language":"C++","funding_links":[],"categories":["Other Interesting automatic work","C++"],"sub_categories":[],"readme":"# TASO: The Tensor Algebra SuperOptimizer for Deep Learning\n\nTASO optimizes the computation graphs of DNN models using automatically generated and verified graph transformations.\nFor an arbitrary DNN model, TASO uses the auto-generated graph transformations to build a large search space of potential computation graphs that are equivalent to the original DNN model.\nTASO employs a cost-based search algorithm to explore the space, and automatically discovers highly optimized computation graphs.\nTASO outperforms the graph optimizers in existing deep learning frameworks by [up to 3x](http://theory.stanford.edu/~aiken/publications/papers/sosp19.pdf).\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/jiazhihao/TASO/blob/master/figures/inference.png\"\u003e\n  End-to-end inference performance comparison on a NVIDIA V100 GPU.\n\u003c/div\u003e\n\n## Install TASO\n\nSee [instructions](INSTALL.md) to install TASO from source.\nWe also provide prebuilt [docker images](https://github.com/jiazhihao/TASO/blob/master/INSTALL.md) with all dependencies pre-installed.\n\n## Use TASO\n\nTASO can directly optimize any pre-trained DNN models in [ONNX](https://onnx.ai), [TensorFlow](https://www.tensorflow.org/guide/saved_model), and [PyTorch](https://pytorch.org/docs/stable/onnx.html) graph formats.\nTASO also provides a Python interface for optimizing arbitrary DNN architectures.\nTASO supports exporting the optimized computation graphs to ONNX, which can be directly used as inputs by most existing deep learning frameworks.\n\n### Optimize ONNX Models\n\nTASO can directly optimize pre-trained ONNX models, and this can be done in just a few lines of Python code.\nThe following code snippet shows how to load a pre-trained DNN model from ONNX, optimize the model, and save the optimized model into a ONNX file.\n```python\nimport taso\nimport onnx\n\nold_model = taso.load_onnx(\"/path/to/load/onnx/model\")\ntaso_graph = taso.optimize(old_model)\nnew_model = taso.export_onnx(taso_graph)\nonnx.save(new_model, \"/path/to/save/new/onnx/model\")\n```\nThe optimized model has the same accuracy as the original and can be directly used by existing deep learning frameworks.\nSome original and TASO-optimized ONNX files are available in the `onnx` folder.\n\u003c!-- The following figure shows the end-to-end inference performance comparison on a NVIDIA V100 GPU. --\u003e\n\n### Optimize TensorFlow Models\n\nTASO can optimize TensorFlow models by converting the model to ONNX using [tf2onnx](https://github.com/onnx/tensorflow-onnx).\n\n* First, install `tf2onnx` from PyPi as follows or [from source](https://github.com/onnx/tensorflow-onnx).\n```\npip install -U tf2onnx\n```\n\n* Second, convert a TensorFlow model to ONNX using `tf2onnx`.\n```\npython -m tf2onnx.convert \\\n       --saved-model /path/to/tensorflow/saved/model \\\n       --output /path/to/onnx/model/file\n```\n\n* Third, use TASO to optimize the model in ONNX by following the [above instructions](https://github.com/jiazhihao/TASO#optimize-onnx-models).\n\n### Optimize PyTorch Models\n\nPyTorch has built-in support for ONNX as a part of the [torch.onnx](https://pytorch.org/docs/master/onnx.html) package.\nTASO can directly optimize PyTorch models in the ONNX format.\n\n### Optimize Arbitrary DNN Models using the Python Interface\n\nTASO can also optimize arbitrary DNN architectures using the TASO Python interface. \nThe following code snippet builds the left-most DNN graph depicted in the figure. TASO automatically performs a series of non-trivial transformations, and eventually discovers the right-most DNN graph, which is 1.3x faster on a V100 GPU. More DNN examples are available in the `examples` folder.\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/jiazhihao/TASO/blob/master/figures/graph_subst.png\"\u003e\n\u003c/div\u003e\n\n```python\nimport taso\nimport onnx\n\n#Build DNN model\ngraph = taso.new_graph()\ninput = graph.new_input(dims=(1,128,56,56))\nw1 = graph.new_weight(dims=(128,128,3,3))\nw2 = graph.new_weight(dims=(128,128,1,1))\nw3 = graph.new_weight(dims=(128,128,3,3))\nleft = graph.conv2d(input=input, weight=w1, strides=(1,1), padding=\"SAME\", activation=\"RELU\")\nleft = graph.conv2d(input=left, weight=w3, strides=(1,1), padding=\"SAME\")\nright = graph.conv2d(input=input, weight=w2, strides=(1,1), padding=\"SAME\", activation=\"RELU\")\noutput = graph.add(left, right)\noutput = graph.relu(output)\n\n#Optimize DNN model\nnew_graph = taso.optimize(graph)\nonnx_model = taso.export_onnx(new_graph)\nonnx.save(onnx_model, \"/path/to/save/new/onnx/model\")\n```\n\n## Publication\n* Zhihao Jia, Oded Padon, James Thomas, Todd Warszawski, Matei Zaharia, and Alex Aiken. [TASO: Optimizing Deep Learning Computation with Automated Generation of Graph Substitutions](https://cs.stanford.edu/~zhihao/papers/sosp19.pdf). In Proceedings of the Symposium on Operating Systems Principles (SOSP), Ontario, Canada, October 2019.\n\n* Zhihao Jia, James Thomas, Todd Warszawski, Mingyu Gao, Matei Zaharia, and Alex Aiken. [Optimizing DNN Computation with Relaxed Graph Substitutions](https://theory.stanford.edu/~aiken/publications/papers/sysml19b.pdf). In Proceedings of the Conference on Systems and Machine Learning (SysML), Palo Alto, CA, April 2019.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjiazhihao%2FTASO","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjiazhihao%2FTASO","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjiazhihao%2FTASO/lists"}