{"id":17227179,"url":"https://github.com/triple-mu/tensorrt2onnx","last_synced_at":"2025-04-14T01:14:35.902Z","repository":{"id":62678352,"uuid":"561632193","full_name":"triple-Mu/TensorRT2ONNX","owner":"triple-Mu","description":"A tool convert TensorRT engine/plan to a fake onnx","archived":false,"fork":false,"pushed_at":"2022-11-22T10:48:47.000Z","size":13,"stargazers_count":38,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-14T01:14:25.110Z","etag":null,"topics":["onnx","tensorrt"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/triple-Mu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-11-04T05:52:09.000Z","updated_at":"2025-02-27T01:58:39.000Z","dependencies_parsed_at":"2023-01-22T14:31:07.517Z","dependency_job_id":null,"html_url":"https://github.com/triple-Mu/TensorRT2ONNX","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/triple-Mu%2FTensorRT2ONNX","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/triple-Mu%2FTensorRT2ONNX/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/triple-Mu%2FTensorRT2ONNX/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/triple-Mu%2FTensorRT2ONNX/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/triple-Mu","download_url":"https://codeload.github.com/triple-Mu/TensorRT2ONNX/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248804825,"owners_count":21164135,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["onnx","tensorrt"],"created_at":"2024-10-15T04:18:26.040Z","updated_at":"2025-04-14T01:14:35.876Z","avatar_url":"https://github.com/triple-Mu.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TensorRT2ONNX\n\nA tool convert TensorRT engine/plan to a fake onnx\n\n## Build an engine using C++ or Python api\n\nSet building config with `DETAILED` flag.\n\n### C++\n\n```cpp\nconfig-\u003esetProfilingVerbosity(ProfilingVerbosity::kDETAILED);\n```\n\n### Python\n\n```python\nconfig.profiling_verbosity = trt.ProfilingVerbosity.DETAILED\n```\n\n## Build an engine from onnx using trtexec tools\n\n```shell\ntrtexec --verbose \\\n        --nvtxMode=verbose \\\n        --buildOnly \\\n        --workspace=8192 \\\n        --onnx=your_onnx.onnx \\\n        --saveEngine=your_engine.engine \\\n        --timingCacheFile=timing.cache \\\n        --fp16 # use fp16\n```\n\nNotice: `--nvtxMode=verbose` is the same as `--profilingVerbosity=detailed`\n\nYou will get a `your_engine.engine` and a `timing.cache`\n\n## Parser network from engine using trtexec tools\n\n```shell\ntrtexec --verbose \\\n        --noDataTransfers \\\n        --useCudaGraph \\\n        --separateProfileRun \\\n        --useSpinWait \\\n        --nvtxMode=verbose \\\n        --loadEngine=your_engine.engine \\\n        --exportLayerInfo=graph.json \\\n        --timingCacheFile=timing.cache\n```\n\nYou will parser `your_engine.engine` network information into `graph.json`\n\n## Install TensorRT2ONNX\n\n```shell\npip3 install trt2onnx -i https://pypi.org/simple\n```\n\n## Build a fake onnx from graph json\n\n```python\nimport onnx\nfrom trt2onnx import build_onnx\n\n# build a fake onnx from json\nonnx_graph = build_onnx('graph.json')\n\n# save the fake onnx as `fake.onnx`\nonnx.save(onnx_graph, 'fake.onnx')\n```\n\n## Build a fake onnx from engine\n\nYou must build engine with flag `ProfilingVerbosity=DETAILED`.\n\n```python\nimport onnx\nfrom trt2onnx import build_onnx\n\n# build a fake onnx from engine\nonnx_graph = build_onnx('your_engine.engine')\n\n# save the fake onnx as `fake.onnx`\nonnx.save(onnx_graph, 'fake.onnx')\n```\n\n**NOTICE !!**\n\nIf you build engine use your own plugin,\nplease load the `*.so` before `build_onnx` function.\n\n```python\nimport ctypes\n# load your plugin first\nctypes.cdll.LoadLibrary('your_plugin_0.so')\nctypes.cdll.LoadLibrary('your_plugin_1.so')\n...\n```\n\n## A demo for resnet50\n\n```python\nimport torch\nimport onnx\nfrom trt2onnx import build_onnx\nimport tensorrt as trt\nfrom torchvision.models import resnet50, ResNet50_Weights\ndevice = torch.device('cuda:0')\nresnet = resnet50(weights=ResNet50_Weights.IMAGENET1K_V1).to(device)\nresnet.eval()\nfake_input = torch.randn(1,3,224,224).to(device)\n# dry run\nresnet(fake_input)\n# export onnx you will get `resnet50.onnx`\ntorch.onnx.export(resnet, fake_input, 'resnet50.onnx', opset_version=11)\n# build engine\nlogger = trt.Logger(trt.Logger.ERROR)\nbuilder = trt.Builder(logger)\nconfig = builder.create_builder_config()\nconfig.max_workspace_size = torch.cuda.get_device_properties(device).total_memory\nflag = (1 \u003c\u003c int(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH))\nnetwork = builder.create_network(flag)\nparser = trt.OnnxParser(network, logger)\nparser.parse_from_file('resnet50.onnx')\n# fp16 export\nif builder.platform_has_fast_fp16:\n    config.set_flag(trt.BuilderFlag.FP16)\n# set detail flag\nconfig.profiling_verbosity = trt.ProfilingVerbosity.DETAILED\n# get `resnet50.engine`\nwith open('resnet50.engine','wb') as f, builder.build_engine(network, config) as engine:\n    f.write(engine.serialize())\n# get fake onnx\nfake_onnx = build_onnx('resnet50.engine')\n# save fake onnx\nonnx.save(fake_onnx, 'fake_onnx.onnx')\n```\n\n## Use [Netron](https://github.com/lutzroeder/netron) to view your fake onnx\n\n![image](https://user-images.githubusercontent.com/92794867/199899590-4af79b85-2114-40f2-b43b-c8bcf71830e2.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftriple-mu%2Ftensorrt2onnx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftriple-mu%2Ftensorrt2onnx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftriple-mu%2Ftensorrt2onnx/lists"}