{"id":13438405,"url":"https://github.com/zerollzeng/tiny-tensorrt","last_synced_at":"2025-03-20T05:31:57.880Z","repository":{"id":41564932,"uuid":"203768295","full_name":"zerollzeng/tiny-tensorrt","owner":"zerollzeng","description":"Deploy your model with TensorRT quickly. ","archived":false,"fork":false,"pushed_at":"2023-11-21T13:25:19.000Z","size":1235,"stargazers_count":755,"open_issues_count":9,"forks_count":99,"subscribers_count":28,"default_branch":"master","last_synced_at":"2024-08-01T03:27:15.805Z","etag":null,"topics":["deep-learning","onnx","tensorrt"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zerollzeng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-08-22T10:10:19.000Z","updated_at":"2024-07-17T12:37:06.000Z","dependencies_parsed_at":"2024-01-16T01:37:10.620Z","dependency_job_id":"7b721795-7887-49b6-a7ee-f2bbb2200469","html_url":"https://github.com/zerollzeng/tiny-tensorrt","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerollzeng%2Ftiny-tensorrt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerollzeng%2Ftiny-tensorrt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerollzeng%2Ftiny-tensorrt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerollzeng%2Ftiny-tensorrt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zerollzeng","download_url":"https://codeload.github.com/zerollzeng/tiny-tensorrt/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221735087,"owners_count":16872224,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","onnx","tensorrt"],"created_at":"2024-07-31T03:01:05.268Z","updated_at":"2024-10-27T21:30:17.234Z","avatar_url":"https://github.com/zerollzeng.png","language":"C++","readme":"\u003c!--\n * @Description: In User Settings Edit\n * @Author: zerollzeng\n * @Date: 2019-08-23 09:16:35\n * @LastEditTime: 2020-03-06 17:12:14\n * @LastEditors: zerollzeng\n --\u003e\n\n![image](https://user-images.githubusercontent.com/38289304/71065174-aafc3100-21ab-11ea-9bcb-665d38181e74.png)\n\n**This Project is no longer maintained since we already have better alternatives for engine build, you can use TensorRT's python API, or make use of trtexec/polygraphy tool to build the engine quickly**\n\n**For any issue about TensorRT, you can file issue against https://github.com/NVIDIA/TensorRT/issues**\n\n## tiny-tensorrt\nAn easy-to-use nvidia TensorRT wrapper for onnx model with c++ and python api. you are able to deploy your model with tiny-tensorrt in few lines of code!\n\n```c++\nTrt* net = new Trt();\nnet-\u003eSetFP16();\nnet-\u003eBuildEngine(onnxModel, engineFile);\nnet-\u003eCopyFromHostToDevice(input, inputBindIndex);\nnet-\u003eForward();\nnet-\u003eCopyFromDeviceToHost(output, outputBindIndex)\n```\n\n## Install\n\ntiny-tensorrt rely on CUDA, CUDNN and TensorRT. Make sure you has installed those dependencies already. For a quick start, you can use [official docker](https://ngc.nvidia.com/catalog/containers/nvidia:tensorrt)\n\nSupport CUDA version: 10.2, 11.0, 11.1, 11.2, 11.3, 11.4\n\nSupport TensorRT version: 7.0, 7.1, 7.2, 8.0, 8.2 8.4\n\nTo build tiny-tensorrt, you also need some extra packages.\n```bash\nsudo apt-get update -y\nsudo apt-get install cmake zlib1g-dev\n\n## this is for python binding\nsudo apt-get install python3 python3-pip\npip3 install numpy\n\n## clone project and submodule\ngit clone --recurse-submodules -j8 https://github.com/zerollzeng/tiny-tensorrt.git\n\ncd tiny-tensorrt\nmkdir build \u0026\u0026 cd build\n\ncmake .. \u0026\u0026 make\n```\nThen you can intergrate it into your own project with libtinytrt.so and Trt.h, for python module, you get pytrt.so\n\n## Docs\n\nPlease refer to [Wiki](https://github.com/zerollzeng/tiny-tensorrt/wiki)\n\n## About License\n\nFor the 3rd-party module and TensorRT, you need to follow their license\n\nFor the part I wrote, you can do anything you want\n\n","funding_links":[],"categories":["C++"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzerollzeng%2Ftiny-tensorrt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzerollzeng%2Ftiny-tensorrt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzerollzeng%2Ftiny-tensorrt/lists"}