{"id":18972904,"url":"https://github.com/piotrostr/infer-trt","last_synced_at":"2026-04-07T17:30:23.785Z","repository":{"id":78719786,"uuid":"350743168","full_name":"piotrostr/infer-trt","owner":"piotrostr","description":"Interface for TensorRT engines inference along with an example of YOLOv4 engine being used.","archived":false,"fork":false,"pushed_at":"2022-05-07T13:42:27.000Z","size":18,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-01T08:32:15.252Z","etag":null,"topics":["deep-learning","inference-optimization","object-detection","tensorrt"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/piotrostr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-23T14:29:30.000Z","updated_at":"2024-10-26T21:30:10.000Z","dependencies_parsed_at":"2023-04-23T19:08:28.407Z","dependency_job_id":null,"html_url":"https://github.com/piotrostr/infer-trt","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piotrostr%2Finfer-trt","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piotrostr%2Finfer-trt/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piotrostr%2Finfer-trt/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piotrostr%2Finfer-trt/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/piotrostr","download_url":"https://codeload.github.com/piotrostr/infer-trt/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239970703,"owners_count":19727014,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","inference-optimization","object-detection","tensorrt"],"created_at":"2024-11-08T15:10:15.791Z","updated_at":"2026-04-07T17:30:23.730Z","avatar_url":"https://github.com/piotrostr.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# infer-trt\n\nInterface for TensorRT engines inference along with an example of YOLOv4 engine being used.\n\n\n\u003cimg width=\"500\" src=\"https://user-images.githubusercontent.com/63755291/165267626-1d2ddeb7-bf57-4640-ac33-7f8b3a9bf72d.png\"\u003e\n\n### System Requirements\n\n- amd64/linux architecture\n- Nvidia GPU with Tensor cores\n\n### Setup\nTo start one can fork/clone this repo containing the code.\n\n```bash\ngit clone https://github.com/piotrostr/infer-trt\n```\n\nTensorRT setup is quite complex so it is adviced to use the pre-build nvcr.io container.\n\n```bash\ndocker pull nvcr.io/nvidia/tensorrt:22.03-py3\n```\n\nThe example uses the opencv library, which can be built using the official\ninstructions which can be found in the \n\u003ca href=\"https://docs.opencv.org/4.x/d7/d9f/tutorial_linux_install.html\"\u003eofficial docs\u003c/a\u003e.\nBuilding it even with 8-core-cpu takes quite long so I'd advise to install the binary from conda-forge.\n\n```\nconda install -y -c conda-forge/label/gcc7 opencv\n```\n\n### Usage\n\nNormally, the `yolo.onnx` weights will probably be a result of training, if using PyTorch or TF2 it is easy to make an ONNX export. For the sake of this example, pre-trained weights can be curled or wget'd from the ONNX repo.\n\n```\ncurl -o yolo.onnx https://github.com/onnx/models/blob/main/vision/object_detection_segmentation/yolov3/model/yolov3-10.onnx\n```\n\nIn order to obtain the TensorRT engine for a given model the `trtexec` tool can\nbe used to make an export from onnx weights file.\n\n```bash\n ./trtexec \\\n     --onnx=./yolo.onnx \\\n     --best \\\n     --workspace=1024 \\\n     --saveEngine=./yolo.trt \\\n     --optShapes=input:1x3x416x416\n```\n\nThe `trt_model.py` contains a base class to be used for inheritance. Once the\npreprocesing and postprocessing methods are overriden to match the steps\nrequired per given model, it is ready for inference.\nwith its high-level api:\n\n```python\nimport cv2\nimport pycuda.autoinit\nimport tensorrt as trt\n\nfrom yolo import YOLO\n\nyolo = YOLO(trt.Logger())\nimg = cv2.imread('some_img.png')\nlabels, confidences, bboxes = yolo(img)\n```\n### License\n\n\u003ca href=\"https://github.com/piotrostr/yolo-tensorrt/blob/master/LICENSE\"\u003eMIT\u003c/a\u003e©\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpiotrostr%2Finfer-trt","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpiotrostr%2Finfer-trt","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpiotrostr%2Finfer-trt/lists"}