{"id":13436252,"url":"https://github.com/triple-Mu/YOLOv8-TensorRT","last_synced_at":"2025-03-18T20:31:06.908Z","repository":{"id":65557312,"uuid":"585561886","full_name":"triple-Mu/YOLOv8-TensorRT","owner":"triple-Mu","description":"YOLOv8 using TensorRT accelerate !","archived":false,"fork":false,"pushed_at":"2025-03-08T10:22:58.000Z","size":908,"stargazers_count":1525,"open_issues_count":64,"forks_count":262,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-03-15T07:04:45.047Z","etag":null,"topics":["deepstream","detection","jetson","onnx","pose","segment","tensorrt","yolov8"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/triple-Mu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-05T13:42:59.000Z","updated_at":"2025-03-14T05:25:14.000Z","dependencies_parsed_at":"2024-04-27T15:27:14.890Z","dependency_job_id":"21847b6f-a83e-4f3a-a177-635b6e3acffa","html_url":"https://github.com/triple-Mu/YOLOv8-TensorRT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/triple-Mu%2FYOLOv8-TensorRT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/triple-Mu%2FYOLOv8-TensorRT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/triple-Mu%2FYOLOv8-TensorRT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/triple-Mu%2FYOLOv8-TensorRT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/triple-Mu","download_url":"https://codeload.github.com/triple-Mu/YOLOv8-TensorRT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244301370,"owners_count":20430931,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deepstream","detection","jetson","onnx","pose","segment","tensorrt","yolov8"],"created_at":"2024-07-31T03:00:45.910Z","updated_at":"2025-03-18T20:31:06.902Z","avatar_url":"https://github.com/triple-Mu.png","language":"C++","funding_links":[],"categories":["C++","🔥 Latest YOLO Implementations (2024-2025)","Lighter and Deployment Frameworks","Applications","🛠️ **Implementation Resources \u0026 Tools**"],"sub_categories":["🥈 YOLOv8 \u0026 Earlier Versions","🎯 **YOLO Implementations**"],"readme":"# YOLOv8-TensorRT\n\n`YOLOv8` using TensorRT accelerate !\n\n---\n[![Build Status](https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fatrox%2Fsync-dotenv%2Fbadge\u0026style=flat)](https://github.com/triple-Mu/YOLOv8-TensorRT)\n[![Python Version](https://img.shields.io/badge/Python-3.8--3.10-FFD43B?logo=python)](https://github.com/triple-Mu/YOLOv8-TensorRT)\n[![img](https://badgen.net/badge/icon/tensorrt?icon=azurepipelines\u0026label)](https://developer.nvidia.com/tensorrt)\n[![C++](https://img.shields.io/badge/CPP-11%2F14-yellow)](https://github.com/triple-Mu/YOLOv8-TensorRT)\n[![img](https://badgen.net/github/license/triple-Mu/YOLOv8-TensorRT)](https://github.com/triple-Mu/YOLOv8-TensorRT/blob/main/LICENSE)\n[![img](https://badgen.net/github/prs/triple-Mu/YOLOv8-TensorRT)](https://github.com/triple-Mu/YOLOv8-TensorRT/pulls)\n[![img](https://img.shields.io/github/stars/triple-Mu/YOLOv8-TensorRT?color=ccf)](https://github.com/triple-Mu/YOLOv8-TensorRT)\n\n---\n\n\n# Prepare the environment\n\n1. Install `CUDA` follow [`CUDA official website`](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#download-the-nvidia-cuda-toolkit).\n\n   🚀 RECOMMENDED `CUDA` \u003e= 11.4\n\n2. Install `TensorRT` follow [`TensorRT official website`](https://developer.nvidia.com/nvidia-tensorrt-8x-download).\n\n   🚀 RECOMMENDED `TensorRT` \u003e= 8.4\n\n2. Install python requirements.\n\n   ``` shell\n   pip install -r requirements.txt\n   ```\n\n3. Install [`ultralytics`](https://github.com/ultralytics/ultralytics) package for ONNX export or TensorRT API building.\n\n   ``` shell\n   pip install ultralytics\n   ```\n\n5. Prepare your own PyTorch weight such as `yolov8s.pt` or `yolov8s-seg.pt`.\n\n***NOTICE:***\n\nPlease use the latest `CUDA` and `TensorRT`, so that you can achieve the fastest speed !\n\nIf you have to use a lower version of `CUDA` and `TensorRT`, please read the relevant issues carefully !\n\n# Normal Usage\n\nIf you get ONNX from origin [`ultralytics`](https://github.com/ultralytics/ultralytics) repo, you should build engine by yourself.\n\nYou can only use the `c++` inference code to deserialize the engine and do inference.\n\nYou can get more information in [`Normal.md`](docs/Normal.md) !\n\nBesides, other scripts won't work.\n\n# Export End2End ONNX with NMS\n\nYou can export your onnx model by `ultralytics` API and add postprocess such as bbox decoder and `NMS` into ONNX model at the same time.\n\n``` shell\npython3 export-det.py \\\n--weights yolov8s.pt \\\n--iou-thres 0.65 \\\n--conf-thres 0.25 \\\n--topk 100 \\\n--opset 11 \\\n--sim \\\n--input-shape 1 3 640 640 \\\n--device cuda:0\n```\n\n#### Description of all arguments\n\n- `--weights` : The PyTorch model you trained.\n- `--iou-thres` : IOU threshold for NMS plugin.\n- `--conf-thres` : Confidence threshold for NMS plugin.\n- `--topk` : Max number of detection bboxes.\n- `--opset` : ONNX opset version, default is 11.\n- `--sim` : Whether to simplify your onnx model.\n- `--input-shape` : Input shape for you model, should be 4 dimensions.\n- `--device` : The CUDA deivce you export engine .\n\nYou will get an onnx model whose prefix is the same as input weights.\n\n# Build End2End Engine from ONNX\n### 1. Build Engine by TensorRT ONNX Python api\n\nYou can export TensorRT engine from ONNX by [`build.py` ](build.py).\n\nUsage:\n\n``` shell\npython3 build.py \\\n--weights yolov8s.onnx \\\n--iou-thres 0.65 \\\n--conf-thres 0.25 \\\n--topk 100 \\\n--fp16  \\\n--device cuda:0\n```\n\n#### Description of all arguments\n\n- `--weights` : The ONNX model you download.\n- `--iou-thres` : IOU threshold for NMS plugin.\n- `--conf-thres` : Confidence threshold for NMS plugin.\n- `--topk` : Max number of detection bboxes.\n- `--fp16` : Whether to export half-precision engine.\n- `--device` : The CUDA deivce you export engine .\n\nYou can modify `iou-thres` `conf-thres` `topk` by yourself.\n\n### 2. Export Engine by Trtexec Tools\n\nYou can export TensorRT engine by [`trtexec`](https://github.com/NVIDIA/TensorRT/tree/main/samples/trtexec) tools.\n\nUsage:\n\n``` shell\n/usr/src/tensorrt/bin/trtexec \\\n--onnx=yolov8s.onnx \\\n--saveEngine=yolov8s.engine \\\n--fp16\n```\n\n**If you installed TensorRT by a debian package, then the installation path of `trtexec`\nis `/usr/src/tensorrt/bin/trtexec`**\n\n**If you installed TensorRT by a tar package, then the installation path of `trtexec` is under the `bin` folder in the path you decompressed**\n\n# Build TensorRT Engine by TensorRT API\n\nPlease see more information in [`API-Build.md`](docs/API-Build.md)\n\n***Notice !!!*** We don't support YOLOv8-seg model now !!!\n\n# Inference\n\n## 1. Infer with python script\n\nYou can infer images with the engine by [`infer-det.py`](infer-det.py) .\n\nUsage:\n\n``` shell\npython3 infer-det.py \\\n--engine yolov8s.engine \\\n--imgs data \\\n--show \\\n--out-dir outputs \\\n--device cuda:0\n```\n\n#### Description of all arguments\n\n- `--engine` : The Engine you export.\n- `--imgs` : The images path you want to detect.\n- `--show` : Whether to show detection results.\n- `--out-dir` : Where to save detection results images. It will not work when use `--show` flag.\n- `--device` : The CUDA deivce you use.\n- `--profile` : Profile the TensorRT engine.\n\n## 2. Infer with C++\n\nYou can infer with c++ in [`csrc/detect/end2end`](csrc/detect/end2end) .\n\n### Build:\n\nPlease set you own librarys in [`CMakeLists.txt`](csrc/detect/end2end/CMakeLists.txt) and modify `CLASS_NAMES` and `COLORS` in [`main.cpp`](csrc/detect/end2end/main.cpp).\n\n``` shell\nexport root=${PWD}\ncd csrc/detect/end2end\nmkdir -p build \u0026\u0026 cd build\ncmake ..\nmake\nmv yolov8 ${root}\ncd ${root}\n```\n\nUsage:\n\n``` shell\n# infer image\n./yolov8 yolov8s.engine data/bus.jpg\n# infer images\n./yolov8 yolov8s.engine data\n# infer video\n./yolov8 yolov8s.engine data/test.mp4 # the video path\n```\n\n# TensorRT Segment Deploy\n\nPlease see more information in [`Segment.md`](docs/Segment.md)\n\n# TensorRT Pose Deploy\n\nPlease see more information in [`Pose.md`](docs/Pose.md)\n\n# TensorRT Cls Deploy\n\nPlease see more information in [`Cls.md`](docs/Cls.md)\n\n# TensorRT Obb Deploy\n\nPlease see more information in [`Obb.md`](docs/Obb.md)\n\n# DeepStream Detection Deploy\n\nSee more in [`README.md`](csrc/deepstream/README.md)\n\n# Jetson Deploy\n\nOnly test on `Jetson-NX 4GB`.\nSee more in [`Jetson.md`](docs/Jetson.md)\n\n# Profile you engine\n\nIf you want to profile the TensorRT engine:\n\nUsage:\n\n``` shell\npython3 trt-profile.py --engine yolov8s.engine --device cuda:0\n```\n\n# Refuse To Use PyTorch for Model Inference !!!\n\nIf you need to break away from pytorch and use tensorrt inference,\nyou can get more information in [`infer-det-without-torch.py`](infer-det-without-torch.py),\nthe usage is the same as the pytorch version, but its performance is much worse.\n\nYou can use `cuda-python` or `pycuda` for inference.\nPlease install by such command:\n\n```shell\npip install cuda-python\n# or\npip install pycuda\n```\n\nUsage:\n\n``` shell\npython3 infer-det-without-torch.py \\\n--engine yolov8s.engine \\\n--imgs data \\\n--show \\\n--out-dir outputs \\\n--method cudart\n```\n\n#### Description of all arguments\n\n- `--engine` : The Engine you export.\n- `--imgs` : The images path you want to detect.\n- `--show` : Whether to show detection results.\n- `--out-dir` : Where to save detection results images. It will not work when use `--show` flag.\n- `--method` : Choose `cudart` or `pycuda`, default is `cudart`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftriple-Mu%2FYOLOv8-TensorRT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftriple-Mu%2FYOLOv8-TensorRT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftriple-Mu%2FYOLOv8-TensorRT/lists"}