{"id":23894355,"url":"https://github.com/princep/tensorrt-cpp-for-onnx","last_synced_at":"2025-04-10T13:33:55.983Z","repository":{"id":226852753,"uuid":"769299855","full_name":"PrinceP/tensorrt-cpp-for-onnx","owner":"PrinceP","description":"Tensorrt codebase to inference in c++ for all major neural arch using onnx","archived":false,"fork":false,"pushed_at":"2025-02-08T14:48:50.000Z","size":25678,"stargazers_count":31,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-24T12:13:02.480Z","etag":null,"topics":["depth-anything","depth-anything-v2","rt-detr","yolo-world","yolo-world-v2","yolov10","yolov5-face","yolov5-face-landmark","yolov8","yolov8-classification","yolov8-detection","yolov8-obb","yolov8-pose","yolov8-seg","yolov9"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PrinceP.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-08T18:57:48.000Z","updated_at":"2025-03-16T11:59:49.000Z","dependencies_parsed_at":"2024-11-26T03:17:19.020Z","dependency_job_id":null,"html_url":"https://github.com/PrinceP/tensorrt-cpp-for-onnx","commit_stats":null,"previous_names":["princep/tensorrt-cpp-for-onnx"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PrinceP%2Ftensorrt-cpp-for-onnx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PrinceP%2Ftensorrt-cpp-for-onnx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PrinceP%2Ftensorrt-cpp-for-onnx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PrinceP%2Ftensorrt-cpp-for-onnx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PrinceP","download_url":"https://codeload.github.com/PrinceP/tensorrt-cpp-for-onnx/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248225847,"owners_count":21068078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["depth-anything","depth-anything-v2","rt-detr","yolo-world","yolo-world-v2","yolov10","yolov5-face","yolov5-face-landmark","yolov8","yolov8-classification","yolov8-detection","yolov8-obb","yolov8-pose","yolov8-seg","yolov9"],"created_at":"2025-01-04T14:57:13.467Z","updated_at":"2025-04-10T13:33:55.972Z","avatar_url":"https://github.com/PrinceP.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# \u003cdiv align=\"center\"\u003eTENSORRT CPP FOR ONNX\u003c/d\u003e\n \n\nTensorrt codebase in c++ to inference for all major neural arch using onnx and dynamic batching\n\n\n## \u003cdiv align=\"left\"\u003eNVIDIA Driver\u003c/d\u003e\n\n```bash\n\nwget https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda_12.4.0_550.54.14_linux.run\n\nsudo sh cuda_12.4.0_550.54.14_linux.run\n\n```\n\n## \u003cdiv align=\"left\"\u003eDocker\u003c/d\u003e\n\n```bash\n\nsudo docker build -t trt_24.02_opencv  .\n\nsudo docker run --rm --network=\"host\" -v $(pwd):/app -it --runtime nvidia trt_24.02_opencv bash\n```\n\n## \u003cdiv align=\"center\"\u003eModels\u003c/div\u003e\n\n### \u003cdiv align=\"left\"\u003eDepth anything v2\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/DepthAnything/Depth-Anything-V2.git\n\n- Clone the Depth-Anything\n```bash\n\ngit clone https://github.com/DepthAnything/Depth-Anything-V2.git\ncd Depth-Anything-V2/\npip3 install -r requirements.txt\n\nFollow steps for making onnx model files\nDownload the model from here https://github.com/DepthAnything/Depth-Anything-V2?tab=readme-ov-file#prepraration\n\n\ngit clone https://github.com/spacewalk01/depth-anything-tensorrt.git\ncp depth_anything_v2/dpt.py ../depth_anything_v2/\ncp depth_anything_v2/export_v2.py ../\ncd ..\n\npython3.11 export_v2.py --encoder vitb --input-size 518\nModel exported to depth_anything_v2_vitb.onnx\n\npython3.11 export_v2.py --encoder vits --input-size 518\nModel exported to depth_anything_v2_vits.onnx\n\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/depth-anything'\ncp depth_anything_v2_vit\u003cb/s/l\u003e.onnx /app/examples/depth-anything\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./depth_anything_v2 /app/examples/depth-anything/depth_anything_v2_vit\u003cb/s/l\u003e.onnx /app/data/depth-anything/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [depth_anything_v2_vitb, Batchsize = 1, Model size = 518x518]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/depth_anything_v2_demo02.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/depth_anything_v2_demo06.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n### \u003cdiv align=\"left\"\u003eYOLO-World\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/AILab-CVC/YOLO-World\n\n- Clone the YOLO-World\n```bash\n\ngit clone https://github.com/AILab-CVC/YOLO-World\n\nFollow steps for installation\nhttps://github.com/AILab-CVC/YOLO-World?tab=readme-ov-file#1-installation\n\nDefine custom classes(Any name can be defined)\necho '[[\"helmet\"], [\"head\"],[\"sunglasses\"]]' \u003e custom_class.json\n\nPYTHONPATH=./ python3 deploy/export_onnx.py configs/pretrain/yolo_world_v2_s_vlpan_bn_2e-3_100e_4x8gpus_obj365v1_goldg_train_lvis_minival.py yolo_world_v2_s_obj365v1_goldg_pretrain-55b943ea.pth --custom-text custom_class.json --opset 12 --without-nms\n\nAfter the fix https://github.com/AILab-CVC/YOLO-World/pull/416\npython3 deploy/onnx_demo.py ./work_dirs/yolo_world_v2_s_obj365v1_goldg_pretrain-55b943ea.onnx ~/disk1/uncanny/projects/tensorrt-cpp-for-onnx/data/ custom_class.json  --onnx-nms\n\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\nAdjust settings in the ./examples/yolo-world/main.cpp \n0.019/*score_threshold*/, 0.7/*iou_threshold*/, 300/*max_detections*/\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/yolo-world'\ncp cp ./work_dirs/\u003cmodel_version\u003e.onnx /app/examples/yolo-world\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./yolo-world /app/examples/yolo-world/\u003cmodel_version\u003e.onnx /app/data/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [yolo_world_v2_s_obj365v1_goldg_pretrain, Batchsize = 1, Model size = 640x640]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/yoloworld_bus.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/yoloworld_zidane.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center; padding: 10px\"\u003e\n    \u003cimg src=\"./results/yoloworld_test.jpeg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n### \u003cdiv align=\"left\"\u003eRT-DETR\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/lyuwenyu/RT-DETR.git\n\n- Clone the RT-DETR\n```bash\n\ngit clone https://github.com/lyuwenyu/RT-DETR.git\n\nFollow steps from here \n\nVersion1\nhttps://github.com/lyuwenyu/RT-DETR/blob/main/rtdetr_pytorch/README.md#todo\n\nVersion2\nhttps://github.com/lyuwenyu/RT-DETR/tree/main/rtdetrv2_pytorch#quick-start\n\nAny of above can run.\n\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/rt-detr'\ncp \u003cmodel_version\u003e.onnx /app/examples/rt-detr\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./rt-detr /app/examples/rt-detr/\u003cmodel_version\u003e.onnx /app/data/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [RT-DETRv2-S, Batchsize = 2, Model size = 640x640]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/rt-detr_bus.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/rt-detr_zidane.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center; padding: 10px\"\u003e\n    \u003cimg src=\"./results/rt-detr_test.jpeg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n\n\n### \u003cdiv align=\"left\"\u003eYOLOV10\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/THU-MIG/yolov10\n\n- Clone the yolov10\n```bash\n\ngit clone https://github.com/THU-MIG/yolov10\n\nyolo export model=yolov10n/s/m/b/l/x.pt format=onnx opset=13 simplify dynamic\n\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/yolov10'\ncp \u003cmodel_version\u003e.onnx /app/examples/yolov10\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./yolov10 /app/examples/yolov10/\u003cmodel_version\u003e.onnx /app/data/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [YOLOv10m, Batchsize = 2, Model size = 640x640]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v10_bus.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v10_zidane.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center; padding: 10px\"\u003e\n    \u003cimg src=\"./results/v10_test.jpeg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n\n### \u003cdiv align=\"left\"\u003eYOLOV9\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/WongKinYiu/yolov9.git\n\ncommit 380284cb66817e9ffa30a80cad4c1b110897b2fb\n\n- Clone the yolov9\n```bash\n\ngit clone https://github.com/WongKinYiu/yolov9\n\npython3 export.py --weights \u003cmodel_version\u003e.pt --include onnx_end2end\n\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n// Move \u003cmodel_version\u003e-end2end.onnx file to 'examples/yolov9'\ncp \u003cmodel_version\u003e-end2end.onnx /app/examples/yolov9\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./yolov9 /app/examples/yolov9/\u003cmodel_version\u003e-end2end.onnx /app/data/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [YOLOv9-C, Batchsize = 2, Model size = 640x640]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v9_bus.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v9_zidane.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center; padding: 10px\"\u003e\n    \u003cimg src=\"./results/v9_test.jpeg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n### \u003cdiv align=\"left\"\u003eYOLOV8-Detect\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/ultralytics/ultralytics\n\nultralytics==8.1.24\n\n- Install ultralytics package in python\n```python\n\nfrom ultralytics import YOLO\n\nmodel = YOLO('yolov8s.pt')\nmodel.export(format='onnx', dynamic=True)\n```\n```bash\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/yolov8'\ncp \u003cmodel_version\u003e.onnx /app/examples/yolov8\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./yolov8-detect /app/examples/yolov8/\u003cmodel_version\u003e.onnx /app/data/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [YOLOv8s, Batchsize = 2, Model size = 640x640]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8_bus.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8_zidane.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center; padding: 10px\"\u003e\n    \u003cimg src=\"./results/v8_test.jpeg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n### \u003cdiv align=\"left\"\u003eYOLOV8-Segment\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/ultralytics/ultralytics\n\nultralytics==8.1.24\n\n- Install ultralytics package in python\n```python\n\nfrom ultralytics import YOLO\n\n# Load a model\nmodel = YOLO('yolov8n-seg.pt')\n\n# Export the model\nmodel.export(format='onnx', dynamic=True)\n```\n```bash\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/yolov8'\ncp \u003cmodel_version\u003e.onnx /app/examples/yolov8\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./yolov8-segment /app/examples/yolov8/\u003cmodel_version\u003e.onnx /app/data/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [YOLOv8n, Batchsize = 2, Model size = 640x640]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8seg_bus.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8seg_zidane.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center; padding: 10px\"\u003e\n    \u003cimg src=\"./results/v8seg_test.jpeg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n### \u003cdiv align=\"left\"\u003eYOLOV8-Pose\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/ultralytics/ultralytics\n\nultralytics==8.1.24\n\n- Install ultralytics package in python\n```python\n\nfrom ultralytics import YOLO\n\n# Load a model\nmodel = YOLO('yolov8n-pose.pt')\n\n# Export the model\nmodel.export(format='onnx', dynamic=True)\n```\n```bash\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/yolov8'\ncp \u003cmodel_version\u003e.onnx /app/examples/yolov8\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./yolov8-pose /app/examples/yolov8/\u003cmodel_version\u003e.onnx /app/data/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [YOLOv8n, Batchsize = 2, Model size = 640x640]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8pose_bus.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8pose_zidane.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center; padding: 10px\"\u003e\n    \u003cimg src=\"./results/v8pose_test.jpeg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n\n### \u003cdiv align=\"left\"\u003eYOLOV8-OBB\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/ultralytics/ultralytics\n\nultralytics==8.1.24\n\n- Install ultralytics package in python\n```python\n\nfrom ultralytics import YOLO\n\n# Load a model\nmodel = YOLO('yolov8n-obb.pt')\n\n# Export the model\nmodel.export(format='onnx', dynamic=True)\n\n```\n```bash\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/yolov8'\ncp \u003cmodel_version\u003e.onnx /app/examples/yolov8\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./yolov8-obb /app/examples/yolov8/\u003cmodel_version\u003e.onnx /app/data/obb/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [YOLOv8n, Batchsize = 2, Model size = 640x640]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8obb_aerial1.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8obb_aerial2.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center; padding: 10px\"\u003e\n    \u003cimg src=\"./results/v8obb_aerial3.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n### \u003cdiv align=\"left\"\u003eYOLOV8-Classify\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/ultralytics/ultralytics\n\nultralytics==8.1.24\n\n- Install ultralytics package in python\n```python\n\nfrom ultralytics import YOLO\n\n# Load a model\nmodel = YOLO('yolov8n-cls.pt')\n\n# Export the model\nmodel.export(format='onnx', dynamic=True)\n\n```\n```bash\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/yolov8'\ncp \u003cmodel_version\u003e.onnx /app/examples/yolov8\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./yolov8-classify /app/examples/yolov8/\u003cmodel_version\u003e.onnx /app/data/classify/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [YOLOv8n, Batchsize = 2, Model size = 224x224]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8classify_beagle.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/v8classify_chair.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n\n\n\n### \u003cdiv align=\"left\"\u003eYOLOV5-Face\u003c/div\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModel Conversion\u003c/summary\u003e\n\nurl = https://github.com/deepcam-cn/yolov5-face\n\n\n- Install onnx==1.16.2, tqdm, thop, seaborn, torch==1.9, torchvision==0.10\n\n- Changes for dynamic shape\n```txt\ndiff --git a/export.py b/export.py\nindex 1aa7dae..2502e23 100644\n--- a/export.py\n+++ b/export.py\n@@ -75,14 +75,22 @@ if __name__ == '__main__':\n     print('\\nStarting ONNX export with onnx %s...' % onnx.__version__)\n     f = opt.weights.replace('.pt', '.onnx')  # filename\n     model.fuse()  # only for ONNX\n-    input_names=['input']\n-    output_names=['output']\n-    torch.onnx.export(model, img, f, verbose=False, opset_version=12, \n+    \n+    # Dynamic batching support\n+    input_names = [ \"input\" ]\n+    output_names = [ \"output\" ]\n+    dynamic_axes={'input' : {0 : 'batch_size'}, 'output' : {0 : 'batch_size'}}\n+    dynamic_axes['input'][2] = 'height'\n+    dynamic_axes['input'][3] = 'width'\n+    \n+    torch.onnx.export(model, img, f, verbose=True, opset_version=12, \n         input_names=input_names,\n         output_names=output_names,\n+        # dynamic_axes = dynamic_axes\n         dynamic_axes = {'input': {0: 'batch'},\n                         'output': {0: 'batch'}\n                         } if opt.dynamic else None)\n+    # )\n \n     # Checks\n     onnx_model = onnx.load(f)  # load onnx model\n```\n- Convert pytorch to onnx\n```bash\npython3 export.py --weights ./yolov5s-face.pt --dynamic\n```\n\n```bash\ngit clone https://github.com/PrinceP/tensorrt-cpp-for-onnx\n\n// Move \u003cmodel_version\u003e.onnx file to 'examples/yolov5-face'\ncp \u003cmodel_version\u003e.onnx /app/examples/yolov5-face\n\nmkdir build\ncd build\ncmake ..\nmake -j4\n\n./yolov5-face /app/examples/yolov5-face/\u003cmodel_version\u003e.onnx /app/data/yolov5-face/\n\n// Check the results folder\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eResults\u003c/summary\u003e\n\n**Results  [YOLOv5s-face, Batchsize = 2, Model size = 640x640]**\n\n\u003cdiv style=\"display: flex; justify-content: center;\npadding: 10px\"\u003e\n    \u003cimg src=\"./results/yolov5-face_sample.jpg\" width=\"100%\"/\u003e\n\u003c/div\u003e\n\u003c/details\u003e\n\n### \u003cdiv align=\"left\"\u003eNOTES\u003c/div\u003e\n\u003cdetails\u003e\n\u003csummary\u003eIssues\u003c/summary\u003e\n\n- Dynamic batching is supported. The batchsize and image sizes can be updated in the codebase.\n\n- Dynamic batch issue resolved for yolov10: https://github.com/THU-MIG/yolov10/issues/27\n\n- Dynamic batch not present for Yolo-World\n\n- If size issue happens while building. Increase the workspaceSize\n\n```bash\n    Internal error: plugin node /end2end/EfficientNMS_TRT requires XXX bytes of scratch space, but only XXX is available. Try increasing the workspace size with IBuilderConfig::setMemoryPoolLimit().\n```\n```cpp\n    config-\u003esetMaxWorkspaceSize(1U \u003c\u003c 26) \n    //The current memory is 2^26 bytes\n```\n\u003c/details\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprincep%2Ftensorrt-cpp-for-onnx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprincep%2Ftensorrt-cpp-for-onnx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprincep%2Ftensorrt-cpp-for-onnx/lists"}