{"id":19855824,"url":"https://github.com/cuixing158/yolo-tensorrt-cpp","last_synced_at":"2025-02-28T21:41:32.229Z","repository":{"id":205260418,"uuid":"298536012","full_name":"cuixing158/yolo-tensorRT-cpp","owner":"cuixing158","description":"部署量化库，适合pc,jetson,int8量化， yolov3/v4/v5","archived":false,"fork":false,"pushed_at":"2023-11-03T08:38:26.000Z","size":625,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-11T13:54:44.090Z","etag":null,"topics":["tensorrt","tensorrt-engine","tensorrt-inference","yolov3","yolov4","yolov5"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cuixing158.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-09-25T10:04:24.000Z","updated_at":"2024-09-22T12:51:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"53ce33dd-5550-4f89-9b54-35d2947994e7","html_url":"https://github.com/cuixing158/yolo-tensorRT-cpp","commit_stats":null,"previous_names":["cuixing158/yolo-tensorrt-cpp"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuixing158%2Fyolo-tensorRT-cpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuixing158%2Fyolo-tensorRT-cpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuixing158%2Fyolo-tensorRT-cpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cuixing158%2Fyolo-tensorRT-cpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cuixing158","download_url":"https://codeload.github.com/cuixing158/yolo-tensorRT-cpp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241245642,"owners_count":19933296,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["tensorrt","tensorrt-engine","tensorrt-inference","yolov3","yolov4","yolov5"],"created_at":"2024-11-12T14:13:48.884Z","updated_at":"2025-02-28T21:41:32.203Z","avatar_url":"https://github.com/cuixing158.png","language":"C++","readme":"**本项目工程属于tensorRT yolov3/v4/v5 C++量化版本！**\n## OverView\n本工程含有2个项目，一个用于dll_detector产生dll或者so库文件，另一个为测试库文件的项目,yolov3/v4需要事先准备cfg,weights文件， [yolov5](https://github.com/ultralytics/yolov5 )需要事先准备yolov5s.yaml和yolov5s.pt文件。C++代码组织非常好，可以学习参考，另外关于tensorRT的量化过程也不错。此库非常适合windows10,ubuntu,嵌入式jetson环境部署。\u003cbr\u003e\n\n## TensorRT 量化流程\n 量化工作原理为：先判断是否有校订table文件存在，有的话直接读取，没有就对data/目录下的图像进行calibrate生成table，先调用函数readCalibrationCache,然后getBatch,最后writeCalibrationCache，getBatch()在校验过程中\n 调用多次，其他函数调用一次。所有模型文件都转换为cfg,weights,解析是使用tensorRT C++ 自定义的API。\n \n \n## 更新记录\n2020.9.27记录：tensorRT量化进度，审阅代码到calibrator流程，是定义Int8EntropyCalibrator 继承tensorRT库下的 public nvinfer1::IInt8EntropyCalibrator，重写calibrator类.明天需要完成自己的球员网球检测器在量化后的表现\u003cbr\u003e\n2020.9.28记录：量化了网球球员检测模型，速度10ms一帧，320×320，速度并未提高？校准表是中间生成？\u003cbr\u003e\n2020.10.9 记录：弄清楚量化接口的调用过程，以便于部署其他模型的推理量化。在PC上测评tensorRT性能结果见[此项目](https://github.com/cuixing158/yolov3-yolov4) \u003cbr\u003e\n2020.10.10记录：项目中使用engine推理图像大小是在cfg文件中定义的width,height进行的，而非实际输入图像大小。暂时终止此项目，改用onnxruntime进行推理，因为onnxruntime已经[集成了tensorRT推理引擎](https://github.com/microsoft/onnxruntime/blob/master/docs/execution_providers/TensorRT-ExecutionProvider.md)或者[onnx-tensorrt](https://github.com/onnx/onnx-tensorrt)\u003cbr\u003e\n或移步到我的[face_jetson_pytorch](https://github.com/cuixing158/jetson_faceTrack_pytorch)\u003cbr\u003e\n2020.11.11记录：量化部分仍旧回到本库C++ TensorRT模式！环境定向为cuda10.2+cudnn7.4.1+vs2019\n\n![](./configs/yolo-trt.png)\n- [x] yolov5s , yolov5m , yolov5l , yolov5x [tutorial](yolov5_tutorial.md)\n- [x] yolov4 , yolov4-tiny\n- [x] yolov3 , yolov3-tiny\n\n## Features\n\n- [x] inequal net width and height\n- [x] batch inference\n- [x] support FP32,FP16,INT8\n- [ ] daynamic input size\n\n## WRAPPER\n\nPrepare the pretrained __.weights__ and __.cfg__ model. \n\n```c++\nDetector detector;\nConfig config;\n\nstd::vector\u003cBatchResult\u003e res;\ndetector.detect(vec_image, res)\n```\n\n\n### windows10\n- cuda环境dependency：tensorRT,cuda,cudnn版本要对应，TensorRT6.0.1.5+cuda10.1+cudnn7.6.4.38 或者 TensorRT 7.1.3.4 +cuda 11.0 + cudnn 8.0 \n或者 TensorRT7.0+cuda10.2+cudnn7.6.4.38 \n\n- 软件环境dependency : opencv4 , vs2015或其他版本\n- build:\n  \n    open MSVC _sln/sln.sln_ file \n    - dll project : the trt yolo detector dll\n    - demo project : test of the dll\n![图片](configs/config.jpg)\u003cbr\u003e\n\n### ubuntu \u0026 L4T (jetson)\n\nThe project generate the __libdetector.so__ lib, and the sample code.\n**_If you want to use the libdetector.so lib in your own project,this [cmake file](https://github.com/enazoe/yolo-tensorrt/blob/master/scripts/CMakeLists.txt) perhaps could help you ._**\n\n\n```bash\ngit clone https://github.com/enazoe/yolo-tensorrt.git\ncd yolo-tensorrt/\nmkdir build\ncd build/\ncmake ..\nmake\n./yolo-trt\n```\n## API\n\n```c++\nstruct Config\n{\n\tstd::string file_model_cfg = \"configs/yolov4.cfg\";\n\n\tstd::string file_model_weights = \"configs/yolov4.weights\";\n\n\tfloat detect_thresh = 0.9;\n\n\tModelType net_type = YOLOV4;\n\n\tPrecision inference_precison = INT8;\n\t\n\tint gpu_id = 0;\n\n\tstd::string calibration_image_list_file_txt = \"configs/calibration_images.txt\";\n\n\tint n_max_batch = 4;\t\n};\n\nclass API Detector\n{\npublic:\n\texplicit Detector();\n\t~Detector();\n\n\tvoid init(const Config \u0026config);\n\n\tvoid detect(const std::vector\u003ccv::Mat\u003e \u0026mat_image,std::vector\u003cBatchResult\u003e \u0026vec_batch_result);\n\nprivate:\n\tDetector(const Detector \u0026);\n\tconst Detector \u0026operator =(const Detector \u0026);\n\tclass Impl;\n\tImpl *_impl;\n};\n```\n\n## 关于量化的一些原理知识\n对于任意一个实数，量化为整数类型，省略bias，计算公式如下：\u003cbr\u003e\nRealWorldValue = StoredInteger ✕ 2^(−FractionLength) \u003cbr\u003e\n在matlab中使用定点计算可以表述上式，例如pi，可以用以下代码量化其值：\u003cbr\u003e\n```matlab\nntBP = numerictype(1,8,4);% 定义一种有符号8位并小数位占4位的符号位对象类型\nx_BP = fi(pi,true,8) % 有符号8位定点数,小数位长度由软件自动推算\npi_cal = double(x_BP.storedInteger)*2^(-x_BP.FractionLength)+x_BP.Bias % 验证量化公式\n\nyBP1 = quantize(x_BP,ntBP) % 指定x_BP为ntBP类型\n```\noutput:\u003cbr\u003e\n```text\nx_BP = \n   3.156250000000000\n\n          DataTypeMode: Fixed-point: binary point scaling\n            Signedness: Signed\n            WordLength: 8\n        FractionLength: 5\npi_cal =\n   3.156250000000000\nyBP1 = \n   3.125000000000000\n\n          DataTypeMode: Fixed-point: binary point scaling\n            Signedness: Signed\n            WordLength: 8\n        FractionLength: 4\n```\n\n\n\n## REFERENCE\n\n- https://github.com/enazoe/yolo-tensorrt\n- [tensorRTX库重点](https://github.com/wang-xinyu/tensorrtx/tree/master/yolov4 )\n- https://github.com/mj8ac/trt-yolo-app_win64\n- https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps\n- [matlab量化背景1](https://www.mathworks.com/help/fixedpoint/ug/data-types-and-scaling-in-digital-hardware.html#bu22l3v-1 )\n- [What Is int8 Quantization and Why Is It Popular for Deep Neural Networks?](https://www.mathworks.com/company/newsletters/articles/what-is-int8-quantization-and-why-is-it-popular-for-deep-neural-networks.html)\n\n\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcuixing158%2Fyolo-tensorrt-cpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcuixing158%2Fyolo-tensorrt-cpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcuixing158%2Fyolo-tensorrt-cpp/lists"}