{"id":13643397,"url":"https://github.com/Wulingtian/yolov5_tensorrt_int8","last_synced_at":"2025-04-21T01:32:40.096Z","repository":{"id":50462040,"uuid":"334611174","full_name":"Wulingtian/yolov5_tensorrt_int8","owner":"Wulingtian","description":"TensorRT int8 量化部署 yolov5s 模型，实测3.3ms一帧！","archived":false,"fork":false,"pushed_at":"2021-04-23T11:45:30.000Z","size":6981,"stargazers_count":166,"open_issues_count":9,"forks_count":26,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-11-09T15:42:34.353Z","etag":null,"topics":["int8","tensorrt","yolov5"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Wulingtian.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-31T08:48:43.000Z","updated_at":"2024-06-28T06:16:18.000Z","dependencies_parsed_at":"2022-08-12T21:21:37.430Z","dependency_job_id":null,"html_url":"https://github.com/Wulingtian/yolov5_tensorrt_int8","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Wulingtian%2Fyolov5_tensorrt_int8","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Wulingtian%2Fyolov5_tensorrt_int8/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Wulingtian%2Fyolov5_tensorrt_int8/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Wulingtian%2Fyolov5_tensorrt_int8/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Wulingtian","download_url":"https://codeload.github.com/Wulingtian/yolov5_tensorrt_int8/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249982631,"owners_count":21355735,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["int8","tensorrt","yolov5"],"created_at":"2024-08-02T01:01:46.857Z","updated_at":"2025-04-21T01:32:35.153Z","avatar_url":"https://github.com/Wulingtian.png","language":"C++","funding_links":[],"categories":["Lighter and Deployment Frameworks","Applications"],"sub_categories":[],"readme":"# 环境配置\n\n    ubuntu：18.04\n\n    cuda：11.0\n\n    cudnn：8.0\n\n    tensorrt：7.2.16\n\n    OpenCV：3.4.2\n\n    cuda，cudnn，tensorrt和OpenCV安装包（编译好了，也可以自己从官网下载编译）可以从链接: https://pan.baidu.com/s/1dpMRyzLivnBAca2c_DIgGw 密码: 0rct\n\n    cuda安装\n\n    如果系统有安装驱动，运行如下命令卸载\n\n    sudo apt-get purge nvidia*\n\n    禁用nouveau，运行如下命令\n\n    sudo vim /etc/modprobe.d/blacklist.conf\n\n    在末尾添加\n\n    blacklist nouveau\n\n    然后执行\n\n    sudo update-initramfs -u\n\n    chmod +x cuda_11.0.2_450.51.05_linux.run\n\n    sudo ./cuda_11.0.2_450.51.05_linux.run\n\n    是否接受协议: accept\n\n    然后选择Install\n\n    最后回车\n\n    vim ~/.bashrc 添加如下内容：\n\n    export PATH=/usr/local/cuda-11.0/bin:$PATH\n\n    export LD_LIBRARY_PATH=/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH\n\n    source .bashrc 激活环境\n\n    cudnn 安装\n\n    tar -xzvf cudnn-11.0-linux-x64-v8.0.4.30.tgz\n\n    cd cuda/include\n\n    sudo cp *.h /usr/local/cuda-11.0/include\n\n    cd cuda/lib64\n\n    sudo cp libcudnn* /usr/local/cuda-11.0/lib64\n\n    tensorrt及OpenCV安装\n\n    定位到用户根目录\n\n    tar -xzvf TensorRT-7.2.1.6.Ubuntu-18.04.x86_64-gnu.cuda-11.0.cudnn8.0.tar.gz\n\n    cd TensorRT-7.2.1.6/python，该目录有4个python版本的tensorrt安装包\n\n    sudo pip3 install tensorrt-7.2.1.6-cp37-none-linux_x86_64.whl（根据自己的python版本安装）\n\n    pip install pycuda 安装python版本的cuda\n\n    定位到用户根目录\n\n    tar -xzvf opencv-3.4.2.zip 以备推理调用\n    \n# yolov5s模型转换onnx\n\n    pip install onnx\n\n    pip install onnx-simplifier\n\n    git clone https://github.com/ultralytics/yolov5.git\n\n    cd yolov5/models\n\n    vim common.py\n\n    把BottleneckCSP类下的激活函数替换为relu，tensorrt对leakyRelu int8量化不稳定（这是一个深坑，大家记得避开）即修改为self.act = nn.ReLU(inplace=True)\n\n    训练得到模型后\n\n    cd yolov5\n\n    python models/export.py --weights 训练得到的模型权重路径 --img-size 训练图片输入尺寸\n\n    python3 -m onnxsim onnx模型名称 yolov5s-simple.onnx 得到最终简化后的onnx模型\n    \n# onnx模型转换为 int8 tensorrt引擎\n\n    git clone https://github.com/Wulingtian/yolov5_tensorrt_int8_tools.git（求star）\n\n    cd yolov5_tensorrt_int8_tools\n\n    vim convert_trt_quant.py 修改如下参数\n\n    BATCH_SIZE 模型量化一次输入多少张图片\n\n    BATCH 模型量化次数\n\n    height width 输入图片宽和高\n\n    CALIB_IMG_DIR 训练图片路径，用于量化\n\n    onnx_model_path onnx模型路径\n\n    python convert_trt_quant.py 量化后的模型存到models_save目录下\n    \n# tensorrt模型推理\n\n    git clone https://github.com/Wulingtian/yolov5_tensorrt_int8.git（求star）\n\n    cd yolov5_tensorrt_int8\n\n    vim CMakeLists.txt\n\n    修改USER_DIR参数为自己的用户根目录\n\n    vim http://yolov5s_infer.cc 修改如下参数\n\n    output_name1 output_name2 output_name3 yolov5模型有3个输出\n\n    我们可以通过netron查看模型输出名\n\n    pip install netron 安装netron\n\n    vim netron_yolov5s.py 把如下内容粘贴\n\n        import netron\n\n        netron.start('此处填充简化后的onnx模型路径', port=3344)\n\n    python netron_yolov5s.py 即可查看 模型输出名\n\n    trt_model_path 量化的的tensorrt推理引擎（models_save目录下trt后缀的文件）\n\n    test_img 测试图片路径\n\n    INPUT_W INPUT_H 输入图片宽高\n\n    NUM_CLASS 训练的模型有多少类\n\n    NMS_THRESH nms阈值\n\n    CONF_THRESH 置信度\n\n    参数配置完毕\n\n    mkdir build\n\n    cd build\n\n    cmake ..\n\n    make\n\n    ./YoloV5sEngine 输出平均推理时间，以及保存预测图片到当前目录下，至此，部署完成！\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FWulingtian%2Fyolov5_tensorrt_int8","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FWulingtian%2Fyolov5_tensorrt_int8","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FWulingtian%2Fyolov5_tensorrt_int8/lists"}