{"id":26898360,"url":"https://github.com/1125962926/yolo_rknn_acceleration_program","last_synced_at":"2026-05-02T03:32:58.369Z","repository":{"id":284736983,"uuid":"951787790","full_name":"1125962926/YOLO_RKNN_Acceleration_Program","owner":"1125962926","description":"YOLO multi-threaded and hardware-accelerated inference framework based on RKNN","archived":false,"fork":false,"pushed_at":"2025-03-31T00:55:40.000Z","size":11,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-31T01:28:28.827Z","etag":null,"topics":["ffmpeg","gpu","hardware-acceleration","librga","multithread","npu","opencv","rk3588","rkmpp","vpu","yolo"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/1125962926.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-20T08:43:38.000Z","updated_at":"2025-03-31T00:55:43.000Z","dependencies_parsed_at":"2025-03-27T12:44:25.498Z","dependency_job_id":null,"html_url":"https://github.com/1125962926/YOLO_RKNN_Acceleration_Program","commit_stats":null,"previous_names":["1125962926/yolo_multi_thread","1125962926/yolo_rknn_acceleration_program"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1125962926%2FYOLO_RKNN_Acceleration_Program","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1125962926%2FYOLO_RKNN_Acceleration_Program/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1125962926%2FYOLO_RKNN_Acceleration_Program/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/1125962926%2FYOLO_RKNN_Acceleration_Program/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/1125962926","download_url":"https://codeload.github.com/1125962926/YOLO_RKNN_Acceleration_Program/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246591810,"owners_count":20801985,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ffmpeg","gpu","hardware-acceleration","librga","multithread","npu","opencv","rk3588","rkmpp","vpu","yolo"],"created_at":"2025-04-01T05:46:41.061Z","updated_at":"2026-05-02T03:32:58.359Z","avatar_url":"https://github.com/1125962926.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🚀 YOLO RKNN Acceleration Program \n**基于 RK3588 的 YOLO 多线程推理多级硬件加速引擎框架设计** \n\n*YOLO multi-threaded and hardware-accelerated inference framework based on RKNN* \n\n\n## 🏆 性能总结 / Performance Summary\n- ### **141 FPS → 151 FPS** 超越基线  \n  在原项目最高帧数141帧（C++）的基础上，使用 **RKmpp** 硬件解码和 **RGA** 硬件图像前处理，将推理帧数提高至 **151** 帧。\n- ### 关键优化技术 / *Key optimizations*\n  - **RKmpp** 硬件解码 / *RKmpp hardware decoding*\n  - **RGA** 硬件图像预处理 / *RGA hardware image preprocessing*\n\n\n\n## 🛠 技术增强 / Technical Enhancements\n- ### 🎭 多态视频加载器（OpenCV/FFmpeg动态切换）  \n  *Polymorphic video loader (OpenCV/FFmpeg dynamic switching)*\n- ### 🖥️ 命令行参数控制 / *Command-line parameter control*\n- ### 🧠 优化内存管理 / *Optimized memory management*\n\n\n\n## ✨ 新特性支持 / New features\n- ### 图像传输 DMA 优化（计划 2.0 版本加入）\n    ```\n    状态：✅ 已验证理论并实测成功，待合并进本项目\n    ```\n    #### 核心改进：\n    - **技术路径**：将图像数据的传输方式从 CPU 搬运优化为 DMA（直接内存访问）传输。\n\n    - **核心机制**：通过传递文件描述符（fd）而非拷贝数据本身，实现内存零拷贝，显著提升数据传输效率。\n\n    #### 性能收益：\n    - ⚡ **大幅提速**：减少了 CPU 在数据搬运上的参与，直接加快了图像数据的传输速度。\n    - 💾 **显著降低 CPU 占用**：将 CPU 从繁重的数据拷贝任务中解放出来，为系统其他任务释放了宝贵的计算资源。\n\n    #### 指导文章：\n    - 优化方案的设计与代码实现已完成，相关原理和实现细节已在技术文章中深入探讨：[![CSDN](https://img.shields.io/badge/CSDN-Analysis-blue)](https://blog.csdn.net/plmm__/article/details/152009834?spm=1001.2014.3001.5502)\n    \n## 🎯 未实现的优化 / Unimplemented Optimization\n- ### 优化一：模型输入修改\n    - #### 方案一：模型转换阶段修改\n        - 在 `ONNX -\u003e RKNN` 模型转换阶段，将模型的输入设置为 `NV12`，在实际推理的预处理阶段可以跳过 `RGB` 图像的转换，直接使用硬件解码的 `DRM Frame` 传入 `YOLO` 模型，执行推理和后处理，叠加推理结果（画框）。\n        - 这个方案的调整相对可行，但是模型的输入格式非常依赖 `rknn-toolkit2` 的适配程度，大概率需要手动修改 `rknn-toolkit2` 适配程序。本质上也是有 `RGB` 的转换，不过是内嵌到每次模型的推理中，作为一个固定的模块合并到 `NPU` 中执行。这里还需要处理好数据转换的 `stream`，应该也是一个比较麻烦的部分。\n\n    - #### 方案二：重构 YOLO 推理模型的输入格式\n        - 重新训练 `YOLO` 推理模型，将传统的 `RGB` 输入数据修改为 `NV12` 输入数据，重写图像数据解析的部分，完全定制适配 `NV12` 输入图像。\n        - 这个方案的代价有点大，但是优点也很明显。缺点是修改模型需要对 `YOLO` 的模型结构有整体的认识，明确知道图像输入数据的修改，以及图像数据的处理。优点是全链路优化，从解码到显示，中间不存在图像数据的转换，完全适配嵌入式设备和需要解码的场景。\n\n- ### 优化二：图像显示提速\n    - #### 方案一：DRM/KMS 直接合成显示\n        - 使用 libdrm + rockchip_drm.h，将解码 buffer 作为 overlay plane 直接送显。\n        - 可叠加推理结果（画框）：先用 GPU（Mali-G610）渲染 OSD 到另一个 buffer，再用 KMS 合成。\n    \n    - #### 方案二：EGLImage + OpenGL ES 渲染\n        - 将 dma-buf 导入为 EGLImageKHR，绑定到 OpenGL ES 纹理。\n        - 在 GPU 上：\n            - 渲染原始视频帧\n            - 叠加检测框/文字（用 shader 或 Cairo/EGL）\n        - 最终通过 GBM + KMS 或 Wayland/Weston 输出。\n\n## 📚 项目解析 / Project Analysis\n- ### CSDN 博客 / *CSDN Articles*\n    - **Overview**: [![CSDN](https://img.shields.io/badge/CSDN-Overview-blue)](https://blog.csdn.net/plmm__/article/details/146542002)\n    - **Technical Analysis**: [![CSDN](https://img.shields.io/badge/CSDN-Analysis-blue)](https://blog.csdn.net/plmm__/article/details/146556955)\n\n\n\n## 📋 快速开始指南 / Quick Start Guide\n### 1️⃣ 环境准备 / Prerequisites \n- 开发板需要预装 OpenCV，一般出厂系统都有\n\n    *OpenCV pre-installed (usually included in factory systems)*\n\n### 2️⃣ 测试视频 / Test Video\n- 下载 [Baseline](https://github.com/leafqycc/rknn-cpp-Multithreading) Releases 中的测试视频，放项目的根目录\n\n    *Download test video from Baseline Releases and place in project root*\n\n### 3️⃣ (可选) 定频 / (Optional) Frequency Locking\n- 可切换至 root 用户运行 `performance.sh` 定频提高性能和稳定性  \n    *Run as root: `./performance.sh`*\n\n### 4️⃣ 板端编译 / Board-side Compilation\n- 运行 `build.sh`，该脚本会配置并编译 `CMakeLists.txt` \n\n    *Run `build.sh` to configure and compile `CMakeLists.txt`*\n\n- 没有使用 `install` 进行安装，而是直接执行编译后的程序，节约空间 \n\n    *No `install` used, directly execute the compiled program to save space*\n\n### 5️⃣ 执行推理 / Run Inference\n```bash\n./detect.sh\n```\n- 使用 `detect.sh` 进行推理，脚本会根据项目预定的命令行参数进行填写，然后执行编译后的可执行文件\n\n    *Run `detect.sh` to perform inference, the script will fill in the command-line parameters as per the project's predefined parameters, and then execute the compiled executable.*\n\n- 可以根据自己的实际情况修改脚本参数，例如模型路径和视频路径\n\n    *You can modify the script parameters according to your actual situation, such as the model path and video path.*\n    \n\n- 也可以直接执行可执行程序，会打印命令行参数提示\n\n    *You can also directly execute the executable program, which will print the command-line parameter prompts.*\n\n\n\n## 📂 项目结构 / Project Structure\n- `reference` 目录是官方的 demo\n- `clean.sh` 用于清除编译生成的文件\n- ffmpeg 已经移植到项目中\n- `librga` 和 `librknnrt` 已更新至目前的最新版本\n- `performance.sh` 是官方的定频脚本\n\n```bash\n├── 📜 build.sh\n├── 📜 clean.sh\n├── 📜 CMakeLists.txt\n├── 📜 detect.sh\n├── 📂 include/\n│   ├── 🖼️ drm_func.h\n│   ├── 📹 ffmpeg/\n│   ├── ⚙️ parse_config.hpp\n│   ├── 🔍 postprocess.h\n│   ├── ✨ preprocess.h\n│   ├── 📖 reader/\n│   ├── 🖥️ rga/\n│   ├── 🧠 rknn/\n│   ├── 🏊 rknnPool.hpp\n│   ├── 🎯 rkYolo.hpp\n│   ├── 🔗 SharedTypes.hpp\n│   └── 🧵 ThreadPool.hpp\n├── 📂 lib/\n│   ├── 📹 ffmpeg/\n│   ├── 🖥️ librga.so\n│   ├── 🔗 librknn_api.so -\u003e librknnrt.so\n│   └── 🧠 librknnrt.so\n├── 📂 model/\n│   ├── 🏷️ coco_80_labels_list.txt\n│   └── 🖥️ RK3588/\n├── 📜 performance.sh\n├── 📂 reference/\n│   ├── 📹 ffmpeg_mpp_test.cpp\n│   ├── 🖥️ ffmpeg_rga_test.cpp\n│   ├── 🎥 main_video.cc\n│   └── 🖼️ rga_*.cpp\n└── 📂 src/\n    ├── 🎯 main.cpp\n    ├── ⚙️ parse_config.cpp\n    ├── 🔍 postprocess.cpp\n    ├── ✨ preprocess.cpp\n    ├── 📖 reader/\n    └── 🎯 rkYolo.cpp\n```\n\n## 📌 基线 / Baseline \nForked from [leafqycc/rknn-cpp-Multithreading](https://github.com/leafqycc/rknn-cpp-Multithreading)\n\n## 💬 联系方式 / Contact\n### ✉️ 开发者 / Developer​​:\nEmail / QQ: 1125962926@qq.com\n\n欢迎合作优化RKNN加速方案！\n\n*Let's collaborate on optimizing RKNN acceleration!*","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1125962926%2Fyolo_rknn_acceleration_program","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F1125962926%2Fyolo_rknn_acceleration_program","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F1125962926%2Fyolo_rknn_acceleration_program/lists"}