{"id":13438277,"url":"https://github.com/DeepVAC/libdeepvac","last_synced_at":"2025-03-19T18:32:37.679Z","repository":{"id":43198706,"uuid":"284916438","full_name":"DeepVAC/libdeepvac","owner":"DeepVAC","description":"Use PyTorch model in C++ project","archived":false,"fork":false,"pushed_at":"2021-07-29T09:59:12.000Z","size":335,"stargazers_count":137,"open_issues_count":4,"forks_count":23,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-12T09:43:41.761Z","etag":null,"topics":["cpp","deepvac","libdeepvac","libtorch","ncnn","pytorch","tensorrt"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DeepVAC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-04T08:03:22.000Z","updated_at":"2025-02-23T13:48:11.000Z","dependencies_parsed_at":"2022-08-30T13:21:15.932Z","dependency_job_id":null,"html_url":"https://github.com/DeepVAC/libdeepvac","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepVAC%2Flibdeepvac","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepVAC%2Flibdeepvac/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepVAC%2Flibdeepvac/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DeepVAC%2Flibdeepvac/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DeepVAC","download_url":"https://codeload.github.com/DeepVAC/libdeepvac/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244483642,"owners_count":20460155,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","deepvac","libdeepvac","libtorch","ncnn","pytorch","tensorrt"],"created_at":"2024-07-31T03:01:04.206Z","updated_at":"2025-03-19T18:32:37.218Z","avatar_url":"https://github.com/DeepVAC.png","language":"C++","readme":"# libdeepvac\nUse PyTorch model in C++ project.  \n\n这个项目定义了如何在C++项目中使用PyTorch训练的模型。\n\n# 简介\n\n在MLab（云上炼丹师）实验室，我们使用[DeepVAC](https://github.com/DeepVAC/deepvac) 来训练获得新模型，使用本项目来部署模型。\n\nlibdeepvac作为一个Linux库，在以下四个方面发挥了价值：\n- 向下封装了推理引擎，目前封装了LibTorch，即将封装TensorRT、NCNN、TNN；\n- 向上提供Deepvac类，方便用户继承并实现其自定义的模型；\n- 在modules目录下，MLab提供了经典网络的C++实现；\n- 在utils目录下，MLab提供了网络中常见helper函数的C++实现。\n\n# libdeepvac实现的模块\n### SOTA网络的C++实现\n|类名 | 网络 | 作用 |\n|-----|------|------|\n|SyszuxFaceRetina| RetinaNet | 人脸检测|\n|SyszuxOcrPse | PSENet | 文字检测 |\n|SyszuxOcrDB  | DB Net | 文字检测 |\n|SyszuxSegEsp | ESPNetV2 | 语义分割 |\n|SyszuxClsMobile | MobileNetV3 | 分类 |\n|SyszuxDetectYolo | YOLOV5 | 目标检测 |\n|SyszuxClsResnet | ResNet50 | 分类 |\n\n### helper函数实现\n|类名/函数名 | 作用 |\n|-----|------|\n|AlignFace|人脸对齐|\n|nms | 检测框的非极大值抑制|\n|PriorBox| 生成目标检测的候选框 |\n\n\n未来我们会持续在modules、utils目录下提供SOTA网络的C++实现。如果用户（你）需要什么网络的C++实现，可在issues里提交申请。\n\n# 编译平台的支持\nlibdeepvac支持在以下平台上进行编译：\n- x86_64 GNU/Linux（或者叫 AMD64 GNU/Linux）\n- aarch64 GNU/Linux（或者叫 ARM64 GNU/Linux）\n- macOS  \n\n未来不太可能扩展到其它平台。\n\n\n# 编译目标的支持\nlibdeepvac支持以下目标平台的编译：\n- x86_64 GNU/Linux（或者叫 AMD64 GNU/Linux）\n- x86_64 GNU/Linux with CUDA（或者叫 AMD64 GNU/Linux with CUDA）\n- aarch64 GNU/Linux（或者叫 ARM64 GNU/Linux）\n- Android\n- iOS\n- Nvidia Jetson Xavier NX（Volta,384 CUDA cores, 48 Tensor cores, 6-core, 8GB）\n- Nvidia Jetson AGX Xavier (Volta, 512 CUDA cores, 6-core, 32GB )\n- Nvidia Jetson TX2 (Pascal, 256 CUDA cores, 2-core/4-core, 8GB)\n- Nvidia Jetson TX2 NX (Pascal, 256 CUDA cores, 2-core/4-core, 4GB)\n\n# 项目依赖\nlibdeepvac的编译依赖C++14编译器、CMake、opencv、LibTorch。  \n最简便、高效的方式就是使用我们提供的[MLab HomePod](https://github.com/DeepVAC/MLab#mlab-homepod)。使用MLab HomePod也是我们推荐的方式。\n\n# 如何编译libdeepvac\nlibdeepvac基于CMake进行构建。\n\n## 编译开关\n如果要开始编译libdeepvac，需要先熟悉如下几个CMake选项的作用：\n|CMake选项|默认值|常用值| 作用|备注|\n|--------|-----|-----|-----|---|\n|BUILD_STATIC|ON|ON/OFF| ON：编译静态libdeepvac\u003cbr\u003eOFF: 编译动态libdeepvac|OFF时，链接OpenCV静态库会带来hidden symbol问题，此时需链接OpenCV动态库|\n|USE_STATIC_LIBTORCH|OFF|ON/OFF|ON: 使用libtorch静态库\u003cbr\u003eOFF: 使用libtorch动态库|MLab HomePod中内置有libtorch动态库|\n|USE_MKL|OFF|ON/OFF| 是否使用Intel MKL作为LAPACK/BLAS实现|OFF的时候，需要使用SYSTEM_LAPACK_LIBRARIES指定另外的LAPACK/BLAS实现，比如openblas、Eigen等|\n|SYSTEM_LAPACK_LIBRARIES|\"\"|\"-lblas -llapack\"| USE_MKL关闭后需要指定的LAPACK/BLAS库|在系统路径下安装有相应的开发环境|\n|USE_CUDA|OFF| ON/OFF| 是否使用CUDA|需要CUDA硬件，且系统中已经安装有CUDA ToolKit的开发时|\n|USE_TENSORRT|OFF|ON/OFF|是否使用TensorRT|需要CUDA硬件，且系统中已经安装有TensorRT的开发时|\n|USE_NUMA|OFF|ON/OFF| 是否链接-lnuma库|NA|\n|USE_LOADER|OFF|ON/OFF| 是否使用图片装载器|需要C++17编译器|\n|GARRULOUS_GEMFIELD|OFF|ON/OFF| 是否打开调试log|NA|\n|BUILD_ALL_EXAMPLES|OFF|ON/OFF|是否编译所有的examples |NA|\n\n\n## 下载依赖\n**如果你使用的是MLab HomePod 2.0 pro（或者以上版本），则忽略此小节**。  \n如果你使用的是自定义环境，那么你至少需要下载opencv库、libtorch库：\n- opencv动态库：自行apt下载；\n- opencv静态库：https://github.com/DeepVAC/libdeepvac/releases/download/1.9.0/opencv4deepvac.tar.gz \n- libtorch动态库：自行从PyTorch官网下载；\n- libtorch静态库：https://github.com/DeepVAC/libdeepvac/releases/download/1.9.0/libtorch.tar.gz \n\n你亦可以在MLab HomePod 2.0 pro上自行从源码编译上述的依赖库。\n\n\n## CMake命令\n以下命令所使用路径均基于MLab HomePod 2.0 pro（你可以根据自身环境自行更改）。\n#### 预备工作\n```bash\n# update to latest libdeepvac\ngemfield@homepod2:/opt/gemfield/libdeepvac$ git pull --rebase\n# create build directory\ngemfield@homepod2:/opt/gemfield/libdeepvac$ mkdir build\ngemfield@homepod2:/opt/gemfield/libdeepvac$ cd build\n```\n\n#### CMake\nlibdeepvac内置了诸多cmake开关以支持不同的软硬件开发栈：\n- 在X86_64 GPU服务器上，使用CUDA，使用libtorch静态库，且用MKL作为BLAS/LAPACK库 (MLab HomePod 2.0 pro支持)：\n```bash\ncmake -DUSE_MKL=ON -DUSE_CUDA=ON -DUSE_STATIC_LIBTORCH=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=\"/opt/gemfield/libtorch;/opt/gemfield/opencv4deepvac/\" -DCMAKE_INSTALL_PREFIX=../install ..\n```\n- 在X86_64 GPU服务器上，使用CUDA，使用libtorch动态库，且用MKL作为BLAS/LAPACK库 (MLab HomePod 2.0 pro支持)：\n```bash\ncmake -DUSE_MKL=ON -DUSE_CUDA=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=\"/opt/gemfield/opencv4deepvac;/opt/conda/lib/python3.8/site-packages/torch/\" -DCMAKE_INSTALL_PREFIX=../install ..\n```\n- 在X86_64 GPU服务器上，使用TensorRT和libtorch静态库，且用MKL作为BLAS/LAPACK库：\n```bash\ncmake -DUSE_MKL=ON -DUSE_CUDA=ON -DUSE_MAGMA=ON -DUSE_STATIC_LIBTORCH=ON -DUSE_TENSORRT=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=\"/opt/gemfield/opencv4deepvac/;/opt/gemfield/libtorch\" -DCMAKE_INSTALL_PREFIX=../install ..\n```\n- 在Nvidia Jetson Xavier NX上，使用TensorRT，且用系统的blas和lapack库：\n```bash\ncmake -DUSE_CUDA=ON -DUSE_NUMA=ON -DUSE_TENSORRT=ON -DSYSTEM_LAPACK_LIBRARIES=\"-lblas -llapack\" -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=\"/opt/gemfield/opencv4deepvac/;/opt/gemfield/libtorch\" -DCMAKE_INSTALL_PREFIX=../install ..\n```\n\n#### 编译\n```bash\ncmake --build . --config Release\nmake install\n```\n\n# 如何使用libdeepvac库\n如何在自己的项目中使用libdeepvac预编译库呢？\n## 1. 添加find_package(Deepvac REQUIRED)\n在自己项目的CMakeLists.txt中，添加\n```cmake\nfind_package(Deepvac REQUIRED)\n```\n当然，基于libdeepvac的项目也必然基于opencv和libtorch，因此，下面两个find_package也是必须的：\n```cmake\nfind_package(Torch REQUIRED)\nfind_package(OpenCV REQUIRED)\n```\n\n## 2. 使用libdeepvac提供的头文件cmake变量\n在自己项目的CMakeLists.txt中，你可以使用如下cmake变量：\n- DEEPVAC_INCLUDE_DIRS：libdeepvac库的头文件目录；\n- DEEPVAC_LIBTORCH_INCLUDE_DIRS：libtorch库的头文件目录；\n- DEEPVAC_TENSORRT_INCLUDE_DIRS：TensorRT库的头文件目录；\n- DEEPVAC_CV_INCLUDE_DIRS：OpenCV库的头文件目录；\n\n## 3. 使用libdeepvac提供的库文件cmake变量\n- DEEPVAC_LIBRARIES：libdeepvac库；\n- DEEPVAC_LIBTORCH_CPU_LIBRARIES：libtorch cpu版库；\n- DEEPVAC_LIBTORCH_CUDA_LIBRARIES：libtorch cuda版库；\n- DEEPVAC_LIBTORCH_DEFAULT_LIBRARIES：libtorch默认版库（编译时用的cpu还是cuda）；\n- DEEPVAC_LIBCUDA_LIBRARIES：Nvidia cuda runtime库；\n- DEEPVAC_TENSORRT_LIBRARIES：Nvidia TensorRT runtime库；\n- DEEPVAC_CV_LIBRARIES：OpenCV库；\n\n使用举例：\n```\n#头文件\ntarget_include_directories(${your_target} \"${DEEPVAC_LIBTORCH_INCLUDE_DIRS};${DEEPVAC_TENSORRT_INCLUDE_DIRS};${CMAKE_CURRENT_SOURCE_DIR}/include\u003e\")\n\n#库文件\ntarget_link_libraries( ${your_target} ${DEEPVAC_LIBRARIES} ${DEEPVAC_LIBTORCH_CUDA_LIBRARIES} ${DEEPVAC_LIBCUDA_LIBRARIES} ${DEEPVAC_CV_LIBRARIES})\n```\n\n# Benchmark\nlibdeepvac会提供不同目标平台及不同推理引擎的benchmark，当前仅支持libtorch推理引擎。\n\n## 1. X86-64 Linux + LibTorch的benchmark步骤\n- 部署[MLab HomePod](https://github.com/DeepVAC/MLab#1-%E9%83%A8%E7%BD%B2);\n- 克隆本项目：\n```bash\n# 如果是MLab HomePod 2.0 标准版\ngit clone https://github.com/DeepVAC/libdeepvac \u0026\u0026 cd libdeepvac\n\n# 如果是MLab HomePod 2.0 pro版\ncd /opt/gemfield/libdeepvac \u0026\u0026 git pull --rebase\n```\n- 编译\n```bash\n#新建编译目录\nmkdir build\ncd build\n#cmake(如果基于LibTorch动态库)\ncmake -DGARRULOUS_GEMFIELD=ON -DUSE_MKL=ON -DUSE_CUDA=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=\"/opt/gemfield/opencv4deepvac/;/opt/conda/lib/python3.8/site-packages/torch/\" -DCMAKE_INSTALL_PREFIX=../install ..\n#cmake(如果基于LibTorch静态库)\ncmake -DGARRULOUS_GEMFIELD=ON -DUSE_MKL=ON -DUSE_CUDA=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=\"/opt/gemfield/opencv4deepvac/;/opt/gemfield/libtorch/\" -DCMAKE_INSTALL_PREFIX=../install ..\n#编译\nmake -j4\n```\n- 运行benchmark\n```bash\n./bin/test_resnet_benchmark cuda:0 \u003cyour_torch_script.pt\u003e \u003ca_imagenet_test.jpg\u003e\n```\n\n## 2. NA\n\n# 演示\n[SYSZUX-FACE](https://github.com/CivilNet/SYSZUX-FACE)基于本项目实现了人脸检测功能。","funding_links":[],"categories":["C++"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDeepVAC%2Flibdeepvac","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FDeepVAC%2Flibdeepvac","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDeepVAC%2Flibdeepvac/lists"}