{"id":16270248,"url":"https://github.com/allentdan/libtorchsegmentation","last_synced_at":"2025-04-05T02:09:28.895Z","repository":{"id":40611099,"uuid":"336812841","full_name":"AllentDan/LibtorchSegmentation","owner":"AllentDan","description":"A c++ trainable semantic segmentation library based on libtorch (pytorch c++). Backbone: VGG, ResNet, ResNext. Architecture: FPN, U-Net, PAN, LinkNet, PSPNet, DeepLab-V3, DeepLab-V3+ by now.","archived":false,"fork":false,"pushed_at":"2023-12-07T01:43:53.000Z","size":92666,"stargazers_count":444,"open_issues_count":6,"forks_count":91,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-03-29T01:09:26.002Z","etag":null,"topics":["cpp","deeplabv3","deeplabv3plus","fpn","image-segmentation","imagenet","libtorch","libtorch-segment","models","neural-network","pretrained-backbones","pretrained-weights","pspnet","pytorch","pytorch-cpp","pytorch-cpp-frontend","resnet","resnext","semantic-segmentation","unet"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AllentDan.png","metadata":{"files":{"readme":"README-Chinese.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-07T14:56:00.000Z","updated_at":"2025-03-20T12:23:05.000Z","dependencies_parsed_at":"2025-02-24T00:01:00.071Z","dependency_job_id":"6d04cf80-2f3f-4edf-9590-fd37ea543b6b","html_url":"https://github.com/AllentDan/LibtorchSegmentation","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AllentDan%2FLibtorchSegmentation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AllentDan%2FLibtorchSegmentation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AllentDan%2FLibtorchSegmentation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AllentDan%2FLibtorchSegmentation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AllentDan","download_url":"https://codeload.github.com/AllentDan/LibtorchSegmentation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247276188,"owners_count":20912288,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpp","deeplabv3","deeplabv3plus","fpn","image-segmentation","imagenet","libtorch","libtorch-segment","models","neural-network","pretrained-backbones","pretrained-weights","pspnet","pytorch","pytorch-cpp","pytorch-cpp-frontend","resnet","resnext","semantic-segmentation","unet"],"created_at":"2024-10-10T18:09:58.146Z","updated_at":"2025-04-05T02:09:28.878Z","avatar_url":"https://github.com/AllentDan.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"[English](https://github.com/AllentDan/SegmentationCpp) | 中文\n\n\u003cdiv align=\"center\"\u003e\n\n![logo](https://raw.githubusercontent.com/AllentDan/ImageBase/main/OpenSource/LibtorchSegment.png)  \n**基于[LibTorch](https://pytorch.org/)的C++开源图像分割神经网络库.**  \n\n\u003c/div\u003e\n\n**⭐如果有用请给我一个star⭐**\n\n这个库具有以下优点:\n\n - 高级的API (只需一行代码就可创建网络)\n - 7 种模型架构可用于单类或者多类的分割任务 (包括Unet)\n - 15 种编码器网络\n - 所有的编码器都有预训练权重，可以更快更好地收敛\n - 相比于python下的GPU前向推理速度具有30%或以上的提速, cpu下保持速度一致. (Unet测试于RTX 2070S).\n \n### [📚 Libtorch教程 📚](https://github.com/AllentDan/LibtorchTutorials)\n\n如果你想对该开源项目有更多更详细的了解，请前往本人另一个开源项目：[Libtorch教程](https://github.com/AllentDan/LibtorchTutorials) .\n\n### 📋 目录\n 1. [快速开始](#start)\n 2. [例子](#examples)\n 3. [训练自己的数据](#trainingOwn)\n 4. [模型](#models)\n    1. [架构](#architectures)\n    2. [编码器](#encoders)\n 5. [安装](#installation)\n 6. [ToDo](#todo)\n 7. [感谢](#thanks)\n 8. [引用](#citing)\n 9. [证书](#license)\n 10. [相关项目](#related_repos)\n\n### ⏳ 快速开始 \u003ca name=\"start\"\u003e\u003c/a\u003e\n\n#### 1. 用 Libtorch Segment 创建你的第一个分割网络\n\n[这](https://github.com/AllentDan/LibtorchSegmentation/releases/download/weights/segmentor.pt)是一个resnet34的torchscript模型，可以作为骨干网络权重。分割模型是 LibTorch 的 torch::nn::Module的派生类, 可以很容易生成:\n\n```cpp\n#include \"Segmentor.h\"\nauto model = UNet(1, /*num of classes*/\n                  \"resnet34\", /*encoder name, could be resnet50 or others*/\n                  \"path to resnet34.pt\"/*weight path pretrained on ImageNet, it is produced by torchscript*/\n                  );\n```\n - 见 [表](#architectures) 查看所有支持的模型架构\n - 见 [表](#encoders) 查看所有的编码器网络和相应的预训练权重\n\n#### 2. 生成自己的预训练权重\n\n所有编码器均具有预训练的权重。加载预训练权重，以相同的方式训练数据，可能会获得更好的结果（更高的指标得分和更快的收敛速度）。还可以在冻结主干的同时仅训练解码器和分割头。\n\n```python\nimport torch\nfrom torchvision import models\n\n# resnet50 for example\nmodel = models.resnet50(pretrained=True)\nmodel.eval()\nvar=torch.ones((1,3,224,224))\ntraced_script_module = torch.jit.trace(model, var)\ntraced_script_module.save(\"resnet50.pt\")\n```\n\n恭喜你！ 大功告成！ 现在，您可以使用自己喜欢的主干和分割框架来训练模型了。\n\n### 💡 例子 \u003ca name=\"examples\"\u003e\u003c/a\u003e\n - 使用来自PASCAL VOC数据集的图像进行人体分割数据训练模型. \"voc_person_seg\" 目录包含32个json标签及其相应的jpeg图像用于训练，还有8个json标签以及相应的图像用于验证。\n```cpp\nSegmentor\u003cFPN\u003e segmentor;\nsegmentor.Initialize(0/*gpu id, -1 for cpu*/,\n                    512/*resize width*/,\n                    512/*resize height*/,\n                    {\"background\",\"person\"}/*class name dict, background included*/,\n                    \"resnet34\"/*backbone name*/,\n                    \"your path to resnet34.pt\");\nsegmentor.Train(0.0003/*initial leaning rate*/,\n                300/*training epochs*/,\n                4/*batch size*/,\n                \"your path to voc_person_seg\",\n                \".jpg\"/*image type*/,\n                \"your path to save segmentor.pt\");\n```\n\n- 预测测试。项目中提供了以ResNet34为骨干网络的FPN网络，训练了一些周期得到segmentor.pt文件[在这](https://github.com/AllentDan/LibtorchSegmentation/releases/download/weights/segmentor.pt)。 您可以直接测试分割结果:\n```cpp\ncv::Mat image = cv::imread(\"your path to voc_person_seg\\\\val\\\\2007_004000.jpg\");\nSegmentor\u003cFPN\u003e segmentor;\nsegmentor.Initialize(0,512,512,{\"background\",\"person\"},\n                      \"resnet34\",\"your path to resnet34.pt\");\nsegmentor.LoadWeight(\"segmentor.pt\"/*the saved .pt path*/);\nsegmentor.Predict(image,\"person\"/*class name for showing*/);\n```\n预测结果显示如下:\n\n![](https://raw.githubusercontent.com/AllentDan/SegmentationCpp/main/prediction.jpg)\n\n### 🧑‍🚀 训练自己的数据 \u003ca name=\"trainingOwn\"\u003e\u003c/a\u003e\n- 创建自己的数据集. 使用\"pip install\"安装[labelme](https://github.com/wkentaro/labelme)并标注你的图像. 将输出的json文件和图像分成以下文件夹：\n```\nDataset\n├── train\n│   ├── xxx.json\n│   ├── xxx.jpg\n│   └......\n├── val\n│   ├── xxxx.json\n│   ├── xxxx.jpg\n│   └......\n```\n- 训练或测试。就像“ voc_person_seg”的示例一样，用自己的数据集路径替换“ voc_person_seg”。\n- 记得使用[训练技巧](https://github.com/AllentDan/LibtorchSegmentation/blob/main/docs/training%20tricks.md)以提高模型的训练效果。\n\n\n### 📦 Models \u003ca name=\"models\"\u003e\u003c/a\u003e\n\n#### Architectures \u003ca name=\"architectures\"\u003e\u003c/a\u003e\n - [x] Unet [[paper](https://arxiv.org/abs/1505.04597)]\n - [x] FPN [[paper](http://presentations.cocodataset.org/COCO17-Stuff-FAIR.pdf)]\n - [x] PAN [[paper](https://arxiv.org/abs/1805.10180)]\n - [x] PSPNet [[paper](https://arxiv.org/abs/1612.01105)]\n - [x] LinkNet [[paper](https://arxiv.org/abs/1707.03718)]\n - [x] DeepLabV3 [[paper](https://arxiv.org/abs/1706.05587)]\n - [x] DeepLabV3+ [[paper](https://arxiv.org/abs/1802.02611)]\n\n#### Encoders \u003ca name=\"encoders\"\u003e\u003c/a\u003e\n- [x] ResNet\n- [x] ResNext\n- [x] VGG\n\n以下是该项目中受支持的编码器的列表。除resnest外，所有编码器权重都可以通过torchvision生成。选择适当的编码器，然后单击以展开表格，然后选择特定的编码器及其预训练的权重。\n\n\u003cdetails\u003e\n\u003csummary style=\"margin-left: 25px;\"\u003eResNet\u003c/summary\u003e\n\u003cdiv style=\"margin-left: 25px;\"\u003e\n\n|Encoder                         |Weights                         |Params, M                       |\n|--------------------------------|:------------------------------:|:------------------------------:|\n|resnet18                        |imagenet                        |11M                             |\n|resnet34                        |imagenet                        |21M                             |\n|resnet50                        |imagenet                        |23M                             |\n|resnet101                       |imagenet                        |42M                             |\n|resnet152                       |imagenet                        |58M                             |\n\n\u003c/div\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary style=\"margin-left: 25px;\"\u003eResNeXt\u003c/summary\u003e\n\u003cdiv style=\"margin-left: 25px;\"\u003e\n\n|Encoder                         |Weights                         |Params, M                       |\n|--------------------------------|:------------------------------:|:------------------------------:|\n|resnext50_32x4d                 |imagenet                        |22M                             |\n|resnext101_32x8d                |imagenet                        |86M                             |\n\n\u003c/div\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary style=\"margin-left: 25px;\"\u003eResNeSt\u003c/summary\u003e\n\u003cdiv style=\"margin-left: 25px;\"\u003e\n\n|Encoder                         |Weights                         |Params, M                       |\n|--------------------------------|:------------------------------:|:------------------------------:|\n|timm-resnest14d                 |imagenet                        |8M                              |\n|timm-resnest26d                 |imagenet                        |15M                             |\n|timm-resnest50d                 |imagenet                        |25M                             |\n|timm-resnest101e                |imagenet                        |46M                             |\n|timm-resnest200e                |imagenet                        |68M                             |\n|timm-resnest269e                |imagenet                        |108M                            |\n|timm-resnest50d_4s2x40d         |imagenet                        |28M                             |\n|timm-resnest50d_1s4x24d         |imagenet                        |23M                             |\n\n\u003c/div\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary style=\"margin-left: 25px;\"\u003eSE-Net\u003c/summary\u003e\n\u003cdiv style=\"margin-left: 25px;\"\u003e\n\n|Encoder                         |Weights                         |Params, M                       |\n|--------------------------------|:------------------------------:|:------------------------------:|\n|senet154                        |imagenet                        |113M                            |\n|se_resnet50                     |imagenet                        |26M                             |\n|se_resnet101                    |imagenet                        |47M                             |\n|se_resnet152                    |imagenet                        |64M                             |\n|se_resnext50_32x4d              |imagenet                        |25M                             |\n|se_resnext101_32x4d             |imagenet                        |46M                             |\n\n\u003c/div\u003e\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary style=\"margin-left: 25px;\"\u003eVGG\u003c/summary\u003e\n\u003cdiv style=\"margin-left: 25px;\"\u003e\n\n|Encoder                         |Weights                         |Params, M                       |\n|--------------------------------|:------------------------------:|:------------------------------:|\n|vgg11                           |imagenet                        |9M                              |\n|vgg11_bn                        |imagenet                        |9M                              |\n|vgg13                           |imagenet                        |9M                              |\n|vgg13_bn                        |imagenet                        |9M                              |\n|vgg16                           |imagenet                        |14M                             |\n|vgg16_bn                        |imagenet                        |14M                             |\n|vgg19                           |imagenet                        |20M                             |\n|vgg19_bn                        |imagenet                        |20M                             |\n\n\u003c/div\u003e\n\u003c/details\u003e\n\n### 🛠 安装 \u003ca name=\"installation\"\u003e\u003c/a\u003e\n**依赖库:**\n\n- [Opencv 3+](https://opencv.org/releases/)\n- [Libtorch 1.7+](https://pytorch.org/)\n\n**Windows:**\n\n配置libtorch 开发环境. [Visual studio](https://allentdan.github.io/2020/12/16/pytorch%E9%83%A8%E7%BD%B2torchscript%E7%AF%87) 和 [Qt Creator](https://allentdan.github.io/2021/01/21/QT%20Creator%20+%20Opencv4.x%20+%20Libtorch1.7%E9%85%8D%E7%BD%AE/#more)已经通过libtorch1.7x release的验证. \n\n**Linux \u0026\u0026 MacOS:**\n\n安装libtorch和opencv。 \n对于libtorch， 按照官方[教程](https://pytorch.org/tutorials/advanced/cpp_export.html)安装。\n对于opencv， 按照官方安装[步骤](https://github.com/opencv/opencv)。\n\n如果你都配置好了他们，恭喜!!! 下载一个resnet34的预训练权重，[点击下载](https://github.com/AllentDan/LibtorchSegmentation/releases/download/weights/resnet34.pt)和一个示例.pt文件，[点击下载](https://github.com/AllentDan/LibtorchSegmentation/releases/download/weights/segmentor.pt)，放入weights文件夹。 \n\n更改src/main.cpp中的图片路径预训练权重和加载的segmentor权重路径。随后，build路径在终端输入:\n```bash\nexport Torch_DIR='/path/to/libtorch'\ncd build\ncmake ..\nmake\n./LibtorchSegmentation\n```\n\n### ⏳ ToDo \u003ca name=\"todo\"\u003e\u003c/a\u003e\n- [ ] 更多的骨干网络和分割框架\n  - [ ] UNet++ [[paper](https://arxiv.org/pdf/1807.10165.pdf)]\n  - [ ] ResNest\n  - [ ] Se-Net\n  - [ ] ...\n- [x] 数据增强\n  - [x] 随机水平翻转\n  - [x] 随机垂直翻转\n  - [x] 随机缩放和旋转\n  - [ ] ...\n- [x] 训练技巧\n  - [x] 联合损失：dice和交叉熵\n  - [x] 冻结骨干网络\n  - [x] 学习率衰减策略\n  - [ ] ...\n\n\n### 🤝 感谢 \u003ca name=\"thanks\"\u003e\u003c/a\u003e\n以下是目前给予帮助的项目.\n- [official pytorch](https://github.com/pytorch/pytorch)\n- [qubvel SMP](https://github.com/qubvel/segmentation_models.pytorch)\n- [wkentaro labelme](https://github.com/wkentaro/labelme)\n- [nlohmann json](https://github.com/nlohmann/json)\n\n### 📝 引用\n```\n@misc{Chunyu:2021,\n  Author = {Chunyu Dong},\n  Title = {Libtorch Segment},\n  Year = {2021},\n  Publisher = {GitHub},\n  Journal = {GitHub repository},\n  Howpublished = {\\url{https://github.com/AllentDan/SegmentationCpp}}\n}\n```\n\n### 🛡️ 证书 \u003ca name=\"license\"\u003e\u003c/a\u003e\n该项目以 [MIT License](https://github.com/qubvel/segmentation_models.pytorch/blob/master/LICENSE)开源，\n\n## 相关项目 \u003ca name=\"related_repos\"\u003e\u003c/a\u003e\n基于libtorch，我释放了如下开源项目:\n- [LibtorchTutorials](https://github.com/AllentDan/LibtorchTutorials)\n- [LibtorchSegmentation](https://github.com/AllentDan/LibtorchSegmentation)\n- [LibtorchDetection](https://github.com/AllentDan/LibtorchDetection)\n  \n别忘了点赞哟\n![stargazers over time](https://starchart.cc/AllentDan/SegmentationCpp.svg)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallentdan%2Flibtorchsegmentation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallentdan%2Flibtorchsegmentation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallentdan%2Flibtorchsegmentation/lists"}