{"id":13642925,"url":"https://github.com/DataXujing/YOLOv8","last_synced_at":"2025-04-20T21:32:08.409Z","repository":{"id":112306502,"uuid":"587254349","full_name":"DataXujing/YOLOv8","owner":"DataXujing","description":":fire: Official YOLOv8模型训练和部署","archived":false,"fork":false,"pushed_at":"2023-02-02T08:08:01.000Z","size":6813,"stargazers_count":568,"open_issues_count":7,"forks_count":77,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-08-02T01:17:41.946Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataXujing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-01-10T10:20:21.000Z","updated_at":"2024-08-01T01:07:11.000Z","dependencies_parsed_at":"2023-05-03T01:45:22.800Z","dependency_job_id":null,"html_url":"https://github.com/DataXujing/YOLOv8","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataXujing%2FYOLOv8","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataXujing%2FYOLOv8/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataXujing%2FYOLOv8/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataXujing%2FYOLOv8/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataXujing","download_url":"https://codeload.github.com/DataXujing/YOLOv8/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223839147,"owners_count":17211880,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:01:38.146Z","updated_at":"2025-04-20T21:32:08.400Z","avatar_url":"https://github.com/DataXujing.png","language":"Python","funding_links":[],"categories":["Other Versions of YOLO"],"sub_categories":[],"readme":"##  Official YOLOv8 训练自己的数据集并基于NVIDIA TensorRT和华为昇腾端到端模型加速以及安卓手机端部署\n\n\n\n说明： 本项目支持YOLOv8的对应的package的版本是：[ultralytics-8.0.0](https://pypi.org/project/ultralytics/8.0.0/)\n\n### 1.YOLO的一些发展历史\n\n+ **YOLOv1：2015年Joseph Redmon和** **Ali Farhadi等** **人（华盛顿大学）**\n\n+ **YOLOv2：2016年Joseph Redmon\\**和\\**\\**Ali Farhadi\\**等人\\**（华盛顿大学）\\****\n\n+ [**YOLOv3**](http://mp.weixin.qq.com/s?__biz=MzUxNjcxMjQxNg==\u0026mid=2247484179\u0026idx=1\u0026sn=c127ae5aac72f52ca7bb39d78512a190\u0026chksm=f9a2719cced5f88a7d92ef5dbb1c010f957d539a3a6acafe85f1e4fa888a39f252ddb8154175\u0026scene=21#wechat_redirect)**：2018年Joseph Redmon\\**和\\**\\**Ali Farhadi\\**等人\\**（华盛顿大学）\\****\n\n+ [**YOLOv4**](http://mp.weixin.qq.com/s?__biz=MzUxNjcxMjQxNg==\u0026mid=2247498390\u0026idx=1\u0026sn=62ec5122def0ceb967761d628799a43b\u0026chksm=f9a18819ced6010f325c7d9af1e96a110ab64fbb96c2a085d2073e799c16704ab86a0d10547a\u0026scene=21#wechat_redirect)**：2020年Alexey Bochkovskiy和Chien-Yao Wang等人**\n\n+ [**YOLOv5**](http://mp.weixin.qq.com/s?__biz=MzUxNjcxMjQxNg==\u0026mid=2247500275\u0026idx=2\u0026sn=a862a79afa87b5ce85fff8a6da6ab34a\u0026chksm=f9a1b37cced63a6abf54e8a778189278bb9c14d2b3fe8f9d3d7ac403906bbf46ba21c179cc91\u0026scene=21#wechat_redirect)**：2020年Ultralytics公司**\n\n+ **YOLOv6：2022年美团公司**\n\n+ [**YOLOv7**](http://mp.weixin.qq.com/s?__biz=MzUxNjcxMjQxNg==\u0026mid=2247550919\u0026idx=2\u0026sn=884a1f6f2c969d67a4c532511b8240a9\u0026chksm=f9a17548ced6fc5e44e8d7db791181f90e2d8f024c14c7765726d0face59e6c8797caf029f1a\u0026scene=21#wechat_redirect)**：2022年Alexey Bochkovskiy\\**和Chien-Yao Wang\\**等人**\n\n+ **YOLOv8：2023年Ultralytics公司**\n\n上述简单罗列了 **YOLOv数字系列** 的发布时间和作者/单位机构，因为YOLO系列生态太猛了，比如还有知名的PP-YOLO系列、YOLOX等等工作。\n\n### 2. YOLOv8的相关资源\n\n+ YOLOv8 Github: https://github.com/ultralytics/ultralytics\n\n+ ~~YOLOv8的权重：https://github.com/ultralytics/assets/releases~~\n+ YOLOv8文档： https://v8docs.ultralytics.com/\n+ ~~YOLOv8 Python package源码库：https://test.pypi.org/simple/ultralytics/~~\n\n\n\n### 3.YOLOv5 Vs YOLOv8\n\n+ **YOLOv5**\n\n![](docs/yolov5.png)\n\n\n\n1. **Backbone**：CSPDarkNet结构，主要结构思想的体现在C3模块，这里也是梯度分流的主要思想所在的地方；\n2. **PAN-FPN**：双流的FPN，必须香，也必须快，但是量化还是有些需要图优化才可以达到最优的性能，比如cat前后的scale优化等等，这里除了上采样、CBS卷积模块，最为主要的还有C3模块（记住这个C3模块哦）；\n3. **Head**：Coupled Head+Anchor-base，毫无疑问，YOLOv3、YOLOv4、YOLOv5、YOLOv7都是Anchor-Base的，后面会变吗？\n4. **Loss**：分类用BEC Loss，回归用CIoU Loss。\n\n+ **YOLOv8**\n\n![](docs/yolov8.png)\n\n具体改进如下：\n\n1. **Backbone**：使用的依旧是CSP的思想，不过YOLOv5中的C3模块被替换成了C2f模块，实现了进一步的轻量化，同时YOLOv8依旧使用了YOLOv5等架构中使用的SPPF模块；\n2. **PAN-FPN**：毫无疑问YOLOv8依旧使用了PAN的思想，不过通过对比YOLOv5与YOLOv8的结构图可以看到，YOLOv8将YOLOv5中PAN-FPN上采样阶段中的卷积结构删除了，同时也将C3模块替换为了C2f模块；\n3. **Decoupled-Head**：是不是嗅到了不一样的味道？是的，YOLOv8走向了Decoupled-Head；\n4. **Anchor-Free**：YOLOv8抛弃了以往的Anchor-Base，使用了**Anchor-Free**的思想；\n5. **损失函数**：YOLOv8使用VFL Loss作为分类损失，使用DFL Loss+CIOU Loss作为分类损失；\n6. **样本匹配**：YOLOv8抛弃了以往的IOU匹配或者单边比例的分配方式，而是使用了Task-Aligned Assigner匹配方式。\n\n+ **SPP Vs SPPF:**\n\n![](docs/sppvs.png)\n\n![](docs/SPP.png)\n\n![](docs/SPPF.png)\n\n+ **C3 Vs C2f:**\n\n![](docs/c3.png)\n\n\n\n针对C3模块，其主要是借助CSPNet提取分流的思想，同时结合残差结构的思想，设计了所谓的C3 Block，这里的CSP主分支梯度模块为BottleNeck模块，也就是所谓的残差模块。同时堆叠的个数由参数n来进行控制，也就是说不同规模的模型，n的值是有变化的。\n\n其实这里的梯度流主分支，可以是任何之前你学习过的模块，比如，美团提出的YOLOv6中就是用来重参模块RepVGGBlock来替换BottleNeck Block来作为主要的梯度流分支，而百度提出的PP-YOLOE则是使用了RepResNet-Block来替换BottleNeck Block来作为主要的梯度流分支。而YOLOv7则是使用了ELAN Block来替换BottleNeck Block来作为主要的梯度流分支。\n\nC3模块的Pytorch的实现如下：\n\n```python\nclass C3(nn.Module):\n    # CSP Bottleneck with 3 convolutions\n    def __init__(self, c1, c2, n=1, shortcut=True, g=1, e=0.5):  # ch_in, ch_out, number, shortcut, groups, expansion\n        super().__init__()\n        c_ = int(c2 * e)  # hidden channels\n        self.cv1 = Conv(c1, c_, 1, 1)\n        self.cv2 = Conv(c1, c_, 1, 1)\n        self.cv3 = Conv(2 * c_, c2, 1)  # optional act=FReLU(c2)\n        self.m = nn.Sequential(*(Bottleneck(c_, c_, shortcut, g, e=1.0) for _ in range(n)))\n\n    def forward(self, x):\n        return self.cv3(torch.cat((self.m(self.cv1(x)), self.cv2(x)), 1))\n```\n\n\n\n![](docs/c2f.png)\n\n\n\nC2f模块就是参考了C3模块以及ELAN的思想进行的设计，让YOLOv8可以在保证轻量化的同时获得更加丰富的梯度流信息。\n\n![](docs/yolov7.png)\n\nC2f模块对应的Pytorch实现如下：\n\n```python\nclass C2f(nn.Module):\n    # CSP Bottleneck with 2 convolutions\n    def __init__(self, c1, c2, n=1, shortcut=False, g=1, e=0.5):  # ch_in, ch_out, number, shortcut, groups, expansion\n        super().__init__()\n        self.c = int(c2 * e)  # hidden channels\n        self.cv1 = Conv(c1, 2 * self.c, 1, 1)\n        self.cv2 = Conv((2 + n) * self.c, c2, 1)  # optional act=FReLU(c2)\n        self.m = nn.ModuleList(Bottleneck(self.c, self.c, shortcut, g, k=((3, 3), (3, 3)), e=1.0) for _ in range(n))\n\n    def forward(self, x):\n        y = list(self.cv1(x).split((self.c, self.c), 1))\n        y.extend(m(y[-1]) for m in self.m)\n        return self.cv2(torch.cat(y, 1))\n```\n\n+ **PAN-FPN改进了什么？**\n\nYOLOv5的Neck部分的结构图如下：\n\n![](docs/v5FPN.png)\n\nYOLOv6的Neck部分的结构图如下:\n\n![](docs/v6FPN.png)\n\nYOLOv8的结构图：\n\n![](docs/v8FPN.png)\n\n可以看到，相对于YOLOv5或者YOLOv6，YOLOv8将C3模块以及RepBlock替换为了C2f，同时细心可以发现，相对于YOLOv5和YOLOv6，YOLOv8选择将上采样之前的`1×1`卷积去除了，将Backbone不同阶段输出的特征直接送入了上采样操作。\n\n+ **Head部分都变了什么呢？**\n\n先看一下YOLOv5本身的Head（Coupled-Head）：\n\n![](docs/v5head.png)\n\n而YOLOv8则是使用了Decoupled-Head，回归头的通道数也变成了`4*reg_max`的形式：\n\n![](docs/v8head.png)\n\n+ **损失函数**\n\n对于YOLOv8，其分类损失为VFL Loss，其回归损失为CIOU Loss+DFL的形式，这里Reg_max默认为16。\n\nVFL主要改进是提出了非对称的加权操作，FL和QFL都是对称的。而非对称加权的思想来源于论文PISA，该论文指出首先正负样本有不平衡问题，即使在正样本中也存在不等权问题，因为mAP的计算是主正样本。\n\n![](docs/loss.png)\n\n\n\nq是label，正样本时候q为bbox和gt的IoU，负样本时候q=0，当为正样本时候其实没有采用FL，而是普通的BCE，只不过多了一个自适应IoU加权，用于突出主样本。而为负样本时候就是标准的FL了。可以明显发现VFL比QFL更加简单，主要特点是正负样本非对称加权、突出正样本为主样本。\n\n针对这里的DFL（Distribution Focal Loss），其主要是将框的位置建模成一个 general distribution，让网络快速的聚焦于和目标位置距离近的位置的分布。\n\n\n\n\n\n+ **正负样本的匹配**\n\n标签分配是目标检测非常重要的一环，在YOLOv5的早期版本中使用了MaxIOU作为标签分配方法。然而，在实践中发现直接使用边长比也可以达到一样的效果。而YOLOv8则是抛弃了Anchor-Base方法使用Anchor-Free方法，找到了一个替代边长比例的匹配方法: **TaskAligned**。为与NMS搭配，训练样例的Anchor分配需要满足以下两个规则：\n\n1. 正常对齐的Anchor应当可以预测高分类得分，同时具有精确定位；\n2. 不对齐的Anchor应当具有低分类得分，并在NMS阶段被抑制。\n\n基于上述两个目标，TaskAligned设计了一个新的Anchor alignment metric 来在Anchor level 衡量Task-Alignment的水平。并且，Alignment metric 被集成在了 sample 分配和 loss function里来动态的优化每个 Anchor 的预测。\n\n\u003e  Anchor alignment metric：\n\n分类得分和 IoU表示了这两个任务的预测效果，所以，TaskAligned使用分类得分和IoU的高阶组合来衡量Task-Alignment的程度。使用下列的方式来对每个实例计算Anchor-level 的对齐程度：\n$$\nt=s^{\\alpha}+\\mu^{\\beta} \n$$\ns 和 u 分别为分类得分和 IoU 值，α 和 β 为权重超参。从上边的公式可以看出来，t 可以同时控制分类得分和IoU 的优化来实现 Task-Alignment，可以引导网络动态的关注于高质量的Anchor。\n\n\u003e Training sample Assignment：\n\n采用一种简单的分配规则选择训练样本：对每个实例，选择m个具有最大t值的Anchor作为正样本，选择其余的Anchor作为负样本。然后，通过损失函数(针对分类与定位的对齐而设计的损失函数)进行训练。\n\n### 4.YOLOv8环境安装\n\n我们使用的是`ultralytics(8.0.0) python package`,其安装方式如下：\n\n```shell\n#pip install -i https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ ultralytics==0.0.59\n#pip install -e ultralytics\npip install ultralytics\n```\n\n你可以在`/usr/local/lib/pythonx.x/dist-packages/ultralytics `下找到安装包中的YOLOv8的源文件，进行魔改！\n\n### 5.构建自己的训练集\n\nYOLOv8可以进行分类，检测和分割类任务的学习，我们以检测类任务为例，并训练YOLOv8s,其数据集的准备完全和YOLOv5,YOLOv6,YOLOv7的一致，可以参考我们之前的项目\n\n+ https://github.com/DataXujing/YOLO-v5\n+ https://github.com/DataXujing/YOLOv6\n+ https://github.com/DataXujing/YOLOv7\n\n### 6.构建自己训练集的配置文件和模型配置文件\n\n+ 模型配置文件：\n\n```yaml\n#yolov8s.yaml\n# Parameters\nnc: 4  # number of classes\ndepth_multiple: 0.33  # scales module repeats\nwidth_multiple: 0.50  # scales convolution channels\n\n# YOLOv8.0s backbone\nbackbone:\n  # [from, repeats, module, args]\n  - [-1, 1, Conv, [64, 3, 2]]  # 0-P1/2\n  - [-1, 1, Conv, [128, 3, 2]]  # 1-P2/4\n  - [-1, 3, C2f, [128, True]]\n  - [-1, 1, Conv, [256, 3, 2]]  # 3-P3/8\n  - [-1, 6, C2f, [256, True]]\n  - [-1, 1, Conv, [512, 3, 2]]  # 5-P4/16\n  - [-1, 6, C2f, [512, True]]\n  - [-1, 1, Conv, [1024, 3, 2]]  # 7-P5/32\n  - [-1, 3, C2f, [1024, True]]\n  - [-1, 1, SPPF, [1024, 5]]  # 9\n\n# YOLOv8.0s head\nhead:\n  - [-1, 1, nn.Upsample, [None, 2, 'nearest']]\n  - [[-1, 6], 1, Concat, [1]]  # cat backbone P4\n  - [-1, 3, C2f, [512]]  # 13\n\n  - [-1, 1, nn.Upsample, [None, 2, 'nearest']]\n  - [[-1, 4], 1, Concat, [1]]  # cat backbone P3\n  - [-1, 3, C2f, [256]]  # 17 (P3/8-small)\n\n  - [-1, 1, Conv, [256, 3, 2]]\n  - [[-1, 12], 1, Concat, [1]]  # cat head P4\n  - [-1, 3, C2f, [512]]  # 20 (P4/16-medium)\n\n  - [-1, 1, Conv, [512, 3, 2]]\n  - [[-1, 9], 1, Concat, [1]]  # cat head P5\n  - [-1, 3, C2f, [1024]]  # 23 (P5/32-large)\n\n  - [[15, 18, 21], 1, Detect, [nc]]  # Detect(P3, P4, P5)\n\n```\n\n+ 数据集配置文件\n\n```yaml\n#score_data.yaml\n\n# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]\ntrain: ./dataset/score/images/train # train images\nval: ./dataset/score/images/val # val images\n#test: ./dataset/score/images/test # test images (optional)\n\n# Classes\nnames:\n  0: person\n  1: cat\n  2: dog\n  3: horse\n\n```\n\n+ 训练超参数配置文件\n\n我们对训练的超参数进行了简单的修改，通过命令行参数传入，也可以通过配置文件进行配置。\n\n```yaml\ntask: \"detect\" # choices=['detect', 'segment', 'classify', 'init'] # init is a special case. Specify task to run.\nmode: \"train\" # choices=['train', 'val', 'predict'] # mode to run task in.\n\n# Train settings -------------------------------------------------------------------------------------------------------\nmodel: null # i.e. yolov8n.pt, yolov8n.yaml. Path to model file\ndata: null # i.e. coco128.yaml. Path to data file\nepochs: 100 # number of epochs to train for\npatience: 50  # TODO: epochs to wait for no observable improvement for early stopping of training\nbatch: 16 # number of images per batch\nimgsz: 640 # size of input images\nsave: True # save checkpoints\ncache: False # True/ram, disk or False. Use cache for data loading\ndevice: '' # cuda device, i.e. 0 or 0,1,2,3 or cpu. Device to run on\nworkers: 8 # number of worker threads for data loading\nproject: null # project name\nname: null # experiment name\nexist_ok: False # whether to overwrite existing experiment\npretrained: False # whether to use a pretrained model\noptimizer: 'SGD' # optimizer to use, choices=['SGD', 'Adam', 'AdamW', 'RMSProp']\n...\n```\n\n### 7.YOLOv8目标检测任务训练\n\n```shell\nyolo task=detect mode=train model=yolov8s.yaml  data=score_data.yaml epochs=100 batch=64 imgsz=640 pretrained=False optimizer=SGD \n```\n\n![](docs/train_log1.png)\n\n![](docs/PR_curve.png)\n\n### 8.YOLOv8推断Demo\n\n```shell\n# 自己实现的推断程序\npython3 inference.py\n```\n\n| ![](docs/bus.jpg)  | ![](docs/cat1.jpg)   |\n| ------------------ | -------------------- |\n| ![](docs/dog1.jpg) | ![](docs/zidane.jpg) |\n\n\n\n### 9.YOLOv8端到端模TensorRT模型加速\n\n1. pth模型转onnx\n\n```shell\n#CLI\nyolo task=detect mode=export model=./runs/detect/train/weights/last.pt format=onnx simplify=True opset=13\n\n# python\nfrom ultralytics import YOLO\n\nmodel = YOLO(\"./runs/detect/train/weights/last.pt \")  # load a pretrained YOLOv8n model\nmodel.export(format=\"onnx\")  # export the model to ONNX format\n```\n\n2. 增加NMS Plugin \n\n执行`tensorrt/`下的如下代码，添加NMS到YOLOv8模型\n\n+ 添加后处理\n\n```shell\npython3 yolov8_add_postprocess.py\n```\n\n+ 添加NMS plugin\n\n```shell\npython3 yolov8_add_nms.py\n```\n\n生成`last_1_nms.onnx`,打开该文件对比和原onnx文件的区别，发现增加了如下节点(完成了将NMS添加到onnx的目的）：\n\n![](docs/nms.png)\n\n3. onnx转trt engine\n\n```shell\ntrtexec --onnx=last_1_nms.onnx --saveEngine=yolov8s.plan --workspace=3000 --verbose\n```\n\n![](docs/trt.png)\n\n出现上述界面，onnx正常序列化为TRT engine.\n\n4. TRT C++推断\n\n在win 10下基于RTX 1060 TensorRT 8.2.1进行测试，我们的开发环境是VS2017,**所有C++代码已经存放在`tensorrt/`文件夹下**。其推断结果如下图所示（可以发现我们实现了YOLOv8的TensorRT端到端的推断，其推断结果与原训练框架保持一致）：\n\n| ![](tensorrt/yolov8/yolov8/res/bus.jpg)  | ![](tensorrt/yolov8/yolov8/res/cat1.jpg)   |\n| ---------------------------------------- | ------------------------------------------ |\n| ![](tensorrt/yolov8/yolov8/res/dog1.jpg) | ![](tensorrt/yolov8/yolov8/res/zidane.jpg) |\n\n\n\n### 9.YOLOv8端到端华为昇腾模型推断加速\n\n\u003e 由于其他原因，该部分代码不开源。\n\n这一部分我们将在华为昇腾下测试如何端到端实现YOLOv8的推断，华为昇腾目前支持的算子还是很有限的，onnx的NMS算子华为昇腾是支持的，因此我们需要将onnx的NMS算子添加到YOLOv8的onnx文件中，并将模型转化到昇腾架构下运行。这部分代码我们存放在`Ascend/`下。\n\n1. pth转onnx\n\n2. 增加onnx NMS算子结点\n\n3. ATC转.om模型\n\n4. 华为昇腾C++推断\n\n### 10. QT + NCNN 小米手机端部署YOLOv8s\n\n\u003chttps://github.com/DataXujing/ncnn_android_yolov8\u003e\n\n### 参考文献：\n\n+ https://github.com/ultralytics/ultralytics\n\n+ https://mp.weixin.qq.com/s/_OvSTQZlb5jKti0JnIy0tQ\n+ https://github.com/ultralytics/assets/releases\n+ https://v8docs.ultralytics.com/\n+ https://pypi.org/project/ultralytics/0.0.44/#description\n+ https://mp.weixin.qq.com/s/-4pn--3kFI_J1oX6p5GWVQ\n+ https://github.com/uyolo1314/ultralytics\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDataXujing%2FYOLOv8","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FDataXujing%2FYOLOv8","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FDataXujing%2FYOLOv8/lists"}