{"id":13643195,"url":"https://github.com/Sharpiless/Yolov5-distillation-train-inference","last_synced_at":"2025-04-20T21:33:18.621Z","repository":{"id":112377842,"uuid":"365103541","full_name":"Sharpiless/Yolov5-distillation-train-inference","owner":"Sharpiless","description":"Yolov5 distillation training | Yolov5知识蒸馏训练，支持训练自己的数据","archived":false,"fork":false,"pushed_at":"2022-10-29T09:13:04.000Z","size":2477,"stargazers_count":200,"open_issues_count":9,"forks_count":32,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-08-02T01:19:27.956Z","etag":null,"topics":["distillation","konwledge-distillation","model-compression","object-detection","yolov5"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Sharpiless.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-05-07T03:28:26.000Z","updated_at":"2024-07-13T11:14:01.000Z","dependencies_parsed_at":"2023-05-26T03:47:14.576Z","dependency_job_id":null,"html_url":"https://github.com/Sharpiless/Yolov5-distillation-train-inference","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2FYolov5-distillation-train-inference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2FYolov5-distillation-train-inference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2FYolov5-distillation-train-inference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2FYolov5-distillation-train-inference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Sharpiless","download_url":"https://codeload.github.com/Sharpiless/Yolov5-distillation-train-inference/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223839442,"owners_count":17211942,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distillation","konwledge-distillation","model-compression","object-detection","yolov5"],"created_at":"2024-08-02T01:01:43.610Z","updated_at":"2024-11-09T14:31:55.973Z","avatar_url":"https://github.com/Sharpiless.png","language":"Python","funding_links":[],"categories":["Lighter and Deployment Frameworks"],"sub_categories":[],"readme":"# 代码地址：\n\nhttps://github.com/Sharpiless/Yolov5-distillation-train-inference\n\n# 最新版本：\n\n请移步：https://github.com/Sharpiless/yolov5-distillation-5.0\n\n\u003ca align=\"left\" href=\"https://apps.apple.com/app/id1452689527\" target=\"_blank\"\u003e\n\u003cimg width=\"800\" src=\"https://user-images.githubusercontent.com/26833433/98699617-a1595a00-2377-11eb-8145-fc674eb9b1a7.jpg\"\u003e\u003c/a\u003e\n\n![](https://github.com/Sharpiless/Yolov5-distillation-train-inference/blob/main/data/images/YOLOv5%E8%92%B8%E9%A6%8F.png)\n\n# 教师模型权重：\n\n链接：[https://pan.baidu.com/s/13gq5QwCrRNdRXWzSYUeJIw ](https://pan.baidu.com/s/13gq5QwCrRNdRXWzSYUeJIw )\n\n提取码：4ppv \n\n# 蒸馏训练：\n\n```bash\npython train_distill.py --weights yolov5s.pt \\\n    --teacher weights/yolov5l_voc.pt --distill_ratio 0.001 \\\n    --teacher-cfg model/yolov5l.yaml --data data/voc.yaml \\\n    --epochs 30 --batch-size 16\n```\n\n# 训练参数:\n\n\u003e --weights：预训练模型\n\n\u003e --teacher：教师模型权重\n\n\u003e --distill-ratio：蒸馏损失权重\n\n\u003e --with-gt-loss：是否同时使用ground truth\n\n\u003e --soft-loss：是否使用KL散度作为蒸馏的类别损失（缺省使用L2-logits损失）\n\n\u003e --full-output-loss：是否使用[《Object detection at 200 Frames Per Second》](https://arxiv.org/pdf/1805.06361.pdf)中的损失\n\n这篇文章分别对这几个损失函数做出改进，具体思路为只有当teacher network的objectness value高时，才学习bounding box坐标和class probabilities。\n\n![](https://github.com/Sharpiless/Yolov5-distillation-train-inference/blob/main/data/images/full_loss.png)\n\n# 准备数据集：\n\n默认会启用 data/voc.yaml 自动下载VOC数据集进行训练\n\n或者手动运行 data/scripts/get_voc2007.sh 下载\n\n如需修改成自己的数据集，则只需要修改yaml路径即可\n\n# 实验结果：\n\n数据集：\n\nVOC2007（补充的无标签数据使用VOC2012）\n\nGPU：2080Ti*1\n\nBatch Size：16\n\nEpoches：30\n\nBaseline：Yolov5s\n\nTeacher model：Yolov5l（mAP 0.5:0.95 = 0.541）\n\n\n这里假设VOC2012中新增加的数据为无标签数据（2k张）。\n\n|教师模型|训练方法|蒸馏损失|P|R|mAP50|\n|:----|:----|:----|:----|:----|:----|\n|无|正常训练|不使用|0.7756|0.7115|0.7609|\n|Yolov5l|output based|l2|0.7585|0.7198|0.7644|\n|Yolov5l|output based|KL|0.7417|0.7207|0.7536|\n|Yolov5m|output based|l2|0.7682|0.7436|0.7976|\n|Yolov5m|output based|KL|0.7731|0.7313|0.7931|\n\n![训练结果](https://github.com/Sharpiless/yolov5-distillation-5.0/blob/main/images/line.png)\n\n\n参数和细节正在完善，支持KL散度、L2 logits损失和Sigmoid蒸馏损失等\n\n# 待做事项：\n\n- [√] 修改logist输出作为蒸馏损失输入\n- [√] 完善代码结构和相关参数设定\n- [×] 查找为何蒸馏损失不起作用（或者收敛慢）的原因\n- [×] 完善相关实验并测试精度\n- [√] 修改dataloader加快训练速度\n- [√] 修改teacher model的批量推理加快训练速度\n\n# 可能存在的问题：\n\n- 1.训练轮数太少没收敛，可能蒸馏训练收敛满最终结果高\n- 2.教师模型是Yolov5l在VOC训练30轮得到的（mAP 0.5:0.95 = 0.541），质量比标注较差影响蒸馏训练的结果\n- 3.可调整的参数还有很多（教师模型的检测、IOU阈值，蒸馏损失种类，蒸馏损失比率等）\n\n# 我的公众号：\n\n![在这里插入图片描述](https://img-blog.csdnimg.cn/20210310070958646.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDkzNjg4OQ==,size_16,color_FFFFFF,t_70)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSharpiless%2FYolov5-distillation-train-inference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSharpiless%2FYolov5-distillation-train-inference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSharpiless%2FYolov5-distillation-train-inference/lists"}