{"id":13643204,"url":"https://github.com/Sharpiless/yolov5-knowledge-distillation","last_synced_at":"2025-04-20T21:33:14.481Z","repository":{"id":62147440,"uuid":"394873810","full_name":"Sharpiless/yolov5-knowledge-distillation","owner":"Sharpiless","description":"yolov5目标检测模型的知识蒸馏（基于响应的蒸馏）","archived":false,"fork":false,"pushed_at":"2021-08-11T05:35:31.000Z","size":116,"stargazers_count":92,"open_issues_count":3,"forks_count":16,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-11-09T14:42:12.012Z","etag":null,"topics":["knowledge-distillation","object-detection","yolo","yolov5"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Sharpiless.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-11T05:32:19.000Z","updated_at":"2024-10-31T03:07:23.000Z","dependencies_parsed_at":"2022-10-27T09:00:56.298Z","dependency_job_id":null,"html_url":"https://github.com/Sharpiless/yolov5-knowledge-distillation","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2Fyolov5-knowledge-distillation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2Fyolov5-knowledge-distillation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2Fyolov5-knowledge-distillation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sharpiless%2Fyolov5-knowledge-distillation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Sharpiless","download_url":"https://codeload.github.com/Sharpiless/yolov5-knowledge-distillation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249965566,"owners_count":21352926,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["knowledge-distillation","object-detection","yolo","yolov5"],"created_at":"2024-08-02T01:01:43.757Z","updated_at":"2025-04-20T21:33:14.080Z","avatar_url":"https://github.com/Sharpiless.png","language":"Python","readme":"# 代码地址：\n\n[https://github.com/Sharpiless/yolov5-knowledge-distillation](https://github.com/Sharpiless/yolov5-knowledge-distillation)\n\n\u003cimg width=\"800\" src=\"https://user-images.githubusercontent.com/26833433/98699617-a1595a00-2377-11eb-8145-fc674eb9b1a7.jpg\"\u003e\u003c/a\u003e\n\n# 教师模型：\n\n```bash\npython train.py --weights weights/yolov5m.pt \\\n        --cfg models/yolov5m.yaml --data data/voc.yaml --epochs 50 \\\n        --batch-size 8 --device 0 --hyp data/hyp.scratch.yaml \n```\n\n# 蒸馏训练：\n\n```bash\npython train.py --weights weights/yolov5s.pt \\\n        --cfg models/yolov5s.yaml --data data/voc.yaml --epochs 50 \\\n        --batch-size 8 --device 0 --hyp data/hyp.scratch.yaml \\\n        --t_weights yolov5m.pt --distill\n```\n\n# 训练参数:\n\n\u003e --weights：预训练模型\n\n\u003e --t_weights：教师模型权重\n\n\u003e --distill：使用知识蒸馏进行训练\n\n\u003e --dist_loss：l2或者kl\n\n\u003e --temperature：使用知识蒸馏时的温度\n\n\n使用[《Object detection at 200 Frames Per Second》](https://arxiv.org/pdf/1805.06361.pdf)中的损失\n\n这篇文章分别对这几个损失函数做出改进，具体思路为只有当teacher network的objectness value高时，才学习bounding box坐标和class probabilities。\n\n![](https://github.com/Sharpiless/Yolov5-distillation-train-inference/blob/main/data/images/full_loss.png)\n\n![](https://github.com/Sharpiless/Yolov5-distillation-train-inference/blob/main/data/images/all.png)\n\n\n# 实验结果：\n\n\n这里假设VOC2012中新增加的数据为无标签数据（2k张）。\n\n|教师模型|训练方法|蒸馏损失|P|R|mAP50|\n|:----|:----|:----|:----|:----|:----|\n|无|正常训练|不使用|0.7756|0.7115|0.7609|\n|Yolov5l|output based|l2|0.7585|0.7198|0.7644|\n|Yolov5l|output based|KL|0.7417|0.7207|0.7536|\n|Yolov5m|output based|l2|0.7682|0.7436|0.7976|\n|Yolov5m|output based|KL|0.7731|0.7313|0.7931|\n\n![训练结果](https://github.com/Sharpiless/yolov5-distillation-5.0/blob/main/images/line.png)\n\n参数和细节正在完善，支持KL散度、L2 logits损失和Sigmoid蒸馏损失等\n\n## 1. 正常训练：\n\n![正常训练](https://github.com/Sharpiless/yolov5-distillation-5.0/blob/main/images/%E6%AD%A3%E5%B8%B8%E8%AE%AD%E7%BB%83.png)\n\n## 2. L2蒸馏损失：\n\n![L2蒸馏损失](https://github.com/Sharpiless/yolov5-distillation-5.0/blob/main/images/l+l2.png)\n\n# 我的公众号：\n\n![在这里插入图片描述](https://img-blog.csdnimg.cn/20210310070958646.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NDkzNjg4OQ==,size_16,color_FFFFFF,t_70)\n\n# 关于作者\n\u003e B站：[https://space.bilibili.com/470550823](https://space.bilibili.com/470550823)\n\n\u003e CSDN：[https://blog.csdn.net/weixin_44936889](https://blog.csdn.net/weixin_44936889)\n\n\u003e AI Studio：[https://aistudio.baidu.com/aistudio/personalcenter/thirdview/67156](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/67156)\n\n\u003e Github：[https://github.com/Sharpiless](https://github.com/Sharpiless)\n","funding_links":[],"categories":["Lighter and Deployment Frameworks"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSharpiless%2Fyolov5-knowledge-distillation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSharpiless%2Fyolov5-knowledge-distillation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSharpiless%2Fyolov5-knowledge-distillation/lists"}