{"id":13442822,"url":"https://github.com/ChengpengChen/RepGhost","last_synced_at":"2025-03-20T15:31:08.842Z","repository":{"id":63228306,"uuid":"564657072","full_name":"ChengpengChen/RepGhost","owner":"ChengpengChen","description":"RepGhost: A Hardware-Efficient Ghost Module via Re-parameterization","archived":false,"fork":false,"pushed_at":"2023-08-15T11:21:51.000Z","size":3233,"stargazers_count":168,"open_issues_count":1,"forks_count":17,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-08-01T03:42:01.204Z","etag":null,"topics":["light-weight","re-parameterization","repghost","repghostnet"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ChengpengChen.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-11-11T07:31:35.000Z","updated_at":"2024-07-28T13:45:13.000Z","dependencies_parsed_at":"2024-01-18T14:41:19.516Z","dependency_job_id":"55457c34-495d-426a-b3a3-9b03a93b428b","html_url":"https://github.com/ChengpengChen/RepGhost","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChengpengChen%2FRepGhost","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChengpengChen%2FRepGhost/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChengpengChen%2FRepGhost/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChengpengChen%2FRepGhost/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ChengpengChen","download_url":"https://codeload.github.com/ChengpengChen/RepGhost/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221772568,"owners_count":16878131,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["light-weight","re-parameterization","repghost","repghostnet"],"created_at":"2024-07-31T03:01:51.687Z","updated_at":"2024-10-28T03:31:08.830Z","avatar_url":"https://github.com/ChengpengChen.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"## RepGhost: A Hardware-Efficient Ghost Module via Re-parameterization\n\nThe official pytorch implementation of the paper **[RepGhost: A Hardware-Efficient Ghost Module via Re-parameterization](https://arxiv.org/abs/2211.06088)**\n\n#### Chengpeng Chen\\*, Zichao Guo\\*, Haien Zeng, Pengfei Xiong, Jian Dong\n\n\u003eFeature reuse has been a key technique in light-weight convolutional neural networks (CNNs) design. Current methods usually utilize a concatenation operator to keep large channel numbers cheaply (thus large network capacity) by reusing feature maps from other layers. Although concatenation is parameters- and FLOPs-free, its computational cost on hardware devices is non-negligible. To address this, this paper provides a new perspective to realize feature reuse via structural re-parameterization technique. A novel hardware-efficient RepGhost module is proposed for implicit feature reuse via re-parameterization, instead of using concatenation operator. Based on the RepGhost module, we develop our efficient RepGhost bottleneck and RepGhostNet. Experiments on ImageNet and COCO benchmarks demonstrate that the proposed RepGhostNet is much more effective and efficient than GhostNet and MobileNetV3 on mobile devices. Specially, our RepGhostNet surpasses GhostNet 0.5x by 2.5% Top-1 accuracy on ImageNet dataset with less parameters and comparable latency on an ARM-based mobile phone.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"figs/repghost_bottleneck.png\" width=512\u003e\n\u003c/p\u003e\n\n```python\npython 3.9.12\npytorch 1.11.0\ncuda 11.3\ntimm 0.6.7\n```\n\n```bash\ngit clone https://github.com/ChengpengChen/RepGhost\ncd RepGhost\npip install -r requirements.txt\n```\n\n### Training\n```bash\nbash distributed_train.sh 8 --model repghost.repghostnet_0_5x -b 128 --lr 0.6 --sched cosine --epochs 300 --opt sgd -j 7 --warmup-epochs 5 --warmup-lr 1e-4 --weight-decay 1e-5 --drop 0.2 --amp --model-ema --model-ema-decay 0.9999 --remode pixel --reprob 0.2 --output work_dirs/train/  --data_dir {path_to_imagenet_dir}\n```\n\n### Validation\n```bash\npython3 -m torch.distributed.launch --nproc_per_node=8 --master_port=2340 validate.py -b 32 --model-ema --model {model} --resume {checkpoint_path} --data_dir {path_to_imagenet_dir}\n```\n\n### Convert a training-time RepGhost into a fast-inference one\nTo check the conversion example at ```convert.py```. You can also convert RepGhostNet model for fast inference via:\n\n```python\nmodel.convert_to_deploy()\n```\n\n### Results and Pre-trained Models\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"figs/accuracy_to_latency.png\" width=512\u003e\n\u003c/p\u003e\n\n| RepGhostNet | Params(M) | FLOPs(M) | Latency(ms) | Top-1 Acc.(%) | Top-5 Acc.(%) | checkpoints                                                                                                                                                          | logs                                                            |\n|:------------|:----------|:---------|:------------|:--------------|:--------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------|\n| 0.5x        | 2.3       | 43       | 25.1        | 66.9          | 86.9          | [gdrive](https://drive.google.com/file/d/16AGg-kSscFXDpXPZ3cJpYwqeZbUlUoyr/view?usp=share_link) \\ [百度网盘](https://pan.baidu.com/s/1s-tuS8JoHVoCVHWuUiHUFw?pwd=qttp)   | [log](./work_dirs/train/repghostnet_0_5x_43M_66.95/train.log)   |\n| 0.58x       | 2.5       | 60       | 31.9        | 68.9          | 88.4          | [gdrive](https://drive.google.com/file/d/1L6ccPjfnCMt5YK-pNFDfqGYvJyTRyZPR/view?usp=share_link) \\ [百度网盘](https://pan.baidu.com/s/1bnVk2ILONqPEbmTQahZ0Og?pwd=tiyw)   | [log](./work_dirs/train/repghostnet_0_58x_60M_68.94/train.log)  |\n| 0.8x        | 3.3       | 96       | 44.5        | 72.2          | 90.5          | [gdrive](https://drive.google.com/file/d/13gmUpwiJF_O05f3-3UeEyKD57veL5cG-/view?usp=share_link) \\ [百度网盘](https://pan.baidu.com/s/1L_EJ0CnQeGpd0QBoOiY7oQ?pwd=rkd8)   | [log](./work_dirs/train/repghostnet_0_8x_96M_72.24/train.log)   |\n| 1.0x        | 4.1       | 142      | 62.2        | 74.2          | 91.5          | [gdrive](https://drive.google.com/file/d/1gzfGln60urfY38elpPHVTyv9b94ukn5o/view?usp=share_link) \\ [百度网盘](https://pan.baidu.com/s/1CEwuBLV05z7zrVbBrku59w?pwd=z4s7)   | [log](./work_dirs/train/repghostnet_1_0x_142M_74.22/train.log)  |\n| 1.11x       | 4.5       | 170      | 71.5        | 75.1          | 92.2          | [gdrive](https://drive.google.com/file/d/14Lk4pKWIUFk1Mb53ooy_GsZbhMmz3iVE/view?usp=share_link) \\ [百度网盘](https://pan.baidu.com/s/1Lb54Jiqyt0Jc6X4F_tUYnw?pwd=dwcb)   | [log](./work_dirs/train/repghostnet_1_11x_170M_75.07/train.log) |\n| 1.3x        | 5.5       | 231      | 92.9        | 76.4          | 92.9          | [gdrive](https://drive.google.com/file/d/1dNHpX2JyiuTcDmmyvr8gnAI9t8RM-Nui/view?usp=share_link) \\ [百度网盘](https://pan.baidu.com/s/19x_OUgxRDvwh2g4E9gN12Q?pwd=uux6)   | [log](./work_dirs/train/repghostnet_1_3x_231M_76.37/train.log)  |\n| 1.5x        | 6.6       | 301      | 116.9       | 77.5          | 93.5          | [gdrive](https://drive.google.com/file/d/1TWAY654Dz8zcwhDBDN6QDWhV7as30P8e/view?usp=share_link) \\ [百度网盘](https://pan.baidu.com/s/15UWOMRQN5vw99QbgiFWMRw?pwd=3uqq)   | [log](./work_dirs/train/repghostnet_1_5x_301M_77.45/train.log)  |\n| 2.0x        | 9.8       | 516      | 190.0       | 78.8          | 94.3          | [gdrive](https://drive.google.com/file/d/12k00eWCXhKxx_fq3ewDhCNX08ftJ-iyP/view?usp=share_link) \\ [百度网盘](https://pan.baidu.com/s/1YbtYvIBt3tTqCzvbcjJuBw?pwd=nq1r)   | [log](./work_dirs/train/repghostnet_2_0x_516M_78.81/train.log)  |\n\n#### Parameters and FLOPs\nWe calculate parameters and FLOPs using a modified [thop](https://github.com/Lyken17/pytorch-OpCounter) in ```tools.py```. It only counts infos of convolutional and full-connected layers, without BN. To use it in your code:\n\n```python\nfrom tools import cal_flops_params\nflops, params = cal_flops_params(model, input_size=(1, 3, 224, 224))\n```\n\n#### Latency\nWe first export our pytorch model to a ONNX one, and then use [MNN](https://github.com/alibaba/MNN) to convert it to MNN format, at last evaluate its latency on an ARM-based mobile phone.\n\n#### Comparisons to MobileOne on iPhone12 CPU\nWe compare RepGhostNet to [MobileOne](https://arxiv.org/abs/2206.04040) on iPhone12 CPU based on [ModelBench](https://github.com/apple/ml-mobileone/tree/main/ModelBench). The result is shown below.\nThe MobileOne series models are $\\mu0$, $\\mu1$, $\\mu2$, S0, S1, S2, S3 and S4.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"figs/accuracy_to_latency_iphone12cpu.png\" width=384\u003e\n\u003c/p\u003e\n\n\n### Citations\nIf RepGhostNet helps your research or work, please consider citing:\n\n```\n@article{chen2022repghost,\n  title={RepGhost: A Hardware-Efficient Ghost Module via Re-parameterization},\n  author={Chen, Chengpeng, and Guo, Zichao, and Zeng, Haien, and Xiong, Pengfei and Dong, Jian},\n  journal={arXiv preprint arXiv:2211.06088},\n  year={2022}\n}\n```\n\n### Contact\n\nIf you have any questions, please contact chencp@live.com.\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FChengpengChen%2FRepGhost","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FChengpengChen%2FRepGhost","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FChengpengChen%2FRepGhost/lists"}