{"id":20153077,"url":"https://github.com/huawei-noah/addernet","last_synced_at":"2025-04-13T04:16:27.127Z","repository":{"id":46101356,"uuid":"242959055","full_name":"huawei-noah/AdderNet","owner":"huawei-noah","description":"Code for paper \" AdderNet: Do We Really Need Multiplications in Deep Learning?\"","archived":false,"fork":false,"pushed_at":"2022-03-19T02:54:28.000Z","size":1386,"stargazers_count":958,"open_issues_count":12,"forks_count":185,"subscribers_count":24,"default_branch":"master","last_synced_at":"2025-04-13T04:16:12.858Z","etag":null,"topics":["convolutional-neural-networks","cvpr2020","efficient-inference","imagenet","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/huawei-noah.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"License.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-25T09:22:03.000Z","updated_at":"2025-04-07T22:43:56.000Z","dependencies_parsed_at":"2022-09-03T02:43:17.821Z","dependency_job_id":null,"html_url":"https://github.com/huawei-noah/AdderNet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huawei-noah%2FAdderNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huawei-noah%2FAdderNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huawei-noah%2FAdderNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/huawei-noah%2FAdderNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/huawei-noah","download_url":"https://codeload.github.com/huawei-noah/AdderNet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248661716,"owners_count":21141451,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","cvpr2020","efficient-inference","imagenet","pytorch"],"created_at":"2024-11-13T23:16:58.599Z","updated_at":"2025-04-13T04:16:27.101Z","avatar_url":"https://github.com/huawei-noah.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AdderNet: Do We Really Need Multiplications in Deep Learning?\r\nThis code is a demo of CVPR 2020 paper [AdderNet: Do We Really Need Multiplications in Deep Learning?](https://openaccess.thecvf.com/content_CVPR_2020/papers/Chen_AdderNet_Do_We_Really_Need_Multiplications_in_Deep_Learning_CVPR_2020_paper.pdf) \r\n\r\nWe present adder networks (AdderNets) to trade massive multiplications in deep neural networks, especially convolutional neural networks (CNNs), for much cheaper additions to reduce computation costs. In AdderNets, we take the L1-norm distance between filters and input feature as the output response. As a result, the proposed AdderNets can achieve 74.9% Top-1 accuracy 91.7% Top-5 accuracy using ResNet-50 on the ImageNet dataset without any multiplication in convolution layer.\r\n\r\nRun `python main.py` to train on CIFAR-10. \r\n\r\n\u003cp align=\"center\"\u003e\r\n\u003cimg src=\"figures/visualization.png\" width=\"800\"\u003e\r\n\u003c/p\u003e\r\n\r\nClassification results on CIFAR-10 and CIFAR-100 datasets.\r\n\r\n| Model     | Method           | CIFAR-10 | CIFAR-100 |\r\n| --------- | ---------------- | -------- | --------- |\r\n| VGG-small | ANN           | 93.72%   | 72.64%    |\r\n|           | PKKD ANN      | 95.03%   | 76.94%    |\r\n|           | SLAC ANN      | 93.96%   | 73.63%    |\r\n|           |                  |          |           |\r\n| ResNet-20 | ANN              | 92.02%   | 67.60%    |\r\n|           | PKKD ANN         | 92.96%   | 69.93%    |\r\n|           | SLAC ANN         | 92.29%   | 68.31%    |\r\n|           | ShiftAddNet* | 89.32%(160epoch)   | -         |\r\n|           |                  |          |           |\r\n| ResNet-32 | ANN              | 93.01%   | 69.17%    |\r\n|           | PKKD ANN         | 93.62%   | 72.41%    |\r\n|           | SLAC ANN         | 93.24%   | 69.83%    |\r\n\r\nClassification results on ImageNet dataset.\r\n\r\n| Model     | Method       | Top-1 Acc | Top-5 Acc |\r\n| --------- | ------------ | --------- | --------- |\r\n| ResNet-18 | CNN          | 69.8%     | 89.1%     |\r\n|           | ANN     | 67.0%     | 87.6%     |\r\n|           | PKKD ANN  | 68.8%     | 88.6%     |\r\n|           | SLAC ANN  | 67.7%     | 87.9%     |\r\n|           |              |           |           |\r\n| ResNet-50 | CNN          | 76.2%     | 92.9%     |\r\n|           | ANN          | 74.9%     | 91.7%     |\r\n|           | PKKD ANN     | 76.8%     | 93.3%     |\r\n|           | SLAC ANN  | 75.3%     | 92.6%     |\r\n\r\n*ShiftAddNet used different training setting.\r\n\r\nSuper-Resolution results on several SR datasets.\r\n\r\n| Scale | Model | Method  | Set5 (PSNR/SSIM) | Set14 (PSNR/SSIM) | B100 (PSNR/SSIM) | Urban100 (PSNR/SSIM) |\r\n| ----- | ----- | ------- | ---------------- | ----------------- | ---------------- | -------------------- |\r\n| ×2    | VDSR  | CNN     | 37.53/0.9587     | 33.03/0.9124      | 31.90/0.8960     | 30.76/0.9140         |\r\n|       |       | ANN     | 37.37/0.9575     | 32.91/0.9112      | 31.82/0.8947     | 30.48/0.9099         |\r\n|       | EDSR  | CNN     | 38.11/0.9601     | 33.92/0.9195      | 32.32/0.9013     | 32.93/0.9351         |\r\n|       |       | ANN     | 37.92/0.9589     | 33.82/0.9183      | 32.23/0.9000     | 32.63/0.9309         |\r\n| ×3    | VDSR  | CNN     | 33.66/0.9213     | 29.77/0.8314      | 28.82/0.7976     | 27.14/0.8279         |\r\n|       |       | ANN     | 33.47/0.9151     | 29.62/0.8276      | 28.72/0.7953     | 26.95/0.8189         |\r\n|       | EDSR  | CNN     | 34.65/0.9282     | 30.52/0.8462      | 29.25/0.8093     | 28.80/0.8653         |\r\n|       |       | ANN     | 34.35/0.9212     | 30.33/0.8420      | 29.13/0.8068     | 28.54/0.8555         |\r\n| ×4    | VDSR  | CNN     | 31.35/0.8838     | 28.01/0.7674      | 27.29/0.7251     | 25.18/0.7524         |\r\n|       |       | ANN     | 31.27/0.8762     | 27.93/0.7630      | 27.25/0.7229     | 25.09/0.7445         |\r\n|       | EDSR  | CNN     | 32.46/0.8968     | 28.80/0.7876      | 27.71/0.7420     | 26.64/0.8033         |\r\n|       |       | ANN     | 32.13/0.8864     | 28.57/0.7800      | 27.58/0.7368     | 26.33/0.7874         |\r\n\r\nAdversarial robustness on CIFAR-10 under white-box attacks without adversarial training.\r\n\r\n\r\n| Model | Method | Clean | FGSM | BIM7 | PGD7 | MIM5 | RFGSM5| \r\n| ----- | ------ | ---- | ----- | ----- | ----- | ----- | ----- |\r\n| ResNet-20 | CNN | 92.68 | 16.33 | 0.00 | 0.00 | 0.01 | 0.00 |\r\n|       | ANN | 91.72 | 18.42 | 0.00 | 0.00 | 0.04 | 0.00 |\r\n|       | CNN-R | 90.62 | 17.23 | 3.46 | 3.67 | 4.23 | 0.06 |\r\n|       | ANN-R | 90.95 | 29.93 | 29.30 | 29.72 | 32.25 | 3.38 |\r\n|       | ANN-R-AWN | 90.55 | 45.93 | 42.62 | 43.39 | 46.52 | 18.36 |\r\n|  |  |  |  |  |  |  |  |\r\n| ResNet-32 | CNN | 92.78 | 23.55 | 0.00 | 0.01 | 0.10 | 0.00 |\r\n|       | ANN | 92.48 | 35.85 | 0.03 | 0.11 | 1.04 | 0.02 |\r\n|       | CNN-R | 91.32 | 20.41 | 5.15 | 5.27 | 6.09 | 0.07 |\r\n|       | ANN-R | 91.68 | 19.74 | 15.96 | 16.08 | 17.48 | 0.07 |\r\n|       | ANN-R-AWN | 91.25 | 61.30 | 59.41 | 59.74 | 61.54 | 39.79 |\r\n\r\nComparisons of mAP on PASCAL VOC.\r\n\r\n| Model | Backbone | Neck | mAP |\r\n| ----- | ------ | ---- | ----- |\r\n| Faster R-CNN | Conv R50 | Conv | 79.5 |\r\n| FCOS | Conv R50 | Conv | 79.1 |\r\n| RetinaNet | Conv R50 | Conv | 77.3 |\r\n| FoveaBox | Conv R50 | Conv | 76.6 |\r\n| Adder-FCOS | Adder R50 | Adder | 76.5 |\r\n\r\n\r\n## Requirements\r\n\r\n- python 3\r\n- pytorch \u003e= 1.1.0\r\n- torchvision\r\n\r\n### Preparation\r\nYou can follow [pytorch/examples](https://github.com/pytorch/examples/tree/master/imagenet) to prepare the ImageNet data.\r\n\r\nThe pretrained models are available in [google drive](https://drive.google.com/drive/folders/11ZdIst5Vwqx9Y5zHhirfaI94_7RhcBZH?usp=sharing) or [baidu cloud](https://pan.baidu.com/s/1pkaWhhKVoWPv-MCUjvxzCw) (access code:126b)\r\n\r\n### Usage\r\nRun `python main.py` to train on CIFAR-10. \r\n\r\nRun `python test.py --data_dir 'path/to/imagenet_root/'` to evaluate on ImageNet `val` set. You will achieve 74.9% Top accuracy and 91.7% Top-5 accuracy on the ImageNet dataset using ResNet-50.\r\n\r\nRun `python test.py --dataset cifar10 --model_dir models/ResNet20-AdderNet.pth --data_dir 'path/to/cifar10_root/'` to evaluate on CIFAR-10. You will achieve 91.8% accuracy on the CIFAR-10 dataset using ResNet-20.\r\n\r\nThe inference and training of AdderNets is slow since the adder filters is implemented without cuda acceleration. You can write [cuda](https://docs.nvidia.com/cuda/cuda-samples/index.html) to achieve higher inference speed. \r\n\r\n## Citation\r\n\t@article{AdderNet,\r\n\t\ttitle={AdderNet: Do We Really Need Multiplications in Deep Learning?},\r\n\t\tauthor={Chen, Hanting and Wang, Yunhe and Xu, Chunjing and Shi, Boxin and Xu, Chao and Tian, Qi and Xu, Chang},\r\n\t\tjournal={CVPR},\r\n\t\tyear={2020}\r\n\t}\r\n\r\n### Contributing\r\nWe appreciate all contributions. If you are planning to contribute back bug-fixes, please do so without any further discussion.\r\n\r\nIf you plan to contribute new features, utility functions or extensions to the core, please first open an issue and discuss the feature with us. Sending a PR without discussion might end up resulting in a rejected PR, because we might be taking the core in a different direction than you might be aware of.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhuawei-noah%2Faddernet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhuawei-noah%2Faddernet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhuawei-noah%2Faddernet/lists"}