{"id":13499086,"url":"https://github.com/PistonY/ResidualAttentionNetwork","last_synced_at":"2025-03-29T04:30:42.883Z","repository":{"id":135457280,"uuid":"148084832","full_name":"PistonY/ResidualAttentionNetwork","owner":"PistonY","description":"A Gluon implement of Residual Attention Network. Best acc on cifar10-97.78%.","archived":false,"fork":false,"pushed_at":"2019-06-12T03:31:15.000Z","size":357497,"stargazers_count":106,"open_issues_count":1,"forks_count":26,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-08-01T22:50:06.848Z","etag":null,"topics":["cifar10","deep-learning","gluon","gluon-cv","mxnet","residual-attention-network","sota"],"latest_commit_sha":null,"homepage":"https://pistony.github.io/ResidualAttentionNetwork/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PistonY.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null},"funding":{"github":"PistonY","patreon":"pistony","open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"custom":null}},"created_at":"2018-09-10T01:55:50.000Z","updated_at":"2024-06-19T22:42:54.000Z","dependencies_parsed_at":null,"dependency_job_id":"286de45c-2a96-4342-ba20-a33356ac79f0","html_url":"https://github.com/PistonY/ResidualAttentionNetwork","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PistonY%2FResidualAttentionNetwork","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PistonY%2FResidualAttentionNetwork/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PistonY%2FResidualAttentionNetwork/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PistonY%2FResidualAttentionNetwork/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PistonY","download_url":"https://codeload.github.com/PistonY/ResidualAttentionNetwork/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222455956,"owners_count":16987577,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cifar10","deep-learning","gluon","gluon-cv","mxnet","residual-attention-network","sota"],"created_at":"2024-07-31T22:00:28.285Z","updated_at":"2024-10-31T17:31:37.267Z","avatar_url":"https://github.com/PistonY.png","language":"Python","funding_links":["https://github.com/sponsors/PistonY","https://patreon.com/pistony"],"categories":["Papers\u0026Codes"],"sub_categories":["Residual Attention Network"],"readme":"# Residual Attention Network \n[![GitHub](https://img.shields.io/github/license/PistonY/ResidualAttentionNetwork.svg)](./LICENSE)\n![Status](https://img.shields.io/badge/status-%E8%90%8C-orange.svg)\n[![996.icu](https://img.shields.io/badge/link-996.icu-red.svg)](https://996.icu)\n![ToDo](http://progressed.io/bar/100?title=ToDo)\n\nA Gluon implement of Residual Attention Network\n\nThis code is refered to this project\n\nhttps://github.com/tengshaofeng/ResidualAttentionNetwork-pytorch\n## Cifar-10 Kaggle\n![4](kaggle/0.9778.png)\n\n## [GluonCV](http://gluon-cv.mxnet.io)\nProject site: https://github.com/dmlc/gluon-cv\n\nI have contribute this project to GluonCV.Now you can easily use pre-trained model in few days.\n\nUsage:\n```python\nfrom gluoncv.model_zoo.residual_attentionnet import *\n```\nInclude which you can use:\n```python\n__all__ = ['ResidualAttentionModel', 'cifar_ResidualAttentionModel',\n           'residualattentionnet56', 'cifar_residualattentionnet56',\n           'residualattentionnet92', 'cifar_residualattentionnet92',\n           'residualattentionnet128', 'cifar_residualattentionnet452',\n           'residualattentionnet164', 'residualattentionnet200',\n           'residualattentionnet236', 'residualattentionnet452']\n```\n## Prerequisites\n\nPython3.6, Numpy, mxnet\n- I use maxnet-cu90 --pre but if not is just ok\n- If you want to train you need a recent NVIDIA GPU\n\n## Results\n- [x] cifar-10: Acc-95.41(**Top-1 err 4.59**) with Attention-92(higher than paper top-1 err 4.99)\n- [x] cifar-10: Acc-95.68(**Top-1 err 4.32**) with Attention-92(use MSRAPrelu init)\n- [x] cifar-10: Acc-97.14(**Top-1 err 2.86**) with Attention-92, using [gluoncv-tricks](https://arxiv.org/pdf/1812.01187.pdf).\n    - BS 256,\n    - +mixup,\n    - +LR warmup,\n    - +No bias decay.\n    - +Cosine decay.\n    - +Cutout\n- [x] cifar-10: Acc-97.57(**Top-1 err 2.43**) with Attention-452, using [gluoncv-tricks](https://arxiv.org/pdf/1812.01187.pdf).\n    - BS 128,\n    - +mixup,\n    - +LR warmup,\n    - +No bias decay.\n    - +Cosine decay.\n    - +Cutout\n- [x] Network scale control: I add 'p,t,r,m' to control network scale.(Gluon-CV)\n    - I add 'p,t,r,m.' control which origin paper proposed.Now you can use Attentnon 56/92/128/164/200/236/452 in Gluon-cv.But I\n    won't update to this project.Because I can't train them and  if I add, the paprm I have trained won't use any more.\n- [x] ImageNet: Attention56 achieves (21.03 5.47) top1/top5 error on ImageNet.Better than paper.(21.76 5.9).(Gluon-cv)\n\n\n## How to train \u0026 test\nFor training cifar10, just run train_cifar.py\n\nFor only testing cifar10, you can simply run below script.\n```python\nimport mxnet as mx\nfrom mxnet import gluon, image\nfrom train_cifar import test\nfrom model.residual_attention_network import ResidualAttentionModel_92_32input_update\n\ndef trans_test(data, label):\n    im = data.astype(np.float32) / 255.\n    auglist = image.CreateAugmenter(data_shape=(3, 32, 32),\n                                    mean=mx.nd.array([0.485, 0.456, 0.406]),\n                                    std=mx.nd.array([0.229, 0.224, 0.225]))\n    for aug in auglist:\n        im = aug(im)\n\n    im = nd.transpose(im, (2, 0, 1))\n    return im, label\n\nctx = mx.gpu()\nval_data = gluon.data.DataLoader(\n    gluon.data.vision.CIFAR10(train=False, transform=trans_test),\n    batch_size=64)\n\nnet = ResidualAttentionModel_92_32input_update()\nnet.hybridize()\nnet.load_parameters('cifar_param/test_iter225999_0.95410.param')\ntest(net, ctx, val_data, 0)\n```\n\n## Paper referenced\nResidual Attention Network for Image Classification (CVPR-2017 Spotlight) By Fei Wang, Mengqing Jiang, Chen Qian, Shuo Yang, Chen Li, Honggang Zhang, Xiaogang Wang, Xiaoou Tang(https://arxiv.org/pdf/1704.06904.pdf)\n\n![1](imgs/Figure1.png)\n**Left**: an example shows the interaction between features and attention masks. **Right**: example images illustrating that different features have different corresponding attention masks in our network. The sky mask diminishes low-level background blue color features. The balloon instance mask highlights high-level balloon bottom part features.\n\u003c/br\u003e\n\n![2](imgs/Figure2.png)\nAttention Network architecture.\n\u003c/br\u003e\n\n![3](imgs/Figure3.png)\n\nThe Attention-56 network outperforms ResNet-152 by a large margin with a 0.4% reduction on top-1 error and a 0.26% reduction on top-5 error. More importantly **Attention-56 network achieves better performance with only 52% parameters and 56% FLOPs compared with ResNet-152**, which suggests that the proposed attention mechanism can significantly improve network performance while reducing the model complexity.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPistonY%2FResidualAttentionNetwork","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPistonY%2FResidualAttentionNetwork","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPistonY%2FResidualAttentionNetwork/lists"}