{"id":13738543,"url":"https://github.com/Alibaba-MIIL/TResNet","last_synced_at":"2025-05-08T16:34:34.286Z","repository":{"id":112736197,"uuid":"250605790","full_name":"Alibaba-MIIL/TResNet","owner":"Alibaba-MIIL","description":"Official Pytorch Implementation of \"TResNet: High-Performance GPU-Dedicated Architecture\" (WACV 2021)","archived":false,"fork":false,"pushed_at":"2023-01-11T07:05:52.000Z","size":1066,"stargazers_count":462,"open_issues_count":2,"forks_count":63,"subscribers_count":19,"default_branch":"master","last_synced_at":"2024-05-23T10:22:42.558Z","etag":null,"topics":["accuracy","architecture","imagenet","multi-label-classification","tresnet"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Alibaba-MIIL.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-03-27T17:53:52.000Z","updated_at":"2024-05-21T09:24:28.000Z","dependencies_parsed_at":"2023-09-13T12:46:23.771Z","dependency_job_id":null,"html_url":"https://github.com/Alibaba-MIIL/TResNet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alibaba-MIIL%2FTResNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alibaba-MIIL%2FTResNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alibaba-MIIL%2FTResNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alibaba-MIIL%2FTResNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Alibaba-MIIL","download_url":"https://codeload.github.com/Alibaba-MIIL/TResNet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253105660,"owners_count":21855072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accuracy","architecture","imagenet","multi-label-classification","tresnet"],"created_at":"2024-08-03T03:02:26.148Z","updated_at":"2025-05-08T16:34:33.945Z","avatar_url":"https://github.com/Alibaba-MIIL.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# TResNet: High Performance GPU-Dedicated Architecture\n\n[paperV2](https://arxiv.org/pdf/2003.13630.pdf) |\n[pretrained models](MODEL_ZOO.md)\n\nOfficial PyTorch Implementation\n\n\u003e Tal Ridnik, Hussam Lawen, Asaf Noy, Itamar Friedman, Emanuel Ben Baruch, Gilad Sharir\u003cbr/\u003e\n\u003e DAMO Academy, Alibaba Group\n\n\n\n**Abstract**\n\n\u003e Many deep learning models, developed in recent years, reach higher\n\u003e ImageNet accuracy than ResNet50, with fewer or comparable FLOPS count.\n\u003e While FLOPs are often seen as a proxy for network efficiency, when\n\u003e measuring actual GPU training and inference throughput, vanilla\n\u003e ResNet50 is usually significantly faster than its recent competitors,\n\u003e offering better throughput-accuracy trade-off. In this work, we\n\u003e introduce a series of architecture modifications that aim to boost\n\u003e neural networks' accuracy, while retaining their GPU training and\n\u003e inference efficiency. We first demonstrate and discuss the bottlenecks\n\u003e induced by FLOPs-optimizations. We then suggest alternative designs\n\u003e that better utilize GPU structure and assets. Finally, we introduce a\n\u003e new family of GPU-dedicated models, called TResNet, which achieve\n\u003e better accuracy and efficiency than previous ConvNets. Using a TResNet\n\u003e model, with similar GPU throughput to ResNet50, we reach 80.7\\%\n\u003e top-1 accuracy on ImageNet. Our TResNet models also transfer well and\n\u003e achieve state-of-the-art accuracy on competitive datasets such as\n\u003e Stanford cars (96.0\\%), CIFAR-10 (99.0\\%), CIFAR-100 (91.5\\%) and\n\u003e Oxford-Flowers (99.1\\%). They also perform well on multi-label classification and object detection tasks.\n\n## 29/11/2021 Update - New article released, offering new classification head with state-of-the-art results\nCheckout our new project, [Ml-Decoder](https://github.com/Alibaba-MIIL/ML_Decoder), which presents a unified classification head for multi-label, single-label and\nzero-shot tasks. Backbones with ML-Decoder reach SOTA results, while also improving speed-accuracy tradeoff.\n\n\u003cp align=\"center\"\u003e\n \u003ctable class=\"tg\"\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e\u003cimg src=\"./figures/main_pic.png\" align=\"center\" width=\"300\"\"\u003e\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e\u003cimg src=\"./figures/ms_coco_scores.png\" align=\"center\" width=\"300\" \u003e\u003c/td\u003e\n\n  \u003c/tr\u003e\n\u003c/table\u003e\n\u003c/p\u003e\n\n## 11/1/2023 Update\nAdded [tests](https://github.com/Alibaba-MIIL/TResNet/blob/master/tests/test_TResNetV2) auto-generated by [CodiumAI](https://www.codium.ai/) tool\n\n\n## 23/4/2021 Update - ImageNet21K Pretraining\nIn a new [article](https://github.com/Alibaba-MIIL/ImageNet21K) we released, we share pretrain weights for TResNet models from ImageNet21K training, that dramatically outperfrom standard pretraining.\nTResNet-M model, for example, improves its ImageNet-1K score, from 80.7% to 83.1% !\nThis kind of improvement is consistently achieved on all downstream tasks.\n\n\n## 28/8/2020: V2 of TResNet Article Released\n\n## Sotabench Comparisons\nComparative results from\n[sotabench benchamrk](https://sotabench.com/benchmarks/image-classification-on-imagenet#code),\ndemonstartaing that TReNset models give excellent speed-accuracy tradoff:\n\u003cp align=\"center\"\u003e\n \u003ctable class=\"tg\"\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e\u003cimg src=\"./figures/sotabench.png\" align=\"center\" width=\"700\" \u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\u003c/p\u003e\n\n## 11/6/2020: V1 of TResNet Article Released\nThe main change - In addition to single label SOTA results, we also\nadded top results for multi-label classification and object detection\ntasks, using TResNet. For example, we set a new SOTA record for MS-COCO\nmulti-label dataset, surpassing the previous top results by more than\n2.5% mAP !\n\u003cp align=\"center\"\u003e\n \u003ctable\u003e\n  \u003ctr align=\"center\"\u003e\n    \u003cth align=\"center\"\u003eBacbkone\u003c/th\u003e\n    \u003cth\u003emAP\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd align=\"center\"\u003e \u003ca href=\"https://arxiv.org/pdf/1911.09243.pdf\"\u003e KSSNet\u003c/a\u003e (previous SOTA)\u003c/td\u003e\n    \u003ctd  align=\"center\"\u003e83.7\u003c/td\u003e\n  \u003c/tr\u003e\n    \u003ctr\u003e\n    \u003ctd  align=\"center\"\u003e\u003ca href=\"https://arxiv.org/pdf/2003.13630.pdf\"\u003eTResNet-L\u003c/a\u003e\u003c/td\u003e\n    \u003ctd  align=\"center\"\u003e\u003cb\u003e86.4\u003c/b\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n  \n\u003c/table\u003e\n\u003c/p\u003e\n\n\n## 2/6/2020: CVPR-Kaggle competitions\nWe participated and won top places in two\nmajor CVPR-Kaggle competitions:\n* [2nd place](https://www.kaggle.com/c/herbarium-2020-fgvc7/discussion/154186)\n  in Herbarium 2020 competition, out of 153 teams.\n* [7th place](https://www.kaggle.com/c/plant-pathology-2020-fgvc7/discussion/154086)\n  in Plant-Pathology 2020 competition, out of 1317 teams.   \n  \u003cbr\u003e *TResNet* was a vital part of our solution for both competitions,\n  allowing us to work on high resolutions and reach top scores while\n  doing fast and efficient experiments.\n  \u003ccenter\u003e\n   \u003ctable class=\"tg\"\u003e\n        \u003cthead\u003e\n      \u003ctd class=\"tg-c3ow\"\u003e\u003cimg src=\"./figures/herbarium_2020.png\" align=\"center\" width=\"300\" \u003e\u003c/td\u003e\n    \u003c/thead\u003e\n    \u003c/table\u003e\n  \u003c/center\u003e\n\n\n## Main Article Results\n#### TResNet Models\nTResNet models accuracy and GPU throughput on ImageNet, compared to ResNet50. All measurements were done on Nvidia V100 GPU, with mixed precision. All models are trained on input resolution of 224.\n\u003cp align=\"center\"\u003e\n \u003ctable\u003e\n  \u003ctr\u003e\n    \u003cth\u003eModels\u003c/th\u003e\n    \u003cth\u003eTop Training Speed \u003cbr\u003e(img/sec)\u003c/th\u003e\n    \u003cth\u003eTop Inference Speed\u003cbr\u003e(img/sec)\u003c/th\u003e\n    \u003cth\u003eMax Train Batch Size\u003c/th\u003e\n    \u003cth\u003eTop-1 Acc.\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eResNet50\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e805\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e2830\u003c/td\u003e\n    \u003ctd\u003e288\u003c/td\u003e\n    \u003ctd\u003e79.0\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eEfficientNetB1\u003c/td\u003e\n    \u003ctd\u003e440\u003c/td\u003e\n    \u003ctd\u003e2740\u003c/td\u003e\n    \u003ctd\u003e196\u003c/td\u003e\n    \u003ctd\u003e79.2\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eTResNet-M\u003c/td\u003e\n    \u003ctd\u003e730\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e2930\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e512\u003c/b\u003e\u003c/td\u003e\n    \u003ctd\u003e80.8\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eTResNet-L\u003c/td\u003e\n    \u003ctd\u003e345\u003c/td\u003e\n    \u003ctd\u003e1390\u003c/td\u003e\n    \u003ctd\u003e316\u003c/td\u003e\n    \u003ctd\u003e81.5\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd\u003eTResNet-XL\u003c/td\u003e\n    \u003ctd\u003e250\u003c/td\u003e\n    \u003ctd\u003e1060\u003c/td\u003e\n    \u003ctd\u003e240\u003c/td\u003e\n    \u003ctd\u003e\u003cb\u003e82.0\u003c/b\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\u003c/p\u003e\n\n#### Comparison To Other Networks\n\nComparison of ResNet50 to top modern networks, with similar top-1 ImageNet accuracy.\n All measurements were done on Nvidia V100 GPU with mixed precision. For gaining optimal speeds, training and inference were measured on 90\\% of maximal possible batch size.\n Except TResNet-M, all the models' ImageNet scores were taken from the [public repository](https://github.com/rwightman/pytorch-image-models), which specialized in providing top implementations for modern networks. Except EfficientNet-B1, which has input resolution of 240, all other models have input resolution of 224.\n\u003cp align=\"center\"\u003e\n\u003ctable class=\"tg\"\u003e\n  \u003ctr\u003e\n    \u003cth class=\"tg-c3ow\"\u003eModel\u003c/th\u003e\n    \u003cth class=\"tg-c3ow\"\u003eTop Training Speed\u003cbr\u003e(img/sec)\u003c/th\u003e\n    \u003cth class=\"tg-c3ow\"\u003eTop Inference Speed\u003cbr\u003e(img/sec)\u003c/th\u003e\n    \u003cth class=\"tg-c3ow\"\u003eTop-1 Acc.\u003c/th\u003e\n    \u003cth class=\"tg-c3ow\"\u003eFlops[G]\u003c/th\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-0pky\"\u003eResNet50\u003c/td\u003e\n   \u003ctd class=\"tg-c3ow\"\u003e\u003cb\u003e805\u003c/b\u003e\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e2830\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e79.0\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e4.1\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-0pky\"\u003eResNet50-D\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e600\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e2670\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e79.3\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e4.4\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-0pky\"\u003eResNeXt50\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e490\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e1940\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e79.4\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e4.3\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-0pky\"\u003eEfficientNetB1\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e440\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e2740\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e79.2\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e0.6\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-0pky\"\u003eSEResNeXt50\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e400\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e1770\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e79.9\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e4.3\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-0pky\"\u003eMixNet-L\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e400\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e1400\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e79.0\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e0.5\u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-0pky\"\u003eTResNet-M\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e730\u003c/td\u003e\n   \u003ctd class=\"tg-c3ow\"\u003e\u003cb\u003e2930\u003c/b\u003e\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e\u003cb\u003e80.8\u003c/b\u003e\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e5.5\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\u003c/p\u003e\n\n \u003cbr/\u003e\n\u003cp align=\"center\"\u003e\n \u003ctable class=\"tg\"\u003e\n  \u003ctr\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e\u003cimg src=\"./figures/table_4.png\" align=\"center\" width=\"400\" \u003e\u003c/td\u003e\n    \u003ctd class=\"tg-c3ow\"\u003e\u003cimg src=\"./figures/table_5.png\" align=\"center\" width=\"400\" \u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\u003c/p\u003e\n\n \n\u003c/p\u003e\n\n#### Transfer Learning SotA Results\nComparison of TResNet to state-of-the-art models on transfer learning datasets (only ImageNet-based transfer learning results). Models inference speed is measured on a mixed precision V100 GPU. Since no official implementation of  Gpipe was provided, its inference speed is unknown\n\n\u003cp align=\"center\"\u003e\n \u003ctable style=\"border-collapse: collapse; border: none; border-spacing: 0px;\"\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd style=\"border-width: 1px; border-style: solid; border-color: black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tDataset\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-top: 1px solid black; border-bottom: 1px solid black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tModel\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-top: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tTop-1\n\t\t\t\u003cbr\u003e\n\t\t\tAcc.\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-top: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tSpeed\n\t\t\t\u003cbr\u003e\n\t\t\timg/sec\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-top: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tInput\n\t\t\u003c/td\u003e\n\t\u003c/tr\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd rowspan=\"2\" style=\"border-left: 1px solid black; border-right: 1px solid black; border-bottom: 2px double black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tCIFAR-10\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tGpipe\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t\u003cb\u003e99.0\u003c/b\u003e\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t-\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t480\n\t\t\u003c/td\u003e\n\t\u003c/tr\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tTResNet-XL\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t\u003cb\u003e99.0\u003c/b\u003e\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t\u003cb\u003e1060\u003c/b\u003e\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t224\n\t\t\u003c/td\u003e\n\t\u003c/tr\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd rowspan=\"2\" style=\"border-left: 1px solid black; border-right: 1px solid black; border-bottom: 2px double black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tCIFAR-100\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tEfficientNet-B7\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t\u003cb\u003e91.7\u003c/b\u003e\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t70\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t600\n\t\t\u003c/td\u003e\n\t\u003c/tr\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tTResNet-XL\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t91.5\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t\u003cb\u003e1060\u003c/b\u003e\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t224\n\t\t\u003c/td\u003e\n\t\u003c/tr\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd rowspan=\"2\" style=\"border-left: 1px solid black; border-right: 1px solid black; border-bottom: 2px double black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t Stanford Cars\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tEfficientNet-B7\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t94.7\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t70\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t600\n\t\t\u003c/td\u003e\n\t\u003c/tr\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tTResNet-L\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t\u003cb\u003e96.0\u003c/b\u003e\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t\u003cb\u003e500\u003c/b\u003e\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 2px double black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t368\n\t\t\u003c/td\u003e\n\t\u003c/tr\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd rowspan=\"2\" style=\"border-left: 1px solid black; border-right: 1px solid black; border-bottom: 1px solid black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t Oxford-Flowers\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tEfficientNet-B7\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t98.8\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t70\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t600\n\t\t\u003c/td\u003e\n\t\u003c/tr\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\tTResNet-L\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t\u003cb\u003e99.1\u003c/b\u003e\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t\u003cb\u003e500\u003c/b\u003e\n\t\t\u003c/td\u003e\n\t\t\u003ctd style=\"border-right: 1px solid black; border-bottom: 1px solid black; text-align: center; padding-right: 3pt; padding-left: 3pt;\"\u003e\n\t\t\t368\n\t\t\u003c/td\u003e\n\t\u003c/tr\u003e\n\u003c/table\u003e\n\u003c/p\u003e\n\n\n## Reproduce Article Scores\nWe provide code for reproducing the validation top-1 score of TResNet\nmodels on ImageNet. First, download pretrained models from\n[here](MODEL_ZOO.md).\n\nThen, run the infer.py script. For example, for tresnet_m (input size 224)\nrun:\n```bash\npython -m infer.py \\\n--val_dir=/path/to/imagenet_val_folder \\\n--model_path=/model/path/to/tresnet_m.pth \\\n--model_name=tresnet_m\n--input_size=224\n```\n## TResNet Training\nDue to IP limitations, we do not provide the exact training code that\nwas used to obtain the article results.\n\nHowever, TResNet is now an integral part of the popular [rwightman /\npytorch-image-models](https://github.com/rwightman/pytorch-image-models)\nrepo. Using that repo, you can reach very similar results to the one\nstated in the article. \n\nFor example, training tresnet_m on [rwightman /\npytorch-image-models](https://github.com/rwightman/pytorch-image-models) with\nthe command line:\n```bash\npython -u -m torch.distributed.launch --nproc_per_node=8 \\\n--nnodes=1 --node_rank=0 ./train.py /data/imagenet/ \\\n-b=190 --lr=0.6 --model-ema --aa=rand-m9-mstd0.5-inc1 \\\n--num-gpu=8 -j=16 --amp \\\n--model=tresnet_m --epochs=300 --mixup=0.2 \\\n--sched='cosine' --reprob=0.4 --remode=pixel\n```\ngave accuracy of 80.5%. \u003cbr\u003e\u003cbr\u003e\n\n\nAlso, during the merge request, we had interesting discussions and\ninsights regarding TResNet design. I am attaching a pdf version the\nmentioned discussions. They can shed more light on TResNet design\nconsiderations and directions for the future.\n\n[TResNet discussion and insights](https://miil-public-eu.oss-eu-central-1.aliyuncs.com/model-zoo/tresnet/TResnet_discussion.pdf)\n\n(taken with permission from\n[here](https://github.com/rwightman/pytorch-image-models/issues/124))\n\n\n\n## Tips For Working With Inplace-ABN\nSee\n[INPLACE_ABN_TIPS](https://github.com/mrT23/TResNet/blob/master/INPLACE_ABN_TIPS.md).\n\n\n## Citation\n\n```\n@misc{ridnik2020tresnet,\n    title={TResNet: High Performance GPU-Dedicated Architecture},\n    author={Tal Ridnik and Hussam Lawen and Asaf Noy and Itamar Friedman},\n    year={2020},\n    eprint={2003.13630},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV}\n}\n```\n\n## Contact\nFeel free to contact me if there are any questions or issues (Tal\nRidnik, tal.ridnik@alibaba-inc.com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlibaba-MIIL%2FTResNet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAlibaba-MIIL%2FTResNet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlibaba-MIIL%2FTResNet/lists"}