{"id":20305521,"url":"https://github.com/vitae-transformer/rsp","last_synced_at":"2025-04-05T09:05:15.920Z","repository":{"id":161363758,"uuid":"621321401","full_name":"ViTAE-Transformer/RSP","owner":"ViTAE-Transformer","description":"The official repo for [TGRS'22] \"An Empirical Study of Remote Sensing Pretraining\"","archived":false,"fork":false,"pushed_at":"2024-11-07T02:08:39.000Z","size":17257,"stargazers_count":142,"open_issues_count":14,"forks_count":8,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-29T08:07:03.083Z","etag":null,"topics":["change-detection","classification","deep-learning","foundation-models","imagenet","object-detection","pre-training","remote-sensing","semantic-segmentation","transfer-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ViTAE-Transformer.png","metadata":{"files":{"readme":"README.md","changelog":"Change Detection/LICENSE","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-30T12:36:13.000Z","updated_at":"2025-03-25T05:47:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"0f846fcf-c5aa-4d33-9332-a61de06ba37c","html_url":"https://github.com/ViTAE-Transformer/RSP","commit_stats":{"total_commits":77,"total_committers":3,"mean_commits":"25.666666666666668","dds":"0.038961038961038974","last_synced_commit":"23cbf5123aef33a1d9f269f5199099fc86668bae"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ViTAE-Transformer%2FRSP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ViTAE-Transformer%2FRSP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ViTAE-Transformer%2FRSP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ViTAE-Transformer%2FRSP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ViTAE-Transformer","download_url":"https://codeload.github.com/ViTAE-Transformer/RSP/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247312077,"owners_count":20918344,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["change-detection","classification","deep-learning","foundation-models","imagenet","object-detection","pre-training","remote-sensing","semantic-segmentation","transfer-learning"],"created_at":"2024-11-14T17:08:48.604Z","updated_at":"2025-04-05T09:05:15.882Z","avatar_url":"https://github.com/ViTAE-Transformer.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"left\"\u003e\n\u003ca href=\"https://paperswithcode.com/sota/aerial-scene-classification-on-ucm-80-as?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/aerial-scene-classification-on-ucm-80-as\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/aerial-scene-classification-on-aid-20-as?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/aerial-scene-classification-on-aid-20-as\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/aerial-scene-classification-on-aid-50-as?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/aerial-scene-classification-on-aid-50-as\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/aerial-scene-classification-on-nwpu-10-as?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/aerial-scene-classification-on-nwpu-10-as\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/aerial-scene-classification-on-nwpu-20-as?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/aerial-scene-classification-on-nwpu-20-as\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/semantic-segmentation-on-isprs-potsdam?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/semantic-segmentation-on-isprs-potsdam\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/semantic-segmentation-on-isaid?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/semantic-segmentation-on-isaid\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/object-detection-in-aerial-images-on-dota-1?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/object-detection-in-aerial-images-on-dota-1\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/object-detection-in-aerial-images-on-hrsc2016?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/object-detection-in-aerial-images-on-hrsc2016\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/change-detection-for-remote-sensing-images-on?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/change-detection-for-remote-sensing-images-on\"\u003e\u003c/a\u003e\n\u003ca href=\"https://paperswithcode.com/sota/building-change-detection-for-remote-sensing?p=an-empirical-study-of-remote-sensing\"\u003e\u003cimg src=\"https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/an-empirical-study-of-remote-sensing/building-change-detection-for-remote-sensing\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003e An Empirical Study of Remote Sensing Pretraining \u003c/h1\u003e \n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://arxiv.org/abs/2204.02825\"\u003e\u003cimg src=\"https://img.shields.io/badge/arXiv-Paper-\u003ccolor\u003e\"\u003e\u003c/a\u003e\n\u003ca href=\"https://ieeexplore.ieee.org/document/9782149\"\u003e\u003cimg src=\"https://img.shields.io/badge/TGRS-Paper-blue\"\u003e\u003c/a\u003e\n\u003ca href=\"https://www.bilibili.com/video/BV1y5411979L?spm_id_from=333.999.0.0\"\u003e\u003cimg src=\"https://img.shields.io/badge/bilibili-Talk-ff69b4\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003ch5 align=\"center\"\u003e\u003cem\u003eDi Wang, Jing Zhang, Bo Du, Gui-Song Xia and Dacheng Tao\u003c/em\u003e\u003c/h5\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#updates\"\u003eUpdates\u003c/a\u003e |\n  \u003ca href=\"#introduction\"\u003eIntroduction\u003c/a\u003e |\n  \u003ca href=\"#usage\"\u003eUsage\u003c/a\u003e |\n  \u003ca href=\"#results-and-models\"\u003eResults \u0026 Models\u003c/a\u003e |\n  \u003ca href=\"#talk\"\u003eTalk\u003c/a\u003e |\n  \u003ca href=\"#statement\"\u003eStatement\u003c/a\u003e |\n\u003c/p \u003e\n\n## Current applications\n\n\u003e **Scene Recognition: Please see [Remote Sensing Pretraining for Scene Recognition](https://github.com/ViTAE-Transformer/RSP/tree/main/Scene%20Recognition)**;\n\n\u003e **Sementic Segmentation: Please see [Remote Sensing Pretraining for Semantic Segmentation](https://github.com/ViTAE-Transformer/RSP/tree/main/Semantic%20Segmentation)**;\n\n\u003e **Object Detection: Please see [Remote Sensing Pretraining for Object Detection](https://github.com/ViTAE-Transformer/RSP/tree/main/Object%20Detection)**;\n\n\u003e **Change Detection: Please see [Remote Sensing Pretraining for Change Detection](https://github.com/ViTAE-Transformer/RSP/tree/main/Change%20Detection)**;\n\n\u003e **ViTAE: Please see [ViTAE-Transformer](https://github.com/ViTAE-Transformer/ViTAE-Transformer)**;\n\n\u003e **Matting: Please see [ViTAE-Transformer for matting](https://github.com/ViTAE-Transformer/ViTAE-Transformer-Matting)**;\n\n## Updates\n\n***018/10/2023***\n\nRSP won the highly cited paper!\n\n\u003cfigure\u003e\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=highlycited.png width=\"100%\"\u003e\n\u003c/div\u003e\n\u003c/figure\u003e\n\n\n***027/05/2022***\n\nThe early access is available!\n  \n***020/05/2022***\n\nThe paper has been accepted by IEEE TGRS.\n\n***011/04/2022***\n\nThe baiduyun links of pretrained models are provided.\n\n***07/04/2022***\n\nThe paper is post on arxiv!\n\n***06/04/2022***\n\nThe pretrained models for ResNet-50, Swin-T and ViTAEv2-S are released. The code for pretraining and downstream tasks are also provided for reference.\n\n## Introduction\n\nThis repository contains codes, models and test results for the paper \"[An Empirical Study of Remote Sensing Pretraining](http://arxiv.org/abs/2204.02825)\". \n\nThe aerial images are usually obtained by a camera in a birdview perspective lying on the planes or satellites, perceiving a large scope of land uses and land covers, whose scene is usually difficult to be interpreted since the interference of the scene-irrelevant regions and the complicated spatial distribution of land objects. Although deep learning has largely reshaped remote sensing research for aerial image understanding and made a great success. However, most of existing deep models are initialized with ImageNet pretrained weights, where the natural images inevitably presents a large domain gap relative to the aerial images, probably limiting the finetuning performance on downstream aerial scene tasks. This issue motivates us to conduct an empirical study of remote sensing pretraining. To this end, we train different networks from scratch with the help of the largest remote sensing scene recognition dataset up to now-MillionAID, to obtain the remote sensing pretrained backbones, including both convolutional neural networks (CNN) and vision transformers such as Swin and [ViTAE](https://arxiv.org./abs/2202.10108), which have shown promising performance on computer vision tasks. Then, we investigate the impact of ImageNet pretraining (IMP) and RSP on a series of downstream tasks including scene recognition, semantic segmentation, object detection, and change detection using the CNN and vision transformers backbones. \n\n\n\u003cfigure\u003e\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=Figs/aerialscene.png width=\"70%\"\u003e\n\u003c/div\u003e\n\u003cfigcaption align = \"center\"\u003e\u003cb\u003eFig. - (a) and (b) are the natural image and aerial image belonging to the \"park\" category. (c) and (d) are two aerial images from the \"school\" category. Despite the distinct view difference of (a) and (b), (b) contains the playground that is unusual in the park scenes but usually exists in the school scenes like (d). On the other hand, (c) and (d) show different colors as well as significantly different spatial distributions of land objects like playground and swimming pool. \u003c/b\u003e\u003c/figcaption\u003e\n\u003c/figure\u003e\n\n## Results and Models\n\n### MillionAID\n*NOTE: please extract model weight from these checkpoints for finetuning.*\n|Backbone | Input size | Acc@1 | Acc@5 | Param(M) | Pretrained model|\n|-------- | ---------- | ----- | ----- | -------- | ----------|\nRSP-ResNet-50-E300 | 224 × 224 | 98.99 | 99.82| 23.6 | [google](https://drive.google.com/file/d/1K3P4_fDfcBRGqpKoSdSa6OXS4xC1xLC9/view?usp=sharing) \u0026 [baidu](https://pan.baidu.com/s/1q7VI1wj2Vp0N5jvaXZOuCA?pwd=5npb)| [//]: \u0026 [Log](https://drive.google.com/file/d/1y1GtAs2vbLHwrIK_dVvXyGXrArhiVsIi/view?usp=sharing)|\nRSP-Swin-T-E300 | 224 × 224 | 98.59 | 99.88 | 27.6| [google](https://drive.google.com/file/d/1G5wjbjIHepmT6VVOuW03bWmyvrhcfe1F/view?usp=sharing) \u0026 [baidu](https://pan.baidu.com/s/1Hs94fI7mF12CX-Sf69gumA?pwd=7579) | [//]: \u0026 [Log](https://drive.google.com/file/d/1Ld7qxWgwOrjba08F3DxrPjfJi_wk9biR/view?usp=sharing)|\nRSP-ViTAEv2-S-E100 | 224 × 224 | 98.97 |  99.88 |18.8 | [google](https://drive.google.com/file/d/1cDB69frN-NxCyoy8lghjx6NiH1JriYUc/view?usp=sharing) \u0026 [baidu](https://pan.baidu.com/s/1riMV07wtMpmrJzEtr_T-Ag?pwd=hm8p) | [//]: \u0026 [Log](https://drive.google.com/file/d/1KEgMqSlu0-q9ZodKAH92qTvCvmcjWhKn/view?usp=sharing)|\n\n## Usage\n\nPlease refer to [Readme.md](https://github.com/ViTAE-Transformer/RSP/blob/main/Scene%20Recognition/README.md) for installation, dataset preparation, training and inference.\n\n## Citation\n\nIf this repo is useful for your research, please consider citation\n\n```\n@ARTICLE{rsp,\n  author={Wang, Di and Zhang, Jing and Du, Bo and Xia, Gui-Song and Tao, Dacheng},\n  journal={IEEE Transactions on Geoscience and Remote Sensing}, \n  title={An Empirical Study of Remote Sensing Pretraining}, \n  year={2023},\n  volume={61},\n  number={},\n  pages={1-20},\n  doi={10.1109/TGRS.2022.3176603}\n}\n```\n\n## Talk\n\nA video talk about this study (In Chinese)\n\n[![](https://github.com/ViTAE-Transformer/RSP/blob/main/Figs/video.png)](https://player.bilibili.com/player.html?aid=469101624\u0026bvid=BV1y5411979L\u0026cid=718511024\u0026page=1)\n\n## Statement\n\nThis project is under MIT licence. For any other questions please contact [di.wang at gmail.com](mailto:wd74108520@gmail.com) or [di_wang at whu.edu.cn](mailto:di_wang@whu.edu.cn).\n\n## Relevant Projects\n\n[1] \u003cstrong\u003eAdvancing Plain Vision Transformer Towards Remote Sensing Foundation Model, IEEE TGRS, 2022\u003c/strong\u003e | [Paper](https://ieeexplore.ieee.org/document/9956816/) | [Github](https://github.com/ViTAE-Transformer/Remote-Sensing-RVSA)\n\u003cbr\u003e\u003cem\u003e\u0026ensp; \u0026ensp; \u0026ensp;Di Wang\u003csup\u003e\u0026#8727;\u003c/sup\u003e, Qiming Zhang\u003csup\u003e\u0026#8727;\u003c/sup\u003e, Yufei Xu\u003csup\u003e\u0026#8727;\u003c/sup\u003e, Jing Zhang, Bo Du, Dacheng Tao and Liangpei Zhang\u003c/em\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvitae-transformer%2Frsp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvitae-transformer%2Frsp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvitae-transformer%2Frsp/lists"}