Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/wzjialang/MFPNet
[Remote Sensing] PyTorch implementation for "Remote Sensing Change Detection Based on Multidirectional Adaptive Feature Fusion and Perceptual Similarity"
https://github.com/wzjialang/MFPNet
attention-mechanism change-detection deep-learning feature-fusion perceptual-loss remote-sensing
Last synced: 3 months ago
JSON representation
[Remote Sensing] PyTorch implementation for "Remote Sensing Change Detection Based on Multidirectional Adaptive Feature Fusion and Perceptual Similarity"
- Host: GitHub
- URL: https://github.com/wzjialang/MFPNet
- Owner: wzjialang
- License: mit
- Created: 2021-07-22T04:28:35.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2022-10-04T18:06:47.000Z (about 2 years ago)
- Last Synced: 2024-07-23T03:40:30.741Z (4 months ago)
- Topics: attention-mechanism, change-detection, deep-learning, feature-fusion, perceptual-loss, remote-sensing
- Language: Python
- Homepage:
- Size: 6.49 MB
- Stars: 32
- Watchers: 1
- Forks: 6
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-remote-sensing-change-detection - Xu J, Luo C, Chen X, et al. Remote Sensing Change Detection Based on Multidirectional Adaptive Feature Fusion and Perceptual Similarity
README
# Remote Sensing Change Detection Based on Multidirectional Adaptive Feature Fusion and Perceptual Similarity
PyTorch implementation for "[Remote Sensing Change Detection Based on Multidirectional Adaptive Feature Fusion and Perceptual Similarity](https://www.mdpi.com/2072-4292/13/15/3053)"
- [03 August 2021] Release the code of MFPNet model.
- [28 June 2022] Release the processed datasets, training/evaluation codes.
- [08 JUly 2022] Release model weights for Season-varying/LEVIR-CD/Google datasets.
## Introduction
Remote sensing change detection (RSCD) is an important yet challenging task in Earth observation. The booming development of convolutional neural networks (CNNs) in computer vision raises new possibilities for RSCD, and many recent RSCD methods have introduced CNNs to achieve promising improvements in performance. This paper proposes a novel multidirectional fusion and perception network for change detection in bi-temporal very-high-resolution remote sensing images. First, we propose an elaborate feature fusion module consisting of a multidirectional fusion pathway (MFP) and an adaptive weighted fusion (AWF) strategy for RSCD to boost the way that information propagates in the network. The MFP enhances the flexibility and diversity of information paths by creating extra top-down and shortcut-connection paths. The AWF strategy conducts weight recalibration for every fusion node to highlight salient feature maps and overcome semantic gaps between different features. Second, a novel perceptual similarity module is designed to introduce perceptual loss into the RSCD task, which adds the perceptual information, such as structure and semantic, for high-quality change maps generation. Extensive experiments on four challenging benchmark datasets demonstrate the superiority of the proposed network comparing with eight state-of-the-art methods in terms of F1, Kappa, and visual qualities.
## Content
### Architecture
Fig.1 Overall architecture of the proposed multidirectional fusion and perception network (MFPNet).
Note that the process with the dashed line only participates in model training.
### Datasets
The processed and original datasets can be downloaded from the table below, we recommended downloading the processed one directly to get a quick start on our codes:
Datasets
Processed Links
Original Links
Season-varying Dataset [1]
[Google Drive]
[Baidu Drive]
[Original]
LEVIR-CD Dataset [2]
[Original]
Google Dataset [3]
[Original]
Zhange Dataset [4]
[Original]
### Setup & Usage for the Code
1. Check the structure of data folders:
```
(root folder)
├── dataset1
| ├── train
| | ├── A
| | ├── B
| | ├── label
| ├── val
| | ├── A
| | ├── B
| | ├── label
| ├── test
| | ├── A
| | ├── B
| | ├── label
├── ...
```
2. Check dependencies:
```
- Python 3.6+
- PyTorch 1.7.0+
- scikit-learn
- cudatoolkit
- cudnn
- OpenCV-Python
```
3. Change paths:
```
- Change the 'metadata_json_path' in 'train.py' to your 'metadata.json' path.
- Change the 'dataset_dir' and 'weight_dir' in 'metadata.json' to your own path.
```
4. Train the MFPNet:
```
python train.py
```
5. Evaluate the MFPNet:
```
- Download model weights (optional).
- Change the 'weight_path' in 'eval.py' to your model weight path.
- python eval.py
```
## Model Weights
Model weights for Season-varying/LEVIR-CD/Google datasets are available via [Google Drive](https://drive.google.com/drive/folders/1-2njQ7Z3IIrjv6YGXoMD2CBZbc1nQuRu?usp=sharing) or [Baidu Drive](https://pan.baidu.com/s/141aQDQ_lMEi83O2t6AcLqg?pwd=1234). Note that the training/dataloader codes are rewritten and improved so the performance is a little different from the paper.
Datasets
F1 (%)
Kappa (%)
Season-varying
97.964
97.691
LEVIR-CD
91.568
91.120
Google
88.058
84.140
## Reference
Appreciate the work from the following repositories:
* [likyoo/Siam-NestedUNet](https://github.com/likyoo/Siam-NestedUNet)
* [zylo117/Yet-Another-EfficientDet-Pytorch](https://github.com/zylo117/Yet-Another-EfficientDet-Pytorch)
## Cite
If this repository is useful for your research, please cite:
```
@Article{rs13153053,
AUTHOR = {Xu, Jialang and Luo, Chunbo and Chen, Xinyue and Wei, Shicai and Luo, Yang},
TITLE = {Remote Sensing Change Detection Based on Multidirectional Adaptive Feature Fusion and Perceptual Similarity},
JOURNAL = {Remote Sensing},
VOLUME = {13},
YEAR = {2021},
NUMBER = {15},
ARTICLE-NUMBER = {3053},
URL = {https://www.mdpi.com/2072-4292/13/15/3053},
ISSN = {2072-4292},
DOI = {10.3390/rs13153053}
}
```