Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/solcummings/earthvision2021-weakly-supervised
1st Place Solution for the EARTHVISION 2021 DynamicEarthNet Challenge - Weakly-Supervised Multi-Class Change Detection Track at CVPRW 2021
https://github.com/solcummings/earthvision2021-weakly-supervised
change-detection deep-learning pytorch remote-sensing semantic-segmentation siamese-neural-network
Last synced: about 2 months ago
JSON representation
1st Place Solution for the EARTHVISION 2021 DynamicEarthNet Challenge - Weakly-Supervised Multi-Class Change Detection Track at CVPRW 2021
- Host: GitHub
- URL: https://github.com/solcummings/earthvision2021-weakly-supervised
- Owner: solcummings
- License: apache-2.0
- Created: 2021-06-18T05:12:54.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2022-11-03T23:33:45.000Z (almost 2 years ago)
- Last Synced: 2024-07-23T04:36:00.736Z (about 2 months ago)
- Topics: change-detection, deep-learning, pytorch, remote-sensing, semantic-segmentation, siamese-neural-network
- Language: Python
- Homepage:
- Size: 6.08 MB
- Stars: 21
- Watchers: 2
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-remote-sensing-change-detection - solcummings/earthvision2021-weakly-supervised
README
# Siamese Attention U-Net for Multi-Class Change Detection
1st place solution for the EARTHVISION 2021 DynamicEarthNet Challenge - Weakly-Supervised Multi-Class Change Detection Track at CVPRW 2021
[[Challenge Site]](https://competitions.codalab.org/competitions/30441)
[[Presentation Slides]](./examples/earthvision2021_presentation.pdf)
[[Presentation Video]](https://youtu.be/nrntHeY8vtg?t=11994)
[[Dataset]](https://competitions.codalab.org/competitions/30441#participate)
## Overview
This work introduces a pixel-wise change detection network named Siamese Attention U-Net that incorporates attention mechanisms in the Siamese U-Net architecture. Experiments show the architectural change alongside training strategies such as semi-supervised learning produce more robust models.
### Results
1. Attention block
When training the model with Jaccard loss, attending the up-sampled features improves mean IoU scores whereas attending skipped features - originally proposed in Attention UNet - do not. The improvement implies a need for prioritizing information in the coarser resolution features.
|Attended features|mean IoU (val)|
|:-:|:-:|
|None|0.2635|
|Skipped features|0.2603|
|Up-sampled features|0.2658|
2. Loss function
Jaccard loss and Dice loss optimize for different metrics, producing slightly varying results. Ensembling models trained exclusively on each loss function improves scores.
|Loss function|mean IoU (val)|
|:-:|:-:|
|Jaccard|0.2658|
|Dice|0.2668|
|Ensemble|0.2676|
3. Semi-supervised learning
Creating hard pseudo labels for the public validation and test dataset, then retraining the model alongside the original training dataset improves scores regardless of loss function. The ensemble of models trained on each loss function is submitted to the public test benchmark.
|Pseudo labels|Loss function|mean IoU (val)|mean IoU (test)|
|:-:|:-:|:-:|:-:|
|None|Jaccard|0.2658||
|val+test|Jaccard|0.2669||
|None|Dice|0.2668|
|val+test|Dice|0.2674||
|None|Ensemble|0.2676||
|val+test|Ensemble|0.2684|0.2423|
## Usage
### Dependencies
- gdal
- numpy
- pandas
- pillow
- pytorch
- pyyaml
- torchvision
- tqdm
### Downloading and Preprocessing Data
```bash
# assumes all data is unzipped to ./earthvision2021/data/source/
cd ./setup/
# convert data and organize directories
bash setup.bash
```
```bash
cd ./earthvision2021/preprocessing/
# create csv file that determines crop regions for training
python crop.py
mv train.csv ../data/train/random_128_binary_attempts128.csv
```
### Training
```bash
cd ./earthvision2021/
# train model according to configurations in ./earthvision2021/config_train.yml
python train.py
```
### Predicting
```bash
cd ./earthvision2021/
# predict using model according to configurations in ./earthvision2021/config_predict.yml
python predict.py
```
### Pseudo Labelling
```bash
cd ./earthvision2021/postprocessing/
# binarizing prediction results to create hard pseudo labels
python label_binarize.py --in_dir ${prediction_dir}
cd ../preprocessing/
# create new csv file of crop regions
python crop.py --subset_file ${filename_text_file}
mv train.csv ../data/train/${csv_file}
# change "aoi_file" in ../config_train.yml to ${csv_file} and retrain
```
### Postprocessing
```bash
cd ./earthvision2021/postprocessing/
# ensemble multiple predictions results
python ensemble.py --in_dir_list ${prediction_dir} ...
# binarize prediction results
python binarize.py --in_dir ${prediction_dir}
```
## Citation
If you find this work useful in your research or publication, please cite this work:
```TeX
@inproceedings{cummings2022_sia,
author={Cummings, Sol and Kondmann, Lukas and Zhu, Xiao Xiang},
booktitle={IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium},
title={Siamese Attention U-Net for Multi-Class Change Detection},
year={2022},
pages={211-214},
doi={10.1109/IGARSS46834.2022.9884834}
}
```