Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/MinZHANG-WHU/Change-Detection-Review
A review of change detection methods, including codes and open data sets for deep learning. From paper: change detection based on artificial intelligence: state-of-the-art and challenges.
https://github.com/MinZHANG-WHU/Change-Detection-Review
artificial-intelligence caffe change-detection code deep-learning keras machine-learning matlab open-datasets python pytorch remote-sensing review sar streetview tensorflow unsupervised-learning
Last synced: 8 days ago
JSON representation
A review of change detection methods, including codes and open data sets for deep learning. From paper: change detection based on artificial intelligence: state-of-the-art and challenges.
- Host: GitHub
- URL: https://github.com/MinZHANG-WHU/Change-Detection-Review
- Owner: MinZHANG-WHU
- Created: 2020-05-26T04:54:37.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2021-07-30T13:43:15.000Z (over 3 years ago)
- Last Synced: 2024-10-25T04:24:31.277Z (16 days ago)
- Topics: artificial-intelligence, caffe, change-detection, code, deep-learning, keras, machine-learning, matlab, open-datasets, python, pytorch, remote-sensing, review, sar, streetview, tensorflow, unsupervised-learning
- Homepage: https://github.com/MinZHANG-WHU/Change-Detection-Review
- Size: 1.19 MB
- Stars: 831
- Watchers: 29
- Forks: 183
- Open Issues: 2
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
- awesome-remote-sensing-change-detection - MinZHANG-WHU/Change-Detection-Review
README
# Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges
## 1. Introduction
Change detection based on remote sensing (RS) data is an important method of detecting changes on the Earth’s surface and has a wide range of applications in urban planning, environmental monitoring, agriculture investigation, disaster assessment, and map revision. In recent years, integrated artificial intelligence (AI) technology has become a research focus in developing new change detection methods. Although some researchers claim that AI-based change detection approaches outperform traditional change detection approaches, it is not immediately obvious how and to what extent AI can improve the performance of change detection. This review focuses on the state-of-the-art methods, applications, and challenges of AI for change detection. Specifically, the implementation process of AI-based change detection is first introduced. Then, the data from different sensors used for change detection, including optical RS data, synthetic aperture radar (SAR) data, street view images, and combined heterogeneous data, are presented, and the available open datasets are also listed. The general frameworks of AI-based change detection methods are reviewed and analyzed systematically, and the unsupervised schemes used in AI-based change detection are further analyzed. Subsequently, the commonly used networks in AI for change detection are described. From a practical point of view, the application domains of AI-based change detection methods are classified based on their applicability. Finally, the major challenges and prospects of AI for change detection are discussed and delineated, including (a) heterogeneous big data processing, (b) unsupervised AI, and (c) the reliability of AI. This review will be beneficial for researchers in understanding this field.
Figure 1. General schematic diagram of change detection.
## 2. Implementation process
Figure 2 provide a general implementation process of AI-based change detection, but the structure of the AI model is diverse and needs to be well designed according to different application situations and the training data. It is worth mentioning that existing mature frameworks such as TensorFlow, Keras, Pytorch, and Caffe, help researchers more easily realize the design, training, and deployment of AI models, and their development documents provide detailed introductions.
Figure 2. Implementation process of AI-based change detection (black arrows indicate workflow and red arrow indicates an example).
### 2.1 Available codes for AI-based methods
Table 1. A list of available codes for AI-based change detection methods.
Methods
Keywords
Publication
(Re-)Implementation
SRCDNet
CNN; Siamese; Attention; Super-resolution; Optical RS
Super-resolution-based change detection network with stacked attention module for images with different resolutions, TGRS, 2021. [paper], [code, dataset]
Pytorch 1.2
ESCNet
CNN; Siamese; Superpixel; Optical RS
An End-to-End superpixel-enhanced change detection network for Very-High-Resolution remote sensing images. TNNLS, 2021. [paper], [code]
Pytorch 1.3
KPCAMNet
CNN; Siamese; KPCA; Unsupervised; Optical RS
Unsupervised change detection in multitemporal VHR images based on deep kernel PCA convolutional mapping network,TCYB, 2021. [paper], [code]
Python
SeCo
CNN (ResNet); Transfer Learning; Optical RS
Seasonal contrast: unsupervised pre-training from uncurated remote sensing data, arXiv, 2021. [paper], [code, dataset]
Pytorch 1.7
CapsNet
Capsule Network(SegCaps); CVA; Siamese; Optical RS
Pseudo-siamese capsule network for aerial remote sensing images change detection, GRSL, 2020. [paper 1], Change Capsule Network for Optical Remote Sensing ImageChange Detection, RS, 2021. [paper 2], [code, dataset]
Keras
BIT_CD
CNN (ResNet18); Siamese; Attention; Transformer; Optical RS
Remote sensing image change detection with transformers, TGRS, 2021. [paper], [code, dataset, pre-trained model]
Pytorch 1.6
IAug_CDNet
CNN (GauGAN+UNet); Siamese; GAN; Supervised; Optical RS
Adversarial instance augmentation for building change detection in remote sensing images, TGRS, 2021. [paper], [code, dataset]
Pytorch
DDNet
CNN; DI+FCM; Unsupervised; SAR
Change detection in synthetic aperture radar images using a dual-domain network, GRSL, 2021. [paper], [code, dataset]
Pytorch
SNUNet-CD
CNN (NestedUNet); Siamese; Attention; Supervised; Optical RS
SNUNet-CD: A densely connected siamese network for change detection of VHR images, GRSL, 2021. [paper], [code, dataset, pre-trained model]
Pytorch 1.4
DSMSCN
CNN; Siamese; Multi-scale; Unsupervised/Supervised; Optical RS
A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sening images, arXiv, 2020. [paper], [code, dataset]
Tensorflow 1.9
SiamCRNN
CNN+RNN; Siamese; Multi-source; Optical RS
Change Detection in Multisource VHR Images via Deep Siamese Convolutional Multiple-Layers Recurrent Neural Network, TGRS, 2020. [paper], [code, dataset]
Tensorflow 1.9
DSIFN
CNN; Attention Mechanism; Optical RS
A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sening images, ISPRS, 2020. [paper], [code, dataset]
Pytorch & Keras
CEECNet
CNN; Attention Mechanism; Similarity Measure; Optical RS
Looking for change? Roll the Dice and demand Attention, arXiv, 2020. [paper], [code, dataset]
MXNet + Python
LamboiseNet
CNN (Light UNet++); Optical RS
Change detection in satellite imagery using deep learning, Master Thesis. [code, dataset, pre-trained model]
Pytorch
DTCDSCN
CNN; Siamese
Building change detection for remote sensing images using a dual task constrained deep siamese convolutional network model, undergoing review. [code, dataset]
Pytorch
Land-Cover-Analysis
CNN (UNet); Post-Classification; Optical RS
Land Use/Land cover change detection in cyclone affected areas using convolutional neural networks. [report], [code, dataset, pre-trained model]
TensorFlow+Keras
CorrFusionNet
CNN; Scene-level; Siamese; Optical RS
Correlation based fusion network towards multi-temporal scene classification and change detection, undergoing review. [code, pre-trained model], [dataset]
TensorFlow 1.8
SSCDNet
CNN (ResNet18); Siamese; Transfer Learning; Semantic; Streetview
Weakly supervised silhouette-based semantic scene change detection, ICRA, 2020. [paper] [code, dataset, pre-trained model]
Pytorch+Python3.6
Heterogeneous_CD
AE (Code-Aligned AE); Unsupervised; Transformation; Heterogeneous; Optical RS
Code-aligned autoencoders for unsupervised change detection in multimodal remote sensing images, arXiv, 2020. [paper] [code, dataset]
TensorFlow 2.0
FDCNN
CNN (VGG16); Transfer Learning; Pure-Siamese; Multi-scale; Optical RS
A feature difference convolutional neural network-based change detection method, TGRS, 2020. [paper] [code, dataset, pre-trained model]
Caffe+Python2.7
STANet
CNN (ResNet-18); Attention Mechanism; Pure-Siamese; Spatial–Temporal Dependency; Optical RS
A spatial-temporal attention-based method and a new dataset for remote sensing image change detection, RS, 2020. [paper] [code, dataset]
Pytorch+Python3.6
X-Net
CNN; Unsupervised; Transformation; Heterogeneous; Optical RS; SAR
Deep image translation with an affinity-based change prior for unsupervised multimodal change detection, 2020. [paper] [code, dataset]
Tensorflow 1.4
ACE-Net
AE (Adversarial Cyclic Encoders); Unsupervised; Transformation; Heterogeneous; Optical RS; SAR
Deep image translation with an affinity-based change prior for unsupervised multimodal change detection, 2020. [paper] [code, dataset]
Tensorflow 1.4
VGG_LR
CNN (VGG16); Transfer Learning; Pure-Siamese; SLIC; Low Ranks; Optical RS
Change detection based on deep features and low rank, GRSL, 2017. [paper] [re-implementation code, dataset, pre-trained model]
Caffe+Matlab
CDNet
CNN; Siamese; Multimodal Data; Point Cloud Data
Detecting building changes between airborne laser scanning and photogrammetric data, RS, 2019. [paper], [code]
Pytorch
SCCN
AE (DAE); Unsupervised; Heterogeneous; Optical RS; SAR
A deep convolutional coupling network for change detection based on heterogeneous optical and radar images, TNNLS, 2018. [paper] [re-implementation code]
TensorFlow 2.0
cGAN
GAN (conditional GAN); Heterogeneous; Optical RS; SAR
A conditional adversarial network for change detection in heterogeneous images, GRSL, 2019. [paper] [re-implementation code]
TensorFlow 2.0
DASNet
CNN (VGG16); Siamese; Attention Mechanism ; Optical RS
DASNet: Dual attentive fully convolutional siamese networks for change detection of high resolution satellite images, arXiv, 2020. [paper] [code, dataset, pre-trained model]
Pytorch+Python3.6
UNetLSTM
CNN (UNet); RNN (LSTM); Integrated Model; Optical RS
Detecting Urban Changes With Recurrent Neural Networks From Multitemporal Sentinel-2 Data, IGARSS, 2019. [paper] [code, dataset, pre-trained model] and [code]
Pytorch+Python3.6
CDMI-Net
CNN (Unet); Pure-Siamese; Multiple Instance Learning; Landslide Mapping; Optical RS
Deep multiple instance learning for landslide mapping, GRSL, 2020. [paper] [code, pre-trained model]
Pytorch+Python3.6
DSFANet
DNN; Unsupervised; Pre-classification; Slow Feature Analysis; Optical RS
Unsupervised deep slow feature analysis for change detection in multi-temporal remote sensing images, TGRS, 2019. [paper] [code, dataset]
TensorFlow 1.7
CD-UNet++
CNN (improved UNet++); Direct Classification; Optical RS
End-to-end change detection for high resolution satellite images using improved UNet++, RS, 2019. [paper] [code]
TensorFlow+Keras
SiameseNet
CNN (VGG16); Pure-Siamese; Optical RS
Siamese network with multi-level features for patch-based change detection in satellite imagery, GlobalSIP, 2018. [paper] [code, dataset]
TensorFlow+Keras
Re3FCN
CNN (ConvLSTM); PCA; 3D convolution; Multi-class changes; Optical RS; Hyperspectral
Change detection in hyperspectral images using recurrent 3D fully convolutional networks, RS, 2018. [paper] [code, dataset]
TensorFlow+Keras
FC-EF, FC-Siam-conc, FC-Siam-diff
CNN (UNet); Pure-Siamese; Optical RS
Fully convolutional siamese networks for change detection, ICIP, 2018. [paper] [code, dataset]
Pytorch
CosimNet
CNN (Deeplab v2); Pure-Siamese; Streetview
Learning to measure changes: fully convolutional siamese metric networks for scene change detection, arXiv, 2018. [paper] [code, dataset, pre-trained model]
Pytorch+Python2.7
Mask R-CNN
Mask R-CNN (ResNet-101); Transfer Learning; Post-Classification; Optical RS
Slum segmentation and change detection: a deep learning approach, NIPS, 2018. [paper] [code, dataset, pre-trained model]
TensorFlow+Keras
CaffeNet
CNN (CaffeNet); Unsupervised; Transfer Learning; Optical RS
Convolutional neural network features based change detection in satellite images, IWPR, 2016. [paper] [code, dataset]
TensorFlow+Keras
CWNN
CNN (CWNN); Unsupervised; Pre-Classification; SAR
Sea ice change detection in SAR images based on convolutional-wavelet neural networks, GRSL, 2019. [paper] [code, dataset]
Matlab
MLFN
CNN (DenseNet); Transfer learning; SAR
Transferred deep learning for sea ice change detection from synthetic aperture radar images, GRSL, 2019. [paper] [code, dataset]
Caffe+Matlab
GarborPCANet
CNN (PCANet); Unsupervised; Pre-Classification; Gabor Wavelets; SAR
Automatic change detection in synthetic aperture radar images based on PCANet, GRSL, 2016. [paper] [code, dataset]
Matlab
Ms-CapsNet
CNN (Ms-CapsNet); Capsule; Attention Mechanism; Adaptive Fusion Convolution; SAR
Change detection in SAR images based on multiscale capsule network, GRSL, 2020. [paper] [code, dataset]
Matlab+Keras2.16
DCNet
CNN; Unsupervised; Pre-Classification; SAR
Change detection from synthetic aperture radar images based on channel weighting-based deep cascade network, JSTARS, 2019. [paper] [code, dataset]
Caffe
ChangeNet
CNN; Siamese; StreetView
ChangeNet: a deep learning architecture for visual change detection, ECCV, 2018. [paper] [code, dataset]
Pytorch
Others will be added soon!
### 2.2 Available codes for traditional methods
Table 2. A list of available codes for traditional change detection methods.
Methods
Keywords
Publication
Implementation
Several Classical Methods
CVA; DPCA; Image Differencing; Image Ratioing; Image Regression; IR-MAD; MAD; PCAkMeans; PCDA; KMeans; OTSU; Fixed Threshold
A toolbox for remote sensing change detection. [code]
Matlab
Matlab Toolbox Change Detection
IR-MAD; IT-PCA; ERM; ICM
A toolbox for unsupervised change detection analysis, IJRS, 2016.[paper] [code]
Matlab
RFR,SVR,GPR
Unsupervised; Image Regression; Heterogeneous; Optical RS; SAR
Unsupervised image regression for heterogeneous change detection, TGRS, 2019. [paper] [code]
Matlab
HPT
Unsupervised; Transformation; Heterogeneous; Optical RS; SAR
Change detection in heterogenous remote sensing images via homogeneous pixel transformation, TIP, 2018. [paper] [re-implementation code]
Matlab
kCCA
Canonical Correlation Analysis; Cross-Sensor; Optical RS
Spectral alignment of multi-temporal cross-sensor images with automated kernel correlation analysis, IJPRS, 2015. [paper] [code]
Matlab
Ker. Diff. RBF
Unsupervised; K-means; Optical RS
Unsupervised change detection with kernels, GRSL, 2012. [paper] [code]
Matlab
FDA-RM
DI-based; Frequency-Domain Analysis; Random Multigraphs; SAR
Synthetic aperture radar image change detection based on frequency domain analysis and random multigraphs, JARS, 2018. [paper] [code]
Matlab
CD-NR-ELM
DI-based; Pre-Classification; Extreme Learning Machine; SAR
Change detection from synthetic aperture radar images based on neighborhood-based ratio and extreme learning machine, JARS, 2016. [paper] [code, dataset]
Matlab
None
Likelihood Ratio; Test Statistic; SAR
Change detection in polarimetric SAR
images, 2015. [report] [code]
Python
PCA K-Means
Unsupervised; DI-based; PCA; K Means; Optical RS
Unsupervised Change Detection in Satellite Images Using Principal Component Analysis and k-Means Clustering, GRSL, 2009. [paper] [re-implementation code, dataset] or [re-implementation code]
Matlab
PTCD
Tensor; Hyperspectral Optical RS
Three-Order Tucker Decomposition and Reconstruction Detector for Unsupervised Hyperspectral Change Detection. JSTARS, 2021. [paper] [code, dataset]
Matlab
GBF-CD
Data Fusion; Graph; EM; KI;
Graph-Based Data Fusion Applied to: Change Detection and Biomass Estimation in Rice Crops. Remote Sensing, 2020 [paper] [code, dataset]
Matlab
Others will be added soon!
## 3. Open datasets
Currently, there are some freely available data sets for change detection, which can be used as benchmark datasets for AI training and accuracy evaluation in future research. Detailed information is presented in Table 3.
Table 3. A list of open datasets for change detection.
Type
Data set
Description
Optical RS
DSIFN Dataset [25]
6 bi-temporal high resolution images from Google Earth. There are 3600 image pairs with size of 512 × 512 for training, 340 for validation, and 48 for test. [Download]
S2MTCP [26]
1520 Sentinel-2 level 1C image pairs focused on urban areas around the world, with 10m spatial resolution and the size of 600x600 pixels. Geometric or radiometric corrections are not performed. [Download]
SYSU-CD [27]
20000 pairs of 0.5-m aerial images of size 256×256 taken between the years 2007 and 2014 in Hong Kong, including 6 change types: (a) newly built urban buildings; (b) suburban dilation; (c) groundwork before construction; (d) change of vegetation; (e) road expansion; (f) sea construction. [Download]
S2Looking [28]
Building change detection dataset consists of 5000 registered bitemporal image pairs (size of 1024*1024, 0.5 ~ 0.8 m/pixel) of rural areas throughout the world and more than 65,920 annotated change instances, separately indicating the newly built and demolished building [Download]
Synthetic and real images Dataset [29]
The database contains 12,000 triples of synthetic images without object shift, 12,000 triples of model images with object shift and 16,000 triples of fragments of real remote sensing images. Performed tests have shown that the proposed CNN is promising and efficient enough in change detection on synthetic and real images [Download]
SEmantic Change detectiON Dataset (SECOND) [24]
a pixel-level annotated semantic change detection dataset, including 4662 pairs of aerial images with 512 x 512 pixels from several platforms and sensors, covering Hangzhou, Chengdu, and Shanghai. It focus on 6 main land-cover classes, i.e. , non-vegetated ground surface, tree, low vegetation, water, buildings and playgrounds , that are frequently involved in natural and man-made geographical changes. [Download]
Hyperspectral change detection dataset [1]
3 different hyperspectral scenes acquired by AVIRIS or HYPERION sensor, with 224 or 242 spectral bands, labeled 5 types of changes related with crop transitions at pixel level. [Download]
River HSIs dataset [2]
2 HSIs in Jiangsu province, China, with 198 bands, labeled as changed and unchanged at pixel level. [Download]
HRSCD [3]
291 co-registered pairs of RGB aerial images, with pixel-level change and land cover annotations, providing hierarchical level change labels, for example, level 1 labels include five classes: no information, artificial surfaces, agricultural areas, forests, wetlands, and water. [Download]
WHU building dataset [4]
2-period aerial images containing 12,796 buildings, provided along with building vector and raster maps. [Download]
SZTAKI Air change benchmark [5, 6]
13 aerial image pairs with 1.5 m spatial resolution, labeled as changed and unchanged at pixel level. [Download]
OSCD [7]
24 pairs of multispectral images acquired by Sentinel-2, labeled as changed and unchanged at pixel level. [Download]
Change detection dataset [8]
4 pairs of multispectral images with different spatial resolutions, labeled as changed and unchanged at pixel level. [Download]
MtS-WH [9]
2 large-size VHR images acquired by IKONOS sensors, with 4 bands and 1 m spatial resolution, labeled 5 types of changes (i.e., parking, sparse houses, residential region, and vegetation region) at scene level. [Download]
ABCD [10]
16,950 pairs of RGB aerial images for detecting washed buildings by tsunami, labeled damaged buildings at scene level. [Download]
xBD [11]
Pre- and post-disaster satellite imageries for building damage assessment, with over 850,000 building polygons from 6 disaster types, labeled at pixel level with 4 damage scales. [Download]
AICD [12]
1000 pairs of synthetic aerial images with artificial changes generated with a rendering engine, labeled as changed and unchanged at pixel level. [Download]
Database of synthetic and real images [13]
24,000 synthetic images and 16,000 fragments of real season-varying RS images obtained by Google Earth, labeled as changed and unchanged at pixel level. [Download]
LEVIR-CD [14]
637 very high-resolution (VHR, 0.5m/pixel) Google Earth (GE) image patch pairs with a size of 1024 × 1024 pixels and contains a total of 31,333 individual change building instances, labeled as changed and unchanged at pixel level. [Download]
Bastrop fire dataset [21]
4 images acquired by different sensors over the Bastrop County, Texas (USA). It is composed by a Landsat 5 TM as the pre-event image and a Landsat 5 TM, a EO-1 ALI and a Landsat 8 as post-event images, labeled as changed and unchanged at pixel level, mainly caused by wildfire. [Download]
Google data set [23]
19 season-varying VHR images pairswith 3 bands of red, green, and blue, a spatial resolution of 0.55 m, and the size ranging from 1006×1168 pixels to 4936×5224 pixels. The image changes include waters, roads, farmland, bare land, forests, buildings, ships, etc. Buildings make up the main changes. acquired during the periods between 2006 and 2019, covering the suburb areas of Guangzhou City, China. [Download]
Optical RS & SAR
California dataset [22]
3 images, including a RS image captured by Landsat 8 with 9 channels on 2017, a SAR image captured by Sentinel-1A (recorded in polarisations VV and VH) after the occurrence of a flood, and a ground truth map. [Download]
Homogeneous CD Dataset [30]
6 scenarios: Scenario 1 with two single-polarizationSAR data sets; Scenario 2 with two PolSAR data sets; Scenario 3 with two optical image data sets. HeterogeneousCD: Scenario 4 with two SAR/optical (multispectral) datasets; Scenario 5 with two multispectral data sets of differentbands acquired from different sensors; Scenario 6 with twoPolSAR/optical (multispectral) data sets. [Download]
Street view
VL-CMU-CD [15]
1362 co-registered pairs of RGB and depth images, labeled ground truth change (e.g., bin, sign, vehicle, refuse, construction, traffic cone, person/cycle, barrier) and sky masks at pixel level. [Download]
PCD 2015 [16]
200 panoramic image pairs in "TSUNAMI" and "GSV" subset, with the size of 224 × 1024 pixels, label as changed and unchanged at pixel level. [Download]
Change detection dataset [17]
Image sequences of city streets captured by a vehicle-mounted camera at two different time points, with the size of 5000 × 2500 pixels, labeled 3D scene structure changes at pixel level. [Download]
CV
CDNet 2012 [18]
6 video categories with 4 to 6 videos sequences in each category, and the groundtruth images contain 5 labels namely: static, hard shadow, outside region of interest, unknown motion (usually around moving objects, due to semi-transparency and motion blur), and motion. [Download]
CDNet 2014 [19,20]
22 additional videos (∼70; 000 pixel-wise annotated frames) spanning 5 new categories that incorporate challenges encountered in many surveillance settings, and provides realistic, camera captured (without CGI), diverse set of indoor and outdoor videos like the CDnet 2012. [Download]
ChangeSim [31]
a challenging dataset aimed at online scene change detection and more, collecting in photo-realistic simulation environments with the presence of environmental non-targeted variations, such as air turbidity and light condition changes, as well as targeted object changes in industrial indoor environments. [Download]
More video datasets
It can be seen that the amount of open datasets that can be used for change detection tasks is small, and some of them have small data sizes. At present, there is still a lack of large SAR datasets that can be used for AI training. Most AI-based change detection methods are based on several SAR data sets that contain limited types of changes, e.g., the Bern dataset, the Ottawa dataset, the Yellow River dataset, and the Mexico dataset, which cannot meet the needs of change detection in areas with complex land cover and various change types. Moreover, their labels are not freely available. Street-view datasets are generally used for research of AI-based change detection methods in computer vision (CV). In CV, change detection based on pictures or video is also a hot research field, and the basic idea is consistent with that based on RS data. Therefore, in addition to street view image datasets, several video datasets in CV can also be used for research on AI-based change detection methods, such as CDNet 2012 and CDNet 2014.
## 4. Applications
The development of AI-based change detection techniques has greatly facilitated many applications and has improved their automation and intelligence. Most AI-based change detection generates binary maps, and these studies only focus on the algorithm itself, without a specific application field. Therefore, it can be considered that they are generally suitable for LULC change detection. In this section, we focus on the techniques that are associated with specific applications, and they can be broadly divided into four categories:
* **Urban contexts**: urban expansion, public space management, and building change detection;
* **Resources and environment**: human-driven environmental changes, hydro-environmental changes, sea ice, surface water, and forest monitoring;
* **Natural disasters**: landslide mapping and damage assessment;
* **Astronomy**: planetary surfaces.
We provide an overview of the various change detection techniques in the literature for the different application categories. The works and data types associated with these applications are listed in Table 4.
Table 4. Summary of main applications of AI-based change detection techniques.
Applications
Data Types
Papers
Urban contexts
Urban expansion
Satellite images
Lyu et.al (2018), Tong et.al (2007)
SAR images
Iino et.al (2017)
Public space management
Street view images
Varghese et.al (2018)
Road surface
UAV images
Truong et.al (2020)
Building change detection
Aerial images
Ji et.al (2019), Sun et.al (2019), Nemoto et.al (2017)
Satellite images
Huang et.al (2019), Zhu et.al (2018)
Satellite/Aerial images
Jiang et.al (2020), Ji et.al (2018), Saha et.al (2020)
Airborne laser scanning data and aerial images
Zhang et.al (2019)
SAR images
Jaturapitpornchai et.al (2019)
Satellite images and GIS map
Ghaffarian et.al (2019)
Resources & environment
Human-driven environmental changes
Satellite images
Chen et.al (2016)
Hydro-environmental changes
Satellite images
Nourani et.al (2018)
Sea ice
SAR images
Gao et.al (2019), Gao et.al (2019)
Surface water
Satellite images
Song et.al (2019), Rokni et.al (2015)
Forest monitoring
Satellite images
Khan et.al (2017), Lindquist et.al (2016), Deilmai et.al (2014), Woodcock et.al (2001), Gopal et.al (1996)
Natural disasters
Landslide mapping
Aerial images
Fang et.al (2020), Lei et.al (2019)
Satellite images
Chen et.al (2018), Ding et.al (2016), Tarantino et.al (2006)
Damage assessment
Satellite images
caused by tsunami [Sublime et.al (2019),Singh et.al (2015)], particular incident [Hedjam et.al (2019)], flood [Peng et.al (2019)], or earthquake [Ji et.al (2019)]
Aerial images
caused by tsunami [Fujita et.al (2017)]
SAR images
caused by fires [Planinšič et.al (2018)], or earthquake [Saha et.al (2018)]
Street view images
caused by tsunami [Sakurada et.al (2015)]
Street view images and GIS map
caused by tsunami [Sakurada et.al (2017)]
Astronomy
Planetary surfaces
Satellite images
Kerner et.al (2019)
## 5. Software programs
There are currently a large number of software with change detection tools, and we have a brief summary of them, see table 5.
Table 5. A list of software for change detection.
Type
Name
Description
Commercial
ERDAS IMAGINE
provides true value, consolidating remote sensing, photogrammetry, LiDAR analysis, basic vector analysis, and radar processing into a single product, including a variety of change detection tools.
ArcGIS
change detection can be calculate between two raster datasets by using the raster calculator tool or deep learning workflow.
ENVI
provides change detection analysis tools and the ENVI deep learning module.
eCognition
can be used for a variety of change mapping, and by leveraging deep learning technology from the Google TensorFlow™ library, eCognition empowers customers with highly sophisticated pattern recognition and correlation tools that automate the classification of objects of interest for faster and more accurate results, more.
PCI Geomatica
provides change detection tools, and can be useful in numerous circumstances in which you may want to analyze change, such as: storm damage, forest-fire damage, flooding, urban sprawl, and more.
SenseTime
SenseRemote remote sensing intelligent solutions
Open source
QGIS
provides many change detection tools.
Orfeo ToolBox
change detection by multivariate alteration detector (MAD) algorithm.
Change Detection ToolBox
MATLAB toolbox for remote sensing change detection.
## 6. Review papers for change detection
The following papers are helpful for researchers to better understand this field of remote sensing change detection, see table 6.
Table 6. A list of review papers on change detection.
Published year
Review paper
1989
Digital change detection techniques using remotely sensed data, IJRS. [paper]
2004
Digital change detection methods in ecosystem monitoring: a review, IJRS. [paper]
2004
Change detection techniques, IJRS. [paper]
2012
Object-based change detection, IJRS. [paper]
2013
Change detection from remotely sensed images: From pixel-based to object-based approaches, ISPRS. [paper]
2016
3D change detection–approaches and applications, ISPRS. [paper]
2016
Deep learning for remote sensing data a technical tutorial on the state of the art, MGRS. [paper]
2017
Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community, JRS. [paper]
2017
Deep Learning in Remote Sensing, MGRS. [paper]
2018
Computational intelligence in optical remote sensing image processing, ASOC. [paper]
2019
A review of change detection in multitemporal hyperspectral images: current techniques, applications, and challenges, MGRS. [paper]
2019
Deep learning in remote sensing applications: A meta-analysis and review, ISPRS. [paper]
2020
Deep Learning for change detection in remote sensing images: comprehensive review and meta-analysis, arXiv. [paper]
2020
Change detection based on artificial intelligence: state-of-the-art and challenges, RS. [paper]
## 7. Reference
[1] Hyperspectral Change Detection Dataset. Available online: https://citius.usc.es/investigacion/datasets/hyperspectral-change-detection-dataset (accessed on 4 May 2020).
[2] Wang, Q.; Yuan, Z.; Du, Q.; Li, X. GETNET: A General End-to-End 2-D CNN Framework for Hyperspectral Image Change Detection. IEEE Trans. Geosci. Remote Sens. 2018, 57, 3–13. [Google Scholar] [CrossRef]
[3] Daudt, R.C.; Le Saux, B.; Boulch, A.; Gousseau, Y. Multitask learning for large-scale semantic change detection. Comput. Vis. Image Underst. 2019, 187, 102783. [Google Scholar] [CrossRef]
[4] Ji, S.; Wei, S.; Lu, M. Fully Convolutional Networks for Multisource Building Extraction from an Open Aerial and Satellite Imagery Data Set. IEEE Trans. Geosci. Remote Sens. 2018, 57, 574–586. [Google Scholar] [CrossRef]
[5] Benedek, C.; Sziranyi, T. Change Detection in Optical Aerial Images by a Multilayer Conditional Mixed Markov Model. IEEE Trans. Geosci. Remote Sens. 2009, 47, 3416–3430. [Google Scholar] [CrossRef]
[6] Benedek, C.; Sziranyi, T. A Mixed Markov model for change detection in aerial photos with large time differences. In Proceedings of the 2008 19th International Conference on Pattern Recognition, Tampa, FL, USA, 8–11 December 2008; pp. 1–4. [Google Scholar]
[7] Daudt, R.C.; Le Saux, B.; Boulch, A.; Gousseau, Y. Urban change detection for multispectral earth observation using convolutional neural networks. In Proceedings of the IGARSS 2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018; pp. 2115–2118. [Google Scholar]
[8] Zhang, M.; Shi, W. A Feature Difference Convolutional Neural Network-Based Change Detection Method. IEEE Trans. Geosci. Remote Sens. 2020, 1–15. [Google Scholar] [CrossRef]
[9] Wu, C.; Zhang, L.; Zhang, L. A scene change detection framework for multi-temporal very high resolution remote sensing images. Signal Process. 2016, 124, 184–197. [Google Scholar] [CrossRef]
[10] Fujita, A.; Sakurada, K.; Imaizumi, T.; Ito, R.; Hikosaka, S.; Nakamura, R. Damage detection from aerial images via convolutional neural networks. In Proceedings of the 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), Nagoya Univ, Nagoya, Japan, 08–12 May 2017; pp. 5–8 [Google Scholar]
[11] Gupta, R.; Goodman, B.; Patel, N.; Hosfelt, R.; Sajeev, S.; Heim, E.; Doshi, J.; Lucas, K.; Choset, H.; Gaston, M. Creating xBD: A dataset for assessing building damage from satellite imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 16–20 June 2019; pp. 10–17. [Google Scholar]
[12] Bourdis, N.; Marraud, D.; Sahbi, H. Constrained optical flow for aerial image change detection. In Proceedings of the 2011 IEEE International Geoscience and Remote Sensing Symposium, Vancouver, BC, Canada, 24–29 July 2011; pp. 4176–4179. [Google Scholar] [CrossRef]
[13] Lebedev, M.A.; Vizilter, Y.V.; Vygolov, O.V.; Knyaz, V.A.; Rubis, A.Y. Change detection in remote sensing images using conditional adversarial networks. ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2018, 565–571. [Google Scholar] [CrossRef]
[14] Chen, H.; Shi, Z. A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection. Remote Sensing, 12(10), 1662. [Google Scholar] [CrossRef]
[15] Alcantarilla, P.F.; Stent, S.; Ros, G.; Arroyo, R.; Gherardi, R. Street-view change detection with deconvolutional networks. Auton. Robot. 2018, 42, 1301–1322. [Google Scholar] [CrossRef]
[16] Sakurada, K.; Okatani, T. Change detection from a street image pair using CNN features and superpixel segmentation. In Proceedings of the British Machine Vision Conference (BMVC), Swansea, UK, 7–10 September 2015; pp. 61.1–61.12. [Google Scholar]
[17] Sakurada, K.; Okatani, T.; Deguchi, K. Detecting changes in 3D structure of a scene from multi-view images captured by a vehicle-mounted camera. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 137–144. [Google Scholar]
[18] Goyette, N.; Jodoin, P.-M.; Porikli, F.; Konrad, J.; Ishwar, P. Changedetection. net: A new change detection benchmark dataset. In Proceedings of the 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Providence, RI, USA, 16–21 June 2012; pp. 1–8. [Google Scholar]
[19] Wang, Y.; Jodoin, P.-M.; Porikli, F.; Konrad, J.; Benezeth, Y.; Ishwar, P. CDnet 2014: An Expanded Change Detection Benchmark Dataset. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA, 23–28 June 2014; pp. 393–400. [Google Scholar]
[20] Goyette, N.; Jodoin, P.-M.; Porikli, F.; Konrad, J.; Ishwar, P. A Novel Video Dataset for Change Detection Benchmarking. IEEE Trans. Image Process. 2014, 23, 4663–4679. [Google Scholar] [CrossRef]
[21] Volpi, Michele; Camps-Valls, Gustau; Tuia, Devis (2015). Spectral alignment of multi-temporal cross-sensor images with automated kernel canonical correlation analysis; ISPRS Journal of Photogrammetry and Remote Sensing, vol. 107, pp. 50-63, 2015. [CrossRef]
[22] L. T. Luppino, F. M. Bianchi, G. Moser and S. N. Anfinsen. Unsupervised Image Regression for Heterogeneous Change Detection. IEEE Transactions on Geoscience and Remote Sensing. 2019, vol. 57, no. 12, pp. 9960-9975. [CrossRef]
[23] D. Peng, L. Bruzzone, Y. Zhang, H. Guan, H. Ding and X. Huang, SemiCDNet: A Semisupervised Convolutional Neural Network for Change Detection in High Resolution Remote-Sensing Images. IEEE Transactions on Geoscience and Remote Sensing. 2020. [CrossRef]
[24] Yang, Kunping, et al. Asymmetric Siamese Networks for Semantic Change Detection. arXiv preprint arXiv:2010.05687 (2020). [CrossRef]
[25] Zhang, C., Yue, P., Tapete, D., Jiang, L., Shangguan, B., Huang, L., & Liu, G. A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing. 2020. [CrossRef]
[26] LEENSTRA, Marrit, et al. Self-supervised pre-training enhances change detection in Sentinel-2 imagery. arXiv. 2021. [CrossRef]
[27] SHI, Qian, et al. A Deeply Supervised Attention Metric-Based Network and an Open Aerial Image Dataset for Remote Sensing Change Detection. IEEE Transactions on Geoscience and Remote Sensing. 2021. [CrossRef]
[28] SHEN, Li, et al. S2Looking: A Satellite Side-Looking Dataset for Building Change Detection. arXiv. 2021. [CrossRef]
[29] LEBEDEV, M. A., et al. CHANGE DETECTION IN REMOTE SENSING IMAGES USING CONDITIONAL ADVERSARIAL NETWORKS. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 2018. [CrossRef]
[30] SUN, Yuli, et al. Structure Consistency-Based Graph for Unsupervised Change Detection With Homogeneous and Heterogeneous Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing, 2021. [CrossRef]
[31] PARK, Jin-Man, et al. ChangeSim: Towards End-to-End Online Scene Change Detection in Industrial Indoor Environments. arXiv. 2021. [CrossRef]
## Cite
If you find this review helpful to you, please consider citing our paper. [Open Access]
```
@Article{rs12101688,
AUTHOR = {Shi, Wenzhong and Zhang, Min and Zhang, Rui and Chen, Shanxiong and Zhan, Zhao},
TITLE = {Change Detection Based on Artificial Intelligence: State-of-the-Art and Challenges},
JOURNAL = {Remote Sensing},
VOLUME = {12},
YEAR = {2020},
NUMBER = {10},
ARTICLE-NUMBER = {1688},
URL = {https://www.mdpi.com/2072-4292/12/10/1688},
ISSN = {2072-4292},
DOI = {10.3390/rs12101688}
}
```
## Note
This list will be updated in time, and volunteer contributions are welcome. For questions or sharing, please feel free to [contact us](mailto:[email protected]) or make issues.
##### Reference materials:
* [I-Hope-Peace/ChangeDetectionRepository](https://github.com/I-Hope-Peace/ChangeDetectionRepository)
* [Michele Volpi personal research page](https://sites.google.com/site/michelevolpiresearch/codes)
* [llu025/Heterogeneous_CD](https://github.com/llu025/Heterogeneous_CD)
* [wenhwu/awesome-remote-sensing-change-detection](https://github.com/wenhwu/awesome-remote-sensing-change-detection)
* [neverstoplearn/remote_sensing_change_detection](https://github.com/neverstoplearn/remote_sensing_change_detection)
* [Change Detection in GIS](https://www.gislounge.com/change-detection-in-gis/)
* [Gao Feng personal research page](http://feng-gao.cn/)
* [Bobholamovic/ChangeDetectionToolbox](https://github.com/Bobholamovic/ChangeDetectionToolbox)
