Abstract:Urbanization advances at unprecedented rates, resulting in negative effects on the environment and human well-being. Remote sensing has the potential to mitigate these effects by supporting sustainable development strategies with accurate information on urban growth. Deep learning-based methods have achieved promising urban change detection results from optical satellite image pairs using convolutional neural networks (ConvNets), transformers, and a multi-task learning setup. However, transformers have not been leveraged for urban change detection with multi-temporal data, i.e., >2 images, and multi-task learning methods lack integration approaches that combine change and segmentation outputs. To fill this research gap, we propose a continuous urban change detection method that identifies changes in each consecutive image pair of a satellite image time series. Specifically, we propose a temporal feature refinement (TFR) module that utilizes self-attention to improve ConvNet-based multi-temporal building representations. Furthermore, we propose a multi-task integration (MTI) module that utilizes Markov networks to find an optimal building map time series based on segmentation and dense change outputs. The proposed method effectively identifies urban changes based on high-resolution satellite image time series acquired by the PlanetScope constellation (F1 score 0.551) and Gaofen-2 (F1 score 0.440). Moreover, our experiments on two challenging datasets demonstrate the effectiveness of the proposed method compared to bi-temporal and multi-temporal urban change detection and segmentation methods.

What problem does this paper attempt to address?

The paper primarily aims to address the following two key issues: 1. **Utilizing self-attention mechanisms to handle temporal features in multi-temporal satellite image sequences**: Most current urban change detection methods are designed based on bi-temporal image pairs, which have limitations when dealing with rapid urbanization processes. The paper proposes a continuous urban change detection method that captures long-term dependencies across all timestamps by applying self-attention mechanisms in the time series of satellite images, thereby improving the multi-temporal building representations extracted by Convolutional Neural Networks (ConvNets). 2. **Integration of multi-task prediction results**: Multi-task learning frameworks typically combine building segmentation and change detection tasks to enhance detection performance. However, existing methods rarely focus on how to effectively integrate segmentation and change prediction results. This paper proposes a novel Multi-Task Integration (MTI) module that uses a Markov network to combine segmentation and change predictions, thereby finding the most probable series of building maps at each time point. The main contributions of the paper include: - Proposing a Transformer-based Temporal Feature Refinement (TFR) module that can effectively model temporal information in satellite image sequences, thereby improving building representations. - Introducing a new multi-task integration method that represents segmentation and change outputs in a Markov network to find the most probable building maps in each image time series. - Experimental results on the SpaceNet 7 and Wuhan Urban Semantic Understanding (WUSU) datasets demonstrate that the proposed continuous urban change detection method is more effective compared to existing methods. In summary, this research aims to fill two gaps in the current urban change detection field: first, utilizing self-attention mechanisms to handle temporal features in multi-temporal satellite image sequences; second, integrating segmentation and change prediction results within a multi-task learning framework.

Continuous Urban Change Detection from Satellite Image Time Series with Temporal Feature Refinement and Multi-Task Integration

Semi-Supervised Urban Change Detection Using Multi-Modal Sentinel-1 SAR and Sentinel-2 MSI Data

Neural Network-Based Urban Change Monitoring with Deep-Temporal Multispectral and SAR Remote Sensing Data

SFGT-CD: Semantic Feature-Guided Building Change Detection From Bitemporal Remote-Sensing Images With Transformers

A Deep Multitask Learning Framework Coupling Semantic Segmentation and Fully Convolutional LSTM Networks for Urban Change Detection

Time-series land cover change detection using deep learning-based temporal semantic segmentation

SMBCNet: A Transformer-Based Approach for Change Detection in Remote Sensing Images through Semantic Segmentation

Long-Term Annual Mapping Of Four Cities On Different Continents By Applying A Deep Information Learning Method To Landsat Data

MFATNet: Multi-Scale Feature Aggregation via Transformer for Remote Sensing Image Change Detection

Global-Local Collaborative Learning Network for Optical Remote Sensing Image Change Detection

Building Change Detection on High-Resolution Imagery with a Multi-task Semantic Change Detection Method

Fine-Grained Building Change Detection from Very High-Spatial-Resolution Remote Sensing Images Based on Deep Multitask Learning.

Built-Up Area Change Detection Using Multi-Task Network with Object-Level Refinement

Multi-scale Change-Aware Transformer for Remote Sensing Image Change Detection

MTN: A Multi-Scale Transformer Network for Different Resolution Remote Sensing Images Change Detection.

DTT-CGINet: A Dual Temporal Transformer Network with Multi-Scale Contour-Guided Graph Interaction for Change Detection

CD-TransUNet: A Hybrid Transformer Network for the Change Detection of Urban Buildings Using L-Band SAR Images

Hybrid transformer-CNN networks using superpixel segmentation for remote sensing building change detection

TransY-Net:Learning Fully Transformer Networks for Change Detection of Remote Sensing Images

TChange: A Hybrid Transformer-CNN Change Detection Network

A Novel Automatic Change Detection Method for Urban High-Resolution Remotely Sensed Imagery Based on Multiindex Scene Representation