Abstract:Due to the limitations of current technology and budget, a single satellite sensor cannot obtain high spatiotemporal resolution remote sensing images. Therefore, remote sensing image spatio-temporal fusion technology is considered as an effective solution and has attracted extensive attention. In the field of deep learning, due to the fixed size of the perception field of a convolutional neural network, it is impossible to model the correlation of global features, and the features extracted only through convolution operation lack the ability to capture long-distance features. At the same time, complex fusion methods cannot better integrate temporal and spatial features. In order to solve these problems, we propose a multistage remote sensing image spatio-temporal fusion model based on texture transformer and convolutional neural network. The model combines the advantages of transformer and convolutional network, uses a lightweight convolution network to extract spatial features and temporal discrepancy features, uses transformer to learn global temporal correlation, and finally, fuses temporal features with spatial features. In order to make full use of the features obtained in different stages, we design a cross-stage adaptive fusion module CSAFM. The module adopts the self-attention mechanism to adaptively integrate the features of different scales while considering the temporal and spatial characteristics. To test the robustness of the model, the experiments are carried out on three datasets of CIA, LGC, and DX. Compared with five typical spatio-temporal fusion algorithms, we obtain excellent results, which prove the superiority of MSFusion model.

The fusion of multisensor images with multi-resolutions

Novel Multi-Resolution Image Fusion Algorithm Based on Kalman Filtering

Multi-Frame Image Fusion Method Combining Spatial-Temporal Saliency Detection and Nsct

Multiresolutional multisource image fusion algorithm

The Realistic Fusion of Multi-spectral Images

Multi-Sensor Image Fusion Using Optimized Support Vector Machine and Multiscale Weighted Principal Component Analysis

Simultaneous spatial-temporal image fusion using Kalman filtered compressed sensing

Multi-Focus Image Fusion Algorithm in Sensor Networks

Analyzing of the Fusion Estimation Algorithm of a Class of Dynamic Multiscale System

A Multi-Mems Sensors Information Fusion Algorithm

Multi-Sensor Fusion of Infrared and Visible Images Based on Modified Side Window Filter and Intensity Transformation

The multiscale state fusion estimation for nonlinear systems with multirate sensors

Multisensor decentralized nonlinear fusion using adaptive cubature information filter

MSFusion: Multistage for Remote Sensing Image Spatiotemporal Fusion Based on Texture Transformer and Convolutional Neural Network

Fast mutual modulation fusion for multi-sensor images

Multi-sensor Data Fusion Algorithm Based on Fuzzy Adaptive Kalman Filter

A Class of State Fusion Estimation Algorithm for Multirate Multisensor Systems

Application To Environmental Surveillance: Dynamic Image Estimation Fusion And Optimal Remote Sensing With Fuzzy Integral

Image Fusion Based on Multi-Directional Multiscale Analysis and Immune Optimization

Multi-sensor Image Fusion Using Discrete Wavelet Frame Transform

Asynchronous Multirate Multisensor Information Fusion Algorithm