Spatial, Temporal, and Geometric Fusion for Remote Sensing Images

Hessah Albanwan

2024-04-27

Abstract:Remote sensing (RS) images are important to monitor and survey earth at varying spatial scales. Continuous observations from various RS sources complement single observations to improve applications. Fusion into single or multiple images provides more informative, accurate, complete, and coherent data. Studies intensively investigated spatial-temporal fusion for specific applications like pan-sharpening and spatial-temporal fusion for time-series analysis. Fusion methods can process different images, modalities, and tasks and are expected to be robust and adaptive to various types of images (e.g., spectral images, classification maps, and elevation maps) and scene complexities. This work presents solutions to improve existing fusion methods that process gridded data and consider their type-specific uncertainties. The contributions include: 1) A spatial-temporal filter that addresses spectral heterogeneity of multitemporal images. 2) 3D iterative spatiotemporal filter that enhances spatiotemporal inconsistencies of classification maps. 3) Adaptive semantic-guided fusion that enhances the accuracy of DSMs and compares them with traditional fusion approaches to show the significance of adaptive methods. 4) A comprehensive analysis of DL stereo matching methods against traditional Census-SGM to obtain detailed knowledge on the accuracy of the DSMs at the stereo matching level. We analyze the overall performance, robustness, and generalization capability, which helps identify the limitations of current DSM generation methods. 5) Based on previous analysis, we develop a novel finetuning strategy to enhance transferability of DL stereo matching methods, hence, the accuracy of DSMs. Our work shows the importance of spatial, temporal, and geometric fusion in enhancing RS applications. It shows that the fusion problem is case-specific and depends on the image type, scene content, and application.

Image and Video Processing

What problem does this paper attempt to address?

The main purpose of this paper is to address the challenges in spatial, temporal, and geometric fusion of remote sensing images to improve the effectiveness of remote sensing applications. Specifically, the authors propose several methods to improve existing fusion techniques:

1. **Spatio-temporal filtering method**: Addresses the spectral heterogeneity issue in multi-temporal images.
2. **3D iterative spatio-temporal filtering**: Enhances the spatio-temporal consistency of classification maps.
3. **Adaptive semantic-guided fusion**: Improves the accuracy of Digital Surface Models (DSM) and demonstrates the advantages of adaptive methods by comparing them with non-adaptive methods.
4. **Analysis of deep learning stereo matching methods**: Compares traditional Census Semi-Global Matching (SGM) methods and evaluates the performance, robustness, and generalization ability of deep learning stereo matching methods.
5. **Novel fine-tuning strategy**: Proposes a new fine-tuning strategy to enhance the transferability of deep learning stereo matching methods, thereby improving the accuracy of DSM.

The paper reveals the importance of using spatial, temporal, and geometric fusion to enhance various remote sensing applications and points out that fusion problems are specific, depending on the type of images, scene content, and application scenarios. Therefore, a deep understanding of the nature of images, scene content, application scenarios, and the limitations of existing fusion methods is required to provide the optimal fusion solution.