Abstract:Spatiotemporal fusion (STF) is a solution to generate satellite images with both high-spatial and high-temporal resolutions. The deep learning-based STF algorithms focus on spatial dimensions to build a super-resolution (SR) model or the temporal dimensions to build a change prediction (CP) model, or the task itself to build a data-driven end-to-end model. The multi-source images used for STF usually have large spatial scale gaps and temporal spans. The large spatial scale gaps lead to poor spatial details based on a SR model; the large temporal spans make it difficult to accurately reconstruct changing areas based on a CP model. We propose a weighted dual-branch spatiotemporal fusion network based on complementarity between super-resolution and change prediction (WDBSTF), which includes the SR branch and CP branch, and a weight module representing the complementarity of the two branches. The SR branch makes full use of edge information and high-resolution reference images to obtain high-quality spatial features for image reconstruction. The CP branch decomposes complex problems via a two-layer cascaded network, changes features from the difference image, and selects high-quality spatial features through the attention mechanism. The fusion result of the CP branch has rich image details, but the fusion accuracy in the changing area is low due to the lack of detail. The SR branch has consistent and excellent fusion performances in the changing and no-changing areas, but the image details are not rich enough compared with the CP branch due to the large amplification factor. Next, a weighted network was designed to combine the advantages of the two branches to produce improved fusion results. We evaluated the performance of the WDBSTF in three representative scenarios, and both visual and quantitative evaluations demonstrate the state-of-the-art performance of our algorithm. (On the LGC dataset, our method outperforms the suboptimal method by 2.577% on SSIM. On the AHB dataset, our method outperforms the suboptimal method by 1.684% on SSIM. On the CIA dataset, our method outperforms the suboptimal method by 5.55% on SAM).

Low Space-Complexity Composition Algorithm for Weighted Finite-State Transducers

A Synchronized Pruning Composition Algorithm of Weighted Finite State Transducers for Large Vocabulary Speech Recognition

3-Way Composition of Weighted Finite-State Transducers

Query-based Composition for Large-Scale Language Model in LVCSR

A Study of Large Vocabulary Speech Recognition Decoding Using Finite-State Graphs

An Exact Word Lattice Generation Method in the Weighted Finite-State Transducer Framework

WDBSTF: A Weighted Dual-Branch Spatiotemporal Fusion Network Based on Complementarity Between Super-Resolution and Change Prediction.

A VLSI Architecture of Spatial Combinative Lifting Algorithm Based 2-D DWT/IDWT

Efficient Probabilistic Latent Semantic Analysis with Sparsity Control

Weight-importance sparse training in keyword spotting

Consistent Weighted Sampling Made More Practical

TST: Time-Sparse Transducer for Automatic Speech Recognition

SPC-FA: synergic parallel compact finite automaton to accelerate multi-string matching with low memory.

FOLSOM: A FAST AND MEMORY-EFFICIENT PHRASE-BASED APPROACH TO STATISTICAL MACHINE TRANSLATION

Spatial Combinative Lifting Algorithm Based on Block for 9/7 Wavelet

State-Free Inference of State-Space Models: The Transfer Function Approach

Automatic Composition of Semantic Web Services An Enhanced State Space Search Approach

USM-Lite: Quantization and Sparsity Aware Fine-tuning for Speech Recognition with Universal Speech Models

Enhancing SMT-based Weighted Model Integration by structure awareness

Leveraging Weighted Automata in Compositional Reasoning about Concurrent Probabilistic Systems

EfficientASR: Speech Recognition Network Compression via Attention Redundancy and Chunk-Level FFN Optimization