Abstract:Recently, due to higher requirements for satellite video resolution, video super-resolution (VSR) has been extensively studied. However, the following problems have not been effectively resolved: 1) previous satellite VSR methods cannot achieve continuous-scale (integer and noninteger scale) VSR with a single model; 2) satellite video has complex ground and weak textures, which increases the difficulty of capturing motion information. In addition, existing methods adopt a unified alignment path, which leads to a drop in feature alignment accuracy; and 3) during feature fusion, previous methods ignore the correlation of spatiotemporal information in satellite video and cannot make full use of the spatiotemporal information. To address the above problems, in this article, we propose a novel network for continuous-scale satellite VSR (CSVSR). Specifically, first, for effective motion capture and accurate feature alignment, we design a residual-guided and time-aware dynamic routing alignment module, which can use feature residuals to lock motion areas and then dynamically select the corresponding alignment path based on the temporal distance. Second, we proposed a nonlocal mask-based feature fusion module (NMFFM) to exploit the correlation of the spatiotemporal features and complete effective spatiotemporal feature fusion. Third, to make our network adapt to multitask learning, we develop a scale-aware convolutional (SA-Conv) layer, which lets our network dynamically extract scale-adaptive features according to the input scale factors. Finally, we propose a continuous-scale upsampling module with a global feature implicit function (GFIF), which can achieve continuous-scale mapping from features to pixel values. In addition, we carefully design a novel training strategy to optimize our network. Comprehensive experiments verify that the proposed CSVSR has superior reconstruction performance on continuous-scale factors. The code will be available at https://github.com/chongningni/CSVSR.

Local-Global Dynamic Filtering Network for Video Super-Resolution

Video Super-Resolution Based on Multiple Networks Merging

Video super-resolution with phase-aided deformable alignment network

Video Super-Resolution Via a Spatio-Temporal Alignment Network.

Local–Global Feature Fusion Network for Efficient Hyperspectral Image Super-Resolution

Multi-Stage Feature Fusion Network for Video Super-Resolution

Video super-resolution via dense non-local spatial-temporal convolutional network

Learning Frequency-aware Dynamic Network for Efficient Super-Resolution.

Lightweight Feature De-redundancy and Self-calibration Network for Efficient Image Super-resolution

Deformable Convolution Alignment and Dynamic Scale-Aware Network for Continuous-Scale Satellite Video Super-Resolution

FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring

FFFN: Frame-By-Frame Feedback Fusion Network for Video Super-Resolution

Deep Video Super-Resolution with Flow-Guided Deformable Alignment and Sparsity-based Temporal-Spatial Enhancement

Video Super-Resolution Reconstruction Based on Deep Learning and Spatio-Temporal Feature Self-similarity

Global Spatial-Temporal Information-based Residual ConvLSTM for Video Space-Time Super-Resolution

Attention-guided dual spatial-temporal non-local network for video super-resolution

Enhanced Bidirectional Propagation Network for Video Super-Resolution

Deep Plug-and-Play Video Super-Resolution.

Efficient Spatio-Temporal Network with Gated Fusion for Video Super-Resolution

Global-local Fusion Network for Face Super-Resolution

Local-Global Temporal Difference Learning for Satellite Video Super-Resolution