Abstract:Recently, due to higher requirements for satellite video resolution, video super-resolution (VSR) has been extensively studied. However, the following problems have not been effectively resolved: 1) previous satellite VSR methods cannot achieve continuous-scale (integer and noninteger scale) VSR with a single model; 2) satellite video has complex ground and weak textures, which increases the difficulty of capturing motion information. In addition, existing methods adopt a unified alignment path, which leads to a drop in feature alignment accuracy; and 3) during feature fusion, previous methods ignore the correlation of spatiotemporal information in satellite video and cannot make full use of the spatiotemporal information. To address the above problems, in this article, we propose a novel network for continuous-scale satellite VSR (CSVSR). Specifically, first, for effective motion capture and accurate feature alignment, we design a residual-guided and time-aware dynamic routing alignment module, which can use feature residuals to lock motion areas and then dynamically select the corresponding alignment path based on the temporal distance. Second, we proposed a nonlocal mask-based feature fusion module (NMFFM) to exploit the correlation of the spatiotemporal features and complete effective spatiotemporal feature fusion. Third, to make our network adapt to multitask learning, we develop a scale-aware convolutional (SA-Conv) layer, which lets our network dynamically extract scale-adaptive features according to the input scale factors. Finally, we propose a continuous-scale upsampling module with a global feature implicit function (GFIF), which can achieve continuous-scale mapping from features to pixel values. In addition, we carefully design a novel training strategy to optimize our network. Comprehensive experiments verify that the proposed CSVSR has superior reconstruction performance on continuous-scale factors. The code will be available at https://github.com/chongningni/CSVSR.

Deformable Convolution Alignment and Dynamic Scale-Aware Network for Continuous-Scale Satellite Video Super-Resolution

Deformable Alignment and Scale-Adaptive Feature Extraction Network for Continuous-Scale Satellite Video Super-Resolution.

Video super-resolution with phase-aided deformable alignment network

SAVSR: Arbitrary-Scale Video Super-Resolution Via a Learned Scale-Adaptive Network

Learning a Local-Global Alignment Network for Satellite Video Super-Resolution

Learning Dynamic Scale Awareness and Global Implicit Functions for Continuous-Scale Super-Resolution of Remote Sensing Images.

Scale-Aware Dynamic Network for Continuous-Scale Super-Resolution

A Lightweight Recurrent Aggregation Network for Satellite Video Super-Resolution

Satellite Video Super-Resolution Via Multiscale Deformable Convolution Alignment and Temporal Grouping Projection

Bidirectional Scale-Aware Upsampling Network for Arbitrary-Scale Video Super-Resolution

Grouped Spatio-Temporal Alignment Network for Video Super-Resolution

Hierarchical Feature Aggregation and Self-Learning Network for Remote Sensing Image Continuous-Scale Super-Resolution

A Recurrent Refinement Network for Satellite Video Super-Resolution.

Scale-Adaptive Feature Aggregation for Efficient Space-Time Video Super-Resolution

Multiframe Video Satellite Image Super-Resolution via Attention-Based Residual Learning

S2A: Scale-Attention-Aware Networks for Video Super-Resolution.

Video Super-Resolution Via a Spatio-Temporal Alignment Network.

Multi-Stage Feature Fusion Network for Video Super-Resolution

Deep Plug-and-Play Video Super-Resolution.

Deep Learning Based Super-Resolution of Video Satellite Imagery by the Coupling of Multi-Frame and Single-Frame Models

Multi-scale Non-local Bidirectional Fusion for Video Super-Resolution