Abstract:Recently, due to higher requirements for satellite video resolution, video super-resolution (VSR) has been extensively studied. However, the following problems have not been effectively resolved: 1) previous satellite VSR methods cannot achieve continuous-scale (integer and noninteger scale) VSR with a single model; 2) satellite video has complex ground and weak textures, which increases the difficulty of capturing motion information. In addition, existing methods adopt a unified alignment path, which leads to a drop in feature alignment accuracy; and 3) during feature fusion, previous methods ignore the correlation of spatiotemporal information in satellite video and cannot make full use of the spatiotemporal information. To address the above problems, in this article, we propose a novel network for continuous-scale satellite VSR (CSVSR). Specifically, first, for effective motion capture and accurate feature alignment, we design a residual-guided and time-aware dynamic routing alignment module, which can use feature residuals to lock motion areas and then dynamically select the corresponding alignment path based on the temporal distance. Second, we proposed a nonlocal mask-based feature fusion module (NMFFM) to exploit the correlation of the spatiotemporal features and complete effective spatiotemporal feature fusion. Third, to make our network adapt to multitask learning, we develop a scale-aware convolutional (SA-Conv) layer, which lets our network dynamically extract scale-adaptive features according to the input scale factors. Finally, we propose a continuous-scale upsampling module with a global feature implicit function (GFIF), which can achieve continuous-scale mapping from features to pixel values. In addition, we carefully design a novel training strategy to optimize our network. Comprehensive experiments verify that the proposed CSVSR has superior reconstruction performance on continuous-scale factors. The code will be available at https://github.com/chongningni/CSVSR.

MoTIF: Learning Motion Trajectories with Local Implicit Neural Functions for Continuous Space-Time Video Super-Resolution

Temporal Modulation Network for Controllable Space-Time Video Super-Resolution

HR-INR: Continuous Space-Time Video Super-Resolution via Event Camera

Space-Time Video Super-resolution with Neural Operator

Space-time video super-resolution via multi-scale feature interpolation and temporal feature fusion

Spatial-Temporal Space Hand-in-Hand: Spatial-Temporal Video Super-Resolution Via Cycle-Projected Mutual Learning

Learning Spatial-Temporal Implicit Neural Representations for Event-Guided Video Super-Resolution

CTVSR: Collaborative Spatial-Temporal Transformer for Video Super-Resolution

STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation.

Space-time Super-Resolution with Motion-Perceptive Deformable Alignment

Deep Plug-and-Play Video Super-Resolution.

Learning for Unconstrained Space-Time Video Super-Resolution

Towards Interpretable Video Super-Resolution Via Alternating Optimization

Global Spatial-Temporal Information-based Residual ConvLSTM for Video Space-Time Super-Resolution

Learning Trajectory-Aware Transformer for Video Super-Resolution.

Video super-resolution via mixed spatial-temporal convolution and selective fusion

Store and Fetch Immediately: Everything is All You Need for Space-Time Video Super-resolution.

Space-time video super-resolution using long-term temporal feature aggregation

Deformable Convolution Alignment and Dynamic Scale-Aware Network for Continuous-Scale Satellite Video Super-Resolution

You Only Align Once: Bidirectional Interaction for Spatial-Temporal Video Super-Resolution

Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling