Abstract:Existing video super-resolution (VSR) methods generally adopt a recurrent propagation network to extract spatio-temporal information from the entire video sequences, exhibiting impressive performance. However, the key components in recurrent-based VSR networks significantly impact model efficiency, e.g., the alignment module occupies a substantial portion of model parameters, while the bidirectional propagation mechanism significantly amplifies the inference time. Consequently, developing a compact and efficient VSR method that can be deployed on resource-constrained devices, e.g., smartphones, remains challenging. To this end, we propose a cascaded temporal updating network (CTUN) for efficient VSR. We first develop an implicit cascaded alignment module to explore spatio-temporal correspondences from adjacent frames. Moreover, we propose a unidirectional propagation updating network to efficiently explore long-range temporal information, which is crucial for high-quality video reconstruction. Specifically, we develop a simple yet effective hidden updater that can leverage future information to update hidden features during forward propagation, significantly reducing inference time while maintaining performance. Finally, we formulate all of these components into an end-to-end trainable VSR network. Extensive experimental results show that our CTUN achieves a favorable trade-off between efficiency and performance compared to existing methods. Notably, compared with BasicVSR, our method obtains better results while employing only about 30% of the parameters and running time. The source code and pre-trained models will be available at <a class="link-external link-https" href="https://github.com/House-Leo/CTUN" rel="external noopener nofollow">this https URL</a>.

Unified Video Reconstruction for Rolling Shutter and Global Shutter Cameras

Context-Aware Video Reconstruction for Rolling Shutter Cameras

Ctrl-VIO: Continuous-Time Visual-Inertial Odometry for Rolling Shutter Cameras

UniINR: Event-guided Unified Rolling Shutter Correction, Deblurring, and Interpolation

Deep Shutter Unrolling Network

Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature

UniGaussian: Driving Scene Reconstruction from Multiple Camera Models via Unified Gaussian Representations

Unfolding Framework with Prior of Convolution-Transformer Mixture and Uncertainty Estimation for Video Snapshot Compressive Imaging

Event-based Shutter Unrolling and Motion Deblurring in Dynamic Scenes

Self-Supervised Learning for Rolling Shutter Temporal Super-Resolution

Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction

Direct Sparse Odometry with Rolling Shutter

UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer

UniFormerV2: Unlocking the Potential of Image ViTs for Video Understanding

Self-supervised Learning of Event-guided Video Frame Interpolation for Rolling Shutter Frames

Sampling-Priors-Augmented Deep Unfolding Network for Robust Video Compressive Sensing

MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds

UniDrive: Towards Universal Driving Perception Across Camera Configurations

Fast Rolling Shutter Correction in the Wild

Rolling-Shutter Modelling for Direct Visual-Inertial Odometry

Cascaded Temporal Updating Network for Efficient Video Super-Resolution