Abstract:Modern consumer cameras usually employ the rolling shutter (RS) mechanism, where images are captured by scanning scenes row-by-row, yielding RS distortions for dynamic scenes. To correct RS distortions, existing methods adopt a fully supervised learning manner, where high framerate global shutter (GS) images should be collected as ground-truth supervision. In this paper, we propose a Self-supervised learning framework for Dual reversed RS distortions Correction (SelfDRSC), where a DRSC network can be learned to generate a high framerate GS video only based on dual RS images with reversed distortions. In particular, a bidirectional distortion warping module is proposed for reconstructing dual reversed RS images, and then a self-supervised loss can be deployed to train DRSC network by enhancing the cycle consistency between input and reconstructed dual reversed RS images. Besides start and end RS scanning time, GS images at arbitrary intermediate scanning time can also be supervised in SelfDRSC, thus enabling the learned DRSC network to generate a high framerate GS video. Moreover, a simple yet effective self-distillation strategy is introduced in self-supervised loss for mitigating boundary artifacts in generated GS images. On synthetic dataset, SelfDRSC achieves better or comparable quantitative metrics in comparison to state-of-the-art methods trained in the full supervision manner. On real-world RS cases, our SelfDRSC can produce high framerate GS videos with finer correction textures and better temporary consistency. The source code and trained models are made publicly available at <a class="link-external link-https" href="https://github.com/shangwei5/SelfDRSC" rel="external noopener nofollow">this https URL</a>. We also provide an implementation in HUAWEI Mindspore at <a class="link-external link-https" href="https://github.com/Hunter-Will/SelfDRSC-mindspore" rel="external noopener nofollow">this https URL</a>.

Self-Supervised Learning for Rolling Shutter Temporal Super-Resolution

Self-supervised Learning to Bring Dual Reversed Rolling Shutter Images Alive

Self-supervised Learning of Event-guided Video Frame Interpolation for Rolling Shutter Frames

SelfDRSC++: Self-Supervised Learning for Dual Reversed Rolling Shutter Correction

Learning Bilateral Cost Volume for Rolling Shutter Temporal Super-Resolution

Context-Aware Video Reconstruction for Rolling Shutter Cameras

Self-Supervised Scene Dynamic Recovery from Rolling Shutter Images and Events

EvUnroll: Neuromorphic Events Based Rolling Shutter Image Correction

Learning Adaptive Warping for Real-World Rolling Shutter Correction

Fast Rolling Shutter Correction in the Wild

Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

UniINR: Event-guided Unified Rolling Shutter Correction, Deblurring, and Interpolation

Single Image Rolling Shutter Removal with Diffusion Models

Self-Supervised Deep Blind Video Super-Resolution

Self-Learned Video Super-Resolution with Augmented Spatial and Temporal Context

Self-Supervised Learning for Real-World Super-Resolution from Dual and Multiple Zoomed Observations

Event-based Shutter Unrolling and Motion Deblurring in Dynamic Scenes

Joint Generative Learning and Super-Resolution For Real-World Camera-Screen Degradation

Joint Generative Learning and Super-Resolution for Real-World Camera-Screen Degradation

Learning for Unconstrained Space-Time Video Super-Resolution