Event-based Shutter Unrolling and Motion Deblurring in Dynamic Scenes

Yangguang Wang,Chenxu Jiang,Xu Jia,Yufei Guo,Lei Yu

DOI: https://doi.org/10.1109/lsp.2024.3381894

2024-01-01

IEEE Signal Processing Letters

Abstract:The Rolling Shutter (RS) effect and motion blur are common challenges in images captured by CMOS cameras during dynamic scenes. Inspired by biological vision principles, event cameras capture intensity changes asynchronously with low latency, providing valuable insights into image degradation during exposure. This study addresses the dual challenges of rolling shutter correction and deblurring using event data, merging them into a unified one-stage network. This streamlined approach reduces cumulative errors and inference time compared to traditional two-stage methods. To achieve this, we introduce an Event Representation for Rolling Shutter Deblurring, which explicitly models the conversion relationship between the input RS blurry frame and the latent image using events. To enhance the fusion of image and event information, we present a Time-guided Cross-Modal Attention module. Furthermore, we improve performance by incorporating a Multi-Scale Context-Aware Transformer Block, effectively addressing varying degrees of distortion and blurriness using a multi-scale attention mechanism. Extensive experiments validate that our method outperforms existing state-of-the-art approaches.

engineering, electrical & electronic