Abstract:Event cameras differ from conventional RGB cameras in that they produce asynchronous data sequences. While RGB cameras capture every frame at a fixed rate, event cameras only capture changes in the scene, resulting in sparse and asynchronous data output. Despite the fact that event data carries useful information that can be utilized in motion deblurring of RGB cameras, integrating event and image information remains a challenge. Recent state-of-the-art CNN-based deblurring solutions produce multiple 2-D event frames based on the accumulation of event data over a time period. In most of these techniques, however, the number of event frames is fixed and predefined, which reduces temporal resolution drastically, particularly for scenarios when fast-moving objects are present or when longer exposure times are required. It is also important to note that recent modern cameras (e.g., cameras in mobile phones) dynamically set the exposure time of the image, which presents an additional problem for networks developed for a fixed number of event frames. A Long Short-Term Memory (LSTM)-based event feature extraction module has been developed for addressing these challenges, which enables us to use a dynamically varying number of event frames. Using these modules, we constructed a state-of-the-art deblurring network, Deformable Convolutions and LSTM-based Flexible Event Frame Fusion Network (DLEFNet). It is particularly useful for scenarios in which exposure times vary depending on factors such as lighting conditions or the presence of fast-moving objects in the scene. It has been demonstrated through evaluation results that the proposed method can outperform the existing state-of-the-art networks for deblurring task in synthetic and real-world data sets.

Motion-Oriented Hybrid Spiking Neural Networks for Event-Based Motion Deblurring

MEFNet: Multi-scale Event Fusion Network for Motion Deblurring

Event-Based Fusion for Motion Deblurring with Cross-modal Attention

Beyond Camera Motion Removing: How to Handle Outliers in Deblurring

Motion Deblurring via Spatial-Temporal Collaboration of Frames and Events

Event-based Action Recognition Using Motion Information and Spiking Neural Networks

Learning Event-Based Motion Deblurring

Learning for Motion Deblurring with Hybrid Frames and Events

Spiking Neural Network Recognition Method Based on Dynamic Visual Motion Features

Motion Aware Event Representation-Driven Image Deblurring

An Event-based Feature Representation Method for Event Stream Classification Using Deep Spiking Neural Networks

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Deformable Convolutions and LSTM-based Flexible Event Frame Fusion Network for Motion Deblurring

Event-Driven Video Restoration with Spiking-Convolutional Architecture

Event-Based Multimodal Spiking Neural Network with Attention Mechanism

Spike-guided Motion Deblurring with Unknown Modal Spatiotemporal Alignment

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Event-guided Multi-patch Network with Self-supervision for Non-uniform Motion Deblurring

Unifying Motion Deblurring and Frame Interpolation with Events

Image De-occlusion via Event-enhanced Multi-modal Fusion Hybrid Network