Abstract:Previous deep learning-based event denoising methods mostly suffer from poor interpretability and difficulty in real-time processing due to their complex architecture designs. In this paper, we propose window-based event denoising, which simultaneously deals with a stack of events while existing element-based denoising focuses on one event each time. Besides, we give the theoretical analysis based on probability distributions in both temporal and spatial domains to improve interpretability. In temporal domain, we use timestamp deviations between processing events and central event to judge the temporal correlation and filter out temporal-irrelevant events. In spatial domain, we choose maximum a posteriori (MAP) to discriminate real-world event and noise, and use the learned convolutional sparse coding to optimize the objective function. Based on the theoretical analysis, we build Temporal Window (TW) module and Soft Spatial Feature Embedding (SSFE) module to process temporal and spatial information separately, and construct a novel multi-scale window-based event denoising network, named MSDNet. The high denoising accuracy and fast running speed of our MSDNet enables us to achieve real-time denoising in complex scenes. Extensive experimental results verify the effectiveness and robustness of our MSDNet. Our algorithm can remove event noise effectively and efficiently and improve the performance of downstream tasks.

What problem does this paper attempt to address?

The problem this paper attempts to address is the shortcomings of existing deep learning-based event denoising methods in terms of interpretability and real-time processing. Specifically, existing methods have poor interpretability due to complex architecture designs and are difficult to achieve real-time processing. Therefore, the authors propose a new Multi-Scale Window Event Denoising Network (WedNet) aimed at improving denoising accuracy and running speed while maintaining good performance. ### Main Contributions 1. **Proposed a new Multi-Scale Window Event Denoising Network (WedNet)**: By processing multiple events through windows instead of processing events one by one, the running speed is significantly improved. 2. **Provided detailed theoretical analysis**: Based on probability distribution, a detailed analysis of how to separate real events from noisy event streams was conducted, with analyses performed in both time and space domains. 3. **Constructed Time Window (TW) module and Soft Spatial Feature Extraction (SSFE) module**: These two modules handle temporal and spatial information respectively, making the algorithm more interpretable. ### Method Overview - **Time Window (TW) module**: By analyzing the timestamp deviation of events, it determines the temporal correlation of events and filters out temporally irrelevant events. - **Soft Spatial Feature Extraction (SSFE) module**: Utilizes convolutional sparse coding to optimize the objective function, distinguishing real events from noise and extracting spatial features. - **Hierarchical Spatial Feature Learning (HSFL) unit**: Gradually abstracts multi-scale local features in a hierarchical manner, better utilizing local spatial correlations to achieve window event denoising. ### Experimental Results Extensive experiments and ablation studies show that WedNet performs excellently in terms of denoising performance and robustness, capable of real-time denoising in complex scenarios. Compared to traditional event denoising methods, WedNet not only has higher denoising accuracy but also comparable running speed. ### Conclusion This paper addresses the shortcomings of existing methods in interpretability and real-time processing by proposing a new Multi-Scale Window Event Denoising Network (WedNet). Through detailed theoretical analysis and modular design, WedNet achieves significant improvements in both denoising performance and running speed.

Fast Window-Based Event Denoising with Spatiotemporal Correlation Enhancement

Spatial-Adaptive Network for Single Image Denoising

Event Camera Denoising Using Asynchronous Spatio-Temporal Event Denoising Neural Network

Real-time Monte Carlo Denoising with Weight Sharing Kernel Prediction Network

Event Stream Denoising Method Based on Spatio-Temporal Density and Time Sequence Analysis

Multitask Sparse Representation Model-Inspired Network for Hyperspectral Image Denoising

Denoising of Event-Based Sensors with Spatial-Temporal Correlation

Deep Parameterized Neural Networks for Hyperspectral Image Denoising

AEDNet: Asynchronous Event Denoising with Spatial-Temporal Correlation among Irregular Data

Denoising Method Based on Salient Region Recognition for the Spatiotemporal Event Stream

Denoising of Event-Based Sensors with Deep Neural Networks

Deep Marginalized Sparse Denoising Auto-Encoder for Image Denoising

Spatio-Temporal Video Denoising Based on Attention Mechanism

Denoising Method for Dynamic Vision Sensor Based on Two-Dimensional Event Density

Real-Time Monte Carlo Denoising With Adaptive Fusion Network

Multi-Scale Spatio-Temporal Memory Network for Lightweight Video Denoising

Probabilistic Undirected Graph Based Denoising Method for Dynamic Vision Sensor

Rapid Response DAS Denoising Method Based on Deep Learning

EventZoom: Learning to Denoise and Super Resolve Neuromorphic Events

Multi-scale Network Toward Real-World Image Denoising.

Hyperspectral Image Denoising Based On Multi-Stream Denoising Network