Abstract:Neuromorphic vision is a bio-inspired technology that has triggered a paradigm shift in the computer vision community and is serving as a key enabler for a wide range of applications. This technology has offered significant advantages, including reduced power consumption, reduced processing needs, and communication speedups. However, neuromorphic cameras suffer from significant amounts of measurement noise. This noise deteriorates the performance of neuromorphic event-based perception and navigation algorithms. In this article, we propose a novel noise filtration algorithm to eliminate events that do not represent real log-intensity variations in the observed scene. We employ a graph neural network (GNN)-driven transformer algorithm, called GNN-Transformer, to classify every active event pixel in the raw stream into real log-intensity variation or noise. Within the GNN, a message-passing framework, referred to as EventConv, is carried out to reflect the spatiotemporal correlation among the events while preserving their asynchronous nature. We also introduce the known-object ground-truth labeling (KoGTL) approach for generating approximate ground-truth labels of event streams under various illumination conditions. KoGTL is used to generate labeled datasets, from experiments recorded in challenging lighting conditions, including moon light. These datasets are used to train and extensively test our proposed algorithm. When tested on unseen datasets, the proposed algorithm outperforms state-of-the-art methods by at least 8.8% in terms of filtration accuracy. Additional tests are also conducted on publicly available datasets (ETH Zürich Color-DAVIS346 datasets) to demonstrate the generalization capabilities of the proposed algorithm in the presence of illumination variations and different motion dynamics. Compared to state-of-the-art solutions, qualitative results verified the superior capability of the proposed algorithm to eliminate noise while preserving meaningful events in the scene.

Harnessing Noise for Materials Differentiation in Computational Neuromorphic Imaging

Neuromorphic Imaging With Joint Image Deblurring and Event Denoising

Event Encryption: Rethinking Privacy Exposure for Neuromorphic Imaging

Non-Uniform Exposure Imaging via Neuromorphic Shutter Control

Noise2Image: Noise-Enabled Static Scene Recovery for Event Cameras

NeuroZoom: Denoising and Super Resolving Neuromorphic Events and Spikes.

Joint Filtering of Intensity Images and Neuromorphic Events for High-Resolution Noise-Robust Imaging

Event Probability Mask (EPM) and Event Denoising Convolutional Neural Network (EDnCNN) for Neuromorphic Cameras

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Ev-GS: Event-based Gaussian splatting for Efficient and Accurate Radiance Field Rendering

Exploiting Noise As a Resource for Computation and Learning in Spiking Neural Networks

Event-Based Neuromorphic Vision for Autonomous Driving: A Paradigm Shift for Bio-Inspired Visual Sensing and Perception.

Neuromorphic Imaging with Super-Resolution

Denoising algorithm based on event camera

Neuromorphic Camera Denoising Using Graph Neural Network-Driven Transformers

EventZoom: Learning to Denoise and Super Resolve Neuromorphic Events

Colorful image reconstruction from neuromorphic event cameras with biologically inspired deep color fusion neural networks

Event Camera Denoising Using Asynchronous Spatio-Temporal Event Denoising Neural Network

Adaptive Event Address Map Denoising for Event Cameras

Computational event-driven vision sensors for in-sensor spiking neural networks