Abstract:The best performing learning algorithms devised for event cameras work by first converting events into dense representations that are then processed using standard CNNs. However, these steps discard both the sparsity and high temporal resolution of events, leading to high computational burden and latency. For this reason, recent works have adopted Graph Neural Networks (GNNs), which process events as ``static" spatio-temporal graphs, which are inherently "sparse". We take this trend one step further by introducing Asynchronous, Event-based Graph Neural Networks (AEGNNs), a novel event-processing paradigm that generalizes standard GNNs to process events as ``evolving" spatio-temporal graphs. AEGNNs follow efficient update rules that restrict recomputation of network activations only to the nodes affected by each new event, thereby significantly reducing both computation and latency for event-by-event processing. AEGNNs are easily trained on synchronous inputs and can be converted to efficient, "asynchronous" networks at test time. We thoroughly validate our method on object classification and detection tasks, where we show an up to a 11-fold reduction in computational complexity (FLOPs), with similar or even better performance than state-of-the-art asynchronous methods. This reduction in computation directly translates to an 8-fold reduction in computational latency when compared to standard GNNs, which opens the door to low-latency event-based processing.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the high computational complexity and latency in existing event - camera processing methods. Specifically: 1. **Limitations of existing methods**: - **Image representation**: Many existing methods first convert events into a dense representation (such as an image) and then use a standard convolutional neural network (CNN) for processing. Although this method has good performance, it discards the sparsity and high - time resolution of events, resulting in a heavy computational burden and high latency. - **Batch - processing methods**: Some methods reduce the amount of computation by batch - processing events, but this still requires recomputing all network activations whenever a new event arrives, leading to unnecessary repeated computations. - **Filter - based methods**: These methods can provide low - latency predictions, but usually rely on hand - designed filter equations and are difficult to extend to more complex tasks such as object detection and classification. - **Spiking neural networks (SNN)**: Although they attempt to learn rules from data, they are still in the early stages and lack general and robust learning rules, making it difficult to solve complex high - level tasks. 2. **Solutions proposed in the paper**: - **Asynchronous event graph neural network (AEGNN)**: The paper introduces a new neural network architecture - AEGNN, which is used to process event streams in a sparse and asynchronous manner. AEGNN models events as spatio - temporal graphs and only computes and updates the local sub - graphs affected by new events, thereby significantly reducing the amount of computation and latency. - **Efficient update rules**: AEGNN uses recursive update rules, allowing the network to be deployed in an asynchronous mode at test time and generate the same output as synchronous training. These rules are applicable to most existing graph neural network architectures. - **Preserving time and space information**: AEGNN not only takes advantage of the sparsity of events but also preserves their time information, which is often ignored in other methods. 3. **Experimental results**: - **Computational complexity**: The computational complexity of AEGNN is 11 times lower than that of the state - of - the - art asynchronous methods, while achieving state - of - the - art performance in object recognition and detection tasks. - **Latency**: Compared with standard graph neural networks, the computational latency of AEGNN is reduced by 8 times, which paves the way for low - latency event processing. - **Practical applications**: AEGNN performs excellently on the N - Caltech101 and N - Cars datasets. Especially on the N - Cars dataset, its computational complexity is 20 times lower than that of the second - best asynchronous method while achieving the highest accuracy. In conclusion, this paper aims to solve the high computational complexity and latency problems in existing event - camera processing methods by proposing AEGNN while maintaining or improving task performance.

AEGNN: Asynchronous Event-based Graph Neural Networks

Graph-based Asynchronous Event Processing for Rapid Object Recognition

Pushing the Limits of Asynchronous Graph-based Object Detection with Event Cameras

Event-based Object Detection Using Graph Neural Networks

EvGNN: An Event-driven Graph Neural Network Accelerator for Edge Vision

Event-driven Spiking Neural Networks with Spike-Based Learning

EDeNN: Event Decay Neural Networks for low latency vision

ASIE: An Asynchronous SNN Inference Engine for AER Events Processing

Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision

Point-Voxel Absorbing Graph Representation Learning for Event Stream Based Recognition

Embedded Graph Convolutional Networks for Real-Time Event Data Processing on SoC FPGAs

Instant Graph Neural Networks for Dynamic Graphs

Scalable Spatiotemporal Graph Neural Networks

Graph Neural Networks Inspired by Classical Iterative Algorithms

Graph Neural Networks Gone Hogwild

EventF2S: Asynchronous and Sparse Spiking AER Framework using Neuromorphic-Friendly Algorithm

EAS-SNN: End-to-End Adaptive Sampling and Representation for Event-based Detection with Recurrent Spiking Neural Networks

InkStream: Real-time GNN Inference on Streaming Graphs via Incremental Update

Learning Neural Point Processes with Latent Graphs

Optimising Graph Representation for Hardware Implementation of Graph Convolutional Networks for Event-based Vision

Asynchronous Bioplausible Neuron for SNN for Event Vision