Abstract:Event cameras have been successfully applied to visual place recognition (VPR) tasks by using deep artificial neural networks (ANNs) in recent years. However, previously proposed deep ANN architectures are often unable to harness the abundant temporal information presented in event streams. In contrast, deep spiking networks exhibit more intricate spatiotemporal dynamics and are inherently well-suited to process sparse asynchronous event streams. Unfortunately, directly inputting temporal-dense event volumes into the spiking network introduces excessive time steps, resulting in prohibitively high training costs for large-scale VPR tasks. To address the aforementioned issues, we propose a novel deep spiking network architecture called Spike-EVPR for event-based VPR tasks. First, we introduce two novel event representations tailored for SNN to fully exploit the spatio-temporal information from the event streams, and reduce the video memory occupation during training as much as possible. Then, to exploit the full potential of these two representations, we construct a Bifurcated Spike Residual Encoder (BSR-Encoder) with powerful representational capabilities to better extract the high-level features from the two event representations. Next, we introduce a Shared & Specific Descriptor Extractor (SSD-Extractor). This module is designed to extract features shared between the two representations and features specific to each. Finally, we propose a Cross-Descriptor Aggregation Module (CDA-Module) that fuses the above three features to generate a refined, robust global descriptor of the scene. Our experimental results indicate the superior performance of our Spike-EVPR compared to several existing EVPR pipelines on Brisbane-Event-VPR and DDD20 datasets, with the average Recall@1 increased by 7.61% on Brisbane and 13.20% on DDD20.

An Event-based Feature Representation Method for Event Stream Classification Using Deep Spiking Neural Networks

Spiking Neural Network Recognition Method Based on Dynamic Visual Motion Features

Event-driven Spiking Neural Networks with Spike-Based Learning

An Event-based Categorization Model Using Spatio-temporal Features in a Spiking Neural Network.

Event Stream Learning Using Spatio-Temporal Event Surface

RSNN: Recurrent Spiking Neural Networks for Dynamic Spatial-Temporal Information Processing

Event-based Action Recognition Using Motion Information and Spiking Neural Networks

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Deep Representation Via Convolutional Neural Network for Classification of Spatiotemporal Event Streams

Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks

CSNN: an Augmented Spiking Based Framework with Perceptron-Inception

An Event-Driven Categorization Model for AER Image Sensors Using Multispike Encoding and Learning

An Event-Driven Computational System With Spiking Neurons For Object Recognition

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Spike-EVPR: Deep Spiking Residual Network with Cross-Representation Aggregation for Event-Based Visual Place Recognition

[A bio-inspired hierarchical spiking neural network with biological synaptic plasticity for event camera object recognition].

Ultra-low Latency Spiking Neural Networks with Spatio-Temporal Compression and Synaptic Convolutional Block

Multi-scale Harmonic Mean Time Surfaces for Event-based Object Classification

An Event-Driven Object Recognition Model Using Activated Connected Domain Detection

Spiking Neural Network as Adaptive Event Stream Slicer

Representation Learning using Event-based STDP