Abstract:Autonomous driving demands an integrated approach that encompasses perception, prediction, and planning, all while operating under strict energy constraints to enhance scalability and environmental sustainability. We present Spiking Autonomous Driving (SAD), the first unified Spiking Neural Network (SNN) to address the energy challenges faced by autonomous driving systems through its event-driven and energy-efficient nature. SAD is trained end-to-end and consists of three main modules: perception, which processes inputs from multi-view cameras to construct a spatiotemporal bird's eye view; prediction, which utilizes a novel dual-pathway with spiking neurons to forecast future states; and planning, which generates safe trajectories considering predicted occupancy, traffic rules, and ride comfort. Evaluated on the nuScenes dataset, SAD achieves competitive performance in perception, prediction, and planning tasks, while drawing upon the energy efficiency of SNNs. This work highlights the potential of neuromorphic computing to be applied to energy-efficient autonomous driving, a critical step toward sustainable and safety-critical automotive technology. Our code is available at \url{<a class="link-external link-https" href="https://github.com/ridgerchu/SAD" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

The paper aims to address the energy efficiency issues in autonomous driving systems by leveraging the unique properties of Spiking Neural Networks (SNNs). Specifically, the research proposes a unified spiking neural network model called "Spiking Autonomous Driving" (SAD) to tackle the energy challenges faced by autonomous driving systems in perception, prediction, and planning. Key features of the SAD model include: 1. **Event-Driven and Energy Efficiency**: The SAD model is based on the event-driven nature of SNNs, which allows it to process information efficiently and reduce energy consumption. This is crucial for enhancing the scalability and environmental sustainability of autonomous driving systems. 2. **End-to-End Training**: The SAD model is designed as an end-to-end solution, meaning it can simultaneously handle perception inputs, predict future states, and generate safe driving paths, all integrated within a single model. 3. **Three Main Modules**: The SAD model consists of three main parts: - **Perception Module**: Processes inputs from multi-view cameras and constructs a Bird’s Eye View (BEV) to better understand the surrounding environment. - **Prediction Module**: Utilizes a novel dual-channel structure combined with spiking neurons to predict future states. - **Planning Module**: Generates safe driving trajectories based on predicted occupancy, traffic rules, and ride comfort. 4. **Experimental Validation**: Evaluations on the nuScenes dataset demonstrate that the SAD model achieves performance levels comparable to traditional deep learning methods in perception, prediction, and planning tasks, while significantly improving energy efficiency. In summary, the goal of this research is to showcase the potential of SNNs in the low-power autonomous driving domain, particularly in maintaining high efficiency and accuracy when handling complex tasks.

Autonomous Driving with Spiking Neural Networks

Exploring Deep Spiking Neural Networks for Automated Driving Applications

Autonomous Driving using Spiking Neural Networks on Dynamic Vision Sensor Data: A Case Study of Traffic Light Change Detection

SpikiLi: A Spiking Simulation of LiDAR based Real-time Object Detection for Autonomous Driving

Automotive Object Detection via Learning Sparse Events by Spiking Neurons

Temporal Pulses Driven Spiking Neural Network for Time and Power Efficient Object Recognition in Autonomous Driving

Spiking Neural Networks for Real-Time Pedestrian Street-Crossing Detection Using Dynamic Vision Sensors in Simulated Adverse Weather Conditions

Towards Energy-Preserving Natural Language Understanding with Spiking Neural Networks

A Methodology to Study the Impact of Spiking Neural Network Parameters considering Event-Based Automotive Data

Deep SCNN-based Real-time Object Detection for Self-driving Vehicles Using LiDAR Temporal Data

SNN4Agents: A Framework for Developing Energy-Efficient Embodied Spiking Neural Networks for Autonomous Agents

Energy-Efficient and Intelligent ISAC in V2X Networks with Spiking Neural Networks-Driven DRL

FastSpiker: Enabling Fast Training for Spiking Neural Networks on Event-based Data through Learning Rate Enhancements for Autonomous Embedded Systems

Pedestrian intention prediction in Adverse Weather Conditions with Spiking Neural Networks and Dynamic Vision Sensors

Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology

PT-Spike: A Precise-Time-Dependent Single Spike Neuromorphic Architecture with Efficient Supervised Learning

Spiking Neural Network for Fourier Transform and Object Detection for Automotive Radar

Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network

Spike-BRGNet: Efficient and Accurate Event-based Semantic Segmentation with Boundary Region-Guided Spiking Neural Networks

Spiking-DD: Neuromorphic Event Camera based Driver Distraction Detection with Spiking Neural Network

Autonomous driving controllers with neuromorphic spiking neural networks