Abstract:Event camera-based driver monitoring is emerging as a pivotal area of research, driven by its significant advantages such as rapid response, low latency, power efficiency, enhanced privacy, and prevention of undersampling. Effective detection of driver distraction is crucial in driver monitoring systems to enhance road safety and reduce accident rates. The integration of an optimized sensor such as Event Camera with an optimized network is essential for maximizing these benefits. This paper introduces the innovative concept of sensing without seeing to detect driver distraction, leveraging computationally efficient spiking neural networks (SNN). To the best of our knowledge, this study is the first to utilize event camera data with spiking neural networks for driver distraction. The proposed Spiking-DD network not only achieve state of the art performance but also exhibit fewer parameters and provides greater accuracy than current event-based methodologies.
What problem does this paper attempt to address?
The paper aims to address the issue of driver distraction detection. Specifically, the researchers propose a method based on Event Camera and Spiking Neural Network (SNN), called Spiking-DD, for real-time detection of driver distraction. This method not only achieves state-of-the-art performance but also has fewer parameters and higher accuracy.
### Background and Motivation
1. **Importance of Road Safety**: Regulatory bodies are increasingly emphasizing the role of advanced Driver Monitoring Systems (DMS) in enhancing road safety. For example, the European Union requires that all new vehicles produced from 2025 must be equipped with DMS.
2. **Limitations of Traditional Methods**: Traditional driver monitoring methods often rely on continuous video streams and complex machine learning algorithms, which are computationally intensive and inefficient for real-time applications.
3. **Advantages of Event Cameras**: Unlike traditional frame-based cameras, event cameras operate asynchronously and generate data only when changes occur in the visual field. This makes the data stream more efficient, low-latency, low-power, and enhances privacy protection.
4. **Suitability of Spiking Neural Networks**: SNNs simulate the way biological neurons communicate through discrete spikes, making them suitable for processing time-series data. They can quickly respond to dynamic changes, making them ideal for real-time detection tasks.
### Method Overview
1. **Data Representation**: The Leaky Integrate-and-Fire (LIF) model is used to convert the event stream into binary spikes.
2. **Network Architecture**: A multi-layer SNN is designed, including an input layer, hidden layers, and an output layer. The input layer encodes raw data into spike sequences, the hidden layers consist of multiple LIF neurons, and the output layer classifies whether the driver is distracted.
3. **Loss Function**: The SpikeRate loss mechanism is used to optimize network performance, which calculates the difference between actual and target spike rates.
### Experiments and Results
1. **Dataset**: The Driver Monitoring Dataset (DMD) was used, containing driving data from 20 subjects in 3 different scenarios.
2. **Experimental Setup**: Experiments were conducted on an Nvidia RTX 2080 Ti GPU using the PyTorch framework, Adam optimizer, with an initial learning rate of 0.1, which was reduced by a factor of 0.1 every 4 epochs, for a total of 30 epochs.
3. **Performance Comparison**: Compared to existing models (such as MobileNetv1+LSTM, VideoSwin-Transformer, 3D-CNN, Submanifold-ResNET), Spiking-DD achieved a high accuracy of 94.40% with fewer parameters.
### Conclusion and Future Work
1. **Conclusion**: The Spiking-DD method outperforms existing models in terms of accuracy and efficiency, making it particularly suitable for real-time deployment. The method also integrates a privacy design framework to protect sensitive driver information.
2. **Future Work**: Further evaluation of the method's real-time capabilities in real-world environments and its performance on neural accelerators (such as Intel's Loihi-2 chip) is planned. Additionally, more extensive testing under different hardware and environmental conditions is needed to fully validate the potential of SNNs.
In summary, the paper proposes an innovative driver distraction detection method based on Event Camera and SNNs, demonstrating its great potential in enhancing road safety.