Abstract:Understanding driver activity is vital for in-vehicle systems that aim to reduce the incidence of car accidents rooted in cognitive distraction. Automating real-time behavior recognition while ensuring actions classification with high accuracy is however challenging, given the multitude of circumstances surrounding drivers, the unique traits of individuals, and the computational constraints imposed by in-vehicle embedded platforms. Prior work fails to jointly meet these runtime/accuracy requirements and mostly rely on a single sensing modality, which in turn can be a single point of failure. In this paper, we harness the exceptional feature extraction abilities of deep learning and propose a dedicated Interwoven Deep Convolutional Neural Network (InterCNN) architecture to tackle the problem of accurate classification of driver behaviors in real-time. The proposed solution exploits information from multi-stream inputs, i.e., in-vehicle cameras with different fields of view and optical flows computed based on recorded images, and merges through multiple fusion layers abstract features that it extracts. This builds a tight ensembling system, which significantly improves the robustness of the model. In addition, we introduce a temporal voting scheme based on historical inference instances, in order to enhance the classification accuracy. Experiments conducted with a dataset that we collect in a mock-up car environment demonstrate that the proposed InterCNN with MobileNet convolutional blocks can classify 9 different behaviors with 73.97% accuracy, and 5 'aggregated' behaviors with 81.66% accuracy. We further show that our architecture is highly computationally efficient, as it performs inferences within 15 ms, which satisfies the real-time constraints of intelligent cars. Nevertheless, our InterCNN is robust to lossy input, as the classification remains accurat- when two input streams are occluded.

Feature Refinement for Image-Based Driver Action Recognition Via Multi-Scale Attention Convolutional Neural Network

Double Attention Convolutional Neural Network for Driver Action Recognition

Driver Action Recognition Based on Attention Mechanism.

Recongnition of Distracted Driving Behavior Based on Improved Bi-LSTM Model and Attention Mechanism

Vehicle Behavior Recognition using Multi-Stream 3D Convolutional Neural Network

Improving real-time driver distraction detection via constrained attention mechanism

Event-based Driver Distraction Detection and Action Recognition

Video-based Driver Action Recognition Via Hybrid Spatial–temporal Deep Learning Framework

A Pose-Aware Dynamic Weighting Model Using Feature Integration for Driveraction Recognition

Keypoint-enhanced Adaptive Weighting Model with Effective Frequency Channel Attention for Driver Action Recognition

Two-Stream (2+1)D CNN Based on Frame Difference Attention for Driver Behavior Recognition

Driver behavior recognition based on deep convolutional neural networks

Driving Behaviour Recognition from Still Images by Using Multi-Stream Fusion CNN.

Driving Fatigue Detection Based on the Combination of Multi-Branch 3D-CNN and Attention Mechanism

A multi-aware graph convolutional network for driver drowsiness detection

Driver Action Recognition Using Deformable and Dilated Faster R-CNN with Optimized Region Proposals

A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers

Driver Behavior Recognition via Interwoven Deep Convolutional Neural Nets With Multi-Stream Inputs

Driver Distraction Recognition Based on CBAM Attention Mechanism.

Attend and Guide (AG-Net): A Keypoints-driven Attention-based Deep Network for Image Recognition

Multi-level Attention Fusion for Multimodal Driving Maneuver Recognition