Abstract:Driver behavior recognition has become one of the most important tasks for intelligent vehicles. This task, however, is very challenging since the background contents in real-world driving scenarios are often very complex. More critically, the difference between driving behaviors is often very minor, making it extremely difficult to distinguish them. Existing methods often rely only on RGB frames (or skeleton data), which may fail to capture the minor differences between behaviors and appearance information of objects simultaneously and thus fail to achieve promising performance. To address the above issues, in this paper, we propose a bidirectional posture-appearance interaction network (BPAI-Net), which simultaneously considers RGB frames and skeleton (i.e., posture) data for driver behavior recognition. Specifically, we propose a posture-guided convolutional neural network (PG-CNN) and an appearance-guided graph convolutional network (AG-GCN) to extract appearance and posture features, respectively. To exploit the complementary information between appearance and posture, we use the appearance features from PG-CNN for guiding AG-GCN to exploit the contextual information (e.g., nearby objects) to enhance posture features. Then, we use the enhanced posture features from AG-GCN to help PG-CNN focus on critical local areas of video frames that are related to driver behaviors. In this sense, we are able to use the interaction between two modalities to extract more discriminative features and thus improve the recognition accuracy. Experimental results on Drive&Act dataset show that our method outperforms state-of-the-art methods by a large margin (67.83% vs. 63.64%). Furthermore, we collect a bus driver behavior recognition dataset and yield consistent performance gain against baseline methods, demonstrating the effectiveness of our method in real-world applications. The source code and trained models are available at github.com/SCUT-AILab/BPAI-Net/.

The Research on Detection and Recognition of Driving Posture

Saliency-guided Improvement for Hand Posture Detection and Recognition.

Driving Behaviour Characterisation by Using Phase‐space Reconstruction and Pre‐trained Convolutional Neural Network

Detection of Safety Features of Drivers Based on Image Processing

Estimation of Head Orientation for Safe Driving

A Proactive Recognition System for Detecting Commercial Vehicle Driver’s Distracted Behavior

Research on key technologies of intelligent transportation based on image recognition and anti-fatigue driving

Driver Evaluation and Identification Based on Driving Behavior Data

Driver behavior recognition based on deep convolutional neural networks

Driver Distraction Detection Method Based on Continuous Head Pose Estimation

Research on driver posture recognition and prediction in vehicle

Recognition and prediction of driver’s whole body posture model

Visual Recognition of traffic police gestures with convolutional pose machine and handcrafted features

Bidirectional Posture-Appearance Interaction Network for Driver Behavior Recognition

A Driver Activity Dataset with Multiple RGB-D Cameras and Mmwave Radars

An fNIRS labeling image feature-based customized driving fatigue detection method

Driving Behavior Tracking and Recognition Based on Multisensors Data Fusion

A Survey of Human Action Recognition and Posture Prediction

Recognition of Drivers' Physiological States Based on Body-Seat Pressure Distribution

End-to-End Driving Activities and Secondary Tasks Recognition Using Deep Convolutional Neural Network and Transfer Learning