Abstract:Driver behavior recognition has become one of the most important tasks for intelligent vehicles. This task, however, is very challenging since the background contents in real-world driving scenarios are often very complex. More critically, the difference between driving behaviors is often very minor, making it extremely difficult to distinguish them. Existing methods often rely only on RGB frames (or skeleton data), which may fail to capture the minor differences between behaviors and appearance information of objects simultaneously and thus fail to achieve promising performance. To address the above issues, in this paper, we propose a bidirectional posture-appearance interaction network (BPAI-Net), which simultaneously considers RGB frames and skeleton (i.e., posture) data for driver behavior recognition. Specifically, we propose a posture-guided convolutional neural network (PG-CNN) and an appearance-guided graph convolutional network (AG-GCN) to extract appearance and posture features, respectively. To exploit the complementary information between appearance and posture, we use the appearance features from PG-CNN for guiding AG-GCN to exploit the contextual information (e.g., nearby objects) to enhance posture features. Then, we use the enhanced posture features from AG-GCN to help PG-CNN focus on critical local areas of video frames that are related to driver behaviors. In this sense, we are able to use the interaction between two modalities to extract more discriminative features and thus improve the recognition accuracy. Experimental results on Drive&Act dataset show that our method outperforms state-of-the-art methods by a large margin (67.83% vs. 63.64%). Furthermore, we collect a bus driver behavior recognition dataset and yield consistent performance gain against baseline methods, demonstrating the effectiveness of our method in real-world applications. The source code and trained models are available at github.com/SCUT-AILab/BPAI-Net/.

Recognition of Driving Postures by Nonsubsampled Contourlet Transform and K-Nearest Neighbor Classifier.

Driving Behaviour Characterisation by Using Phase‐space Reconstruction and Pre‐trained Convolutional Neural Network

An Algorithm for Distracted Driving Recognition Based on Pose Features and an Improved KNN

End-to-End Driving Activities and Secondary Tasks Recognition Using Deep Convolutional Neural Network and Transfer Learning

A Proactive Recognition System for Detecting Commercial Vehicle Driver’s Distracted Behavior

Partially Occluded Head Posture Estimation for 2D Images Using Pyramid HoG Features

Driving Profile Modeling and Recognition Based on Soft Computing Approach

Bidirectional Posture-Appearance Interaction Network for Driver Behavior Recognition

Driver Evaluation and Identification Based on Driving Behavior Data

Application of Deep Learning Methods in Driver Behavior Recognition

A Refined Non-Driving Activity Classification Using a Two-Stream Convolutional Neural Network

A Hybrid Deep Learning Model for Recognizing Actions of Distracted Drivers

Cross-Dataset Transfer Driver Expression Recognition via Global Discriminative and Local Structure Knowledge Exploitation in Shared Projection Subspace

Human-Factors-in-Driving-Loop: Driver Identification and Verification via a Deep Learning Approach using Psychological Behavioral Data

Driver Status Recognition by Neighborhood Covering Rules

Driver Distraction Detection Method Based on Continuous Head Pose Estimation

An fNIRS labeling image feature-based customized driving fatigue detection method

A Unified Multi-scale and Multi-task Learning Framework for Driver Behaviors Reasoning

Attention Monitoring and Hazard Assessment with Bio-Sensing and Vision: Empirical Analysis Utilizing CNNs on the KITTI Dataset

WiFi-Based Gesture Recognition for Vehicular Infotainment System—An Integrated Approach