Abstract:Understanding driver activity is vital for in-vehicle systems that aim to reduce the incidence of car accidents rooted in cognitive distraction. Automating real-time behavior recognition while ensuring actions classification with high accuracy is however challenging, given the multitude of circumstances surrounding drivers, the unique traits of individuals, and the computational constraints imposed by in-vehicle embedded platforms. Prior work fails to jointly meet these runtime/accuracy requirements and mostly rely on a single sensing modality, which in turn can be a single point of failure. In this paper, we harness the exceptional feature extraction abilities of deep learning and propose a dedicated Interwoven Deep Convolutional Neural Network (InterCNN) architecture to tackle the problem of accurate classification of driver behaviors in real-time. The proposed solution exploits information from multi-stream inputs, i.e., in-vehicle cameras with different fields of view and optical flows computed based on recorded images, and merges through multiple fusion layers abstract features that it extracts. This builds a tight ensembling system, which significantly improves the robustness of the model. In addition, we introduce a temporal voting scheme based on historical inference instances, in order to enhance the classification accuracy. Experiments conducted with a dataset that we collect in a mock-up car environment demonstrate that the proposed InterCNN with MobileNet convolutional blocks can classify 9 different behaviors with 73.97% accuracy, and 5 'aggregated' behaviors with 81.66% accuracy. We further show that our architecture is highly computationally efficient, as it performs inferences within 15 ms, which satisfies the real-time constraints of intelligent cars. Nevertheless, our InterCNN is robust to lossy input, as the classification remains accurat- when two input streams are occluded.

Toward Driver Intention Prediction for Intelligent Vehicles: A Deep Learning Approach

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

Driver Intention Anticipation Based on In-Cabin and Driving Scene Monitoring

A Hybrid Approach for Turning Intention Prediction Based on Time Series Forecasting and Deep Learning

Driver Intent-Based Intersection Autonomous Driving Collision Avoidance Reinforcement Learning Algorithm

A Data-Driven Approach to Traffic Vehicle Intent Recognition and Trajectory Prediction

Intention-Aware Vehicle Trajectory Prediction Based on Spatial-Temporal Dynamic Attention Network for Internet of Vehicles.

Driver intention prediction based on multi-dimensional cross-modality information interaction

Spatiotemporal Feature Enhancement Aids the Driving Intention Inference of Intelligent Vehicles

DeepTake: Prediction of Driver Takeover Behavior using Multimodal Data

DeepSignals: Predicting Intent of Drivers Through Visual Signals

DIPNet: Driver intention prediction for a safe takeover transition in autonomous vehicles

Driving Maneuvers Prediction Based on Cognition-Driven and Data-Driven Method.

Deep learning approach for unified recognition of driver speed and lateral intentions using naturalistic driving data

Brain4Cars: Car That Knows Before You Do via Sensory-Fusion Deep Learning Architecture

An ensemble deep learning approach for driver lane change intention inference

Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks

Driver Behavior Recognition via Interwoven Deep Convolutional Neural Nets With Multi-Stream Inputs

Convolutional neural network-based intention forecasting and lane change path predicting of the human driver

Deep learning approach for accurate and stable recognition of driver's lateral intentions using naturalistic driving data

Local and Global Contextual Features Fusion for Pedestrian Intention Prediction