Abstract:This paper explores Deep Learning (DL) methods that are used or have the potential to be used for traffic video analysis, emphasizing driving safety for both Autonomous Vehicles (AVs) and human-operated vehicles. We present a typical processing pipeline, which can be used to understand and interpret traffic videos by extracting operational safety metrics and providing general hints and guidelines to improve traffic safety. This processing framework includes several steps, including video enhancement, video stabilization, semantic and incident segmentation, object detection and classification, trajectory extraction, speed estimation, event analysis, modeling and anomaly detection. Our main goal is to guide traffic analysts to develop their own custom-built processing frameworks by selecting the best choices for each step and offering new designs for the lacking modules by providing a comparative analysis of the most successful conventional and DL-based algorithms proposed for each step. We also review existing open-source tools and public datasets that can help train DL models. To be more specific, we review exemplary traffic problems and mentioned requires steps for each problem. Besides, we investigate connections to the closely related research areas of drivers' cognition evaluation, Crowd-sourcing-based monitoring systems, Edge Computing in roadside infrastructures, Automated Driving Systems (ADS)-equipped vehicles, and highlight the missing gaps. Finally, we review commercial implementations of traffic monitoring systems, their future outlook, and open problems and remaining challenges for widespread use of such systems.

Vehicle-Related Scene Understanding Using Deep Learning

Moving vehicle tracking and scene understanding: A hybrid approach

Depth Estimation of Traffic Scenes from Image Sequence Using Deep Learning.

A Deep Learning Method for Lane Changing Situation Assessment and Decision Making.

A system of vision sensor based deep neural networks for complex driving scene analysis in support of crash risk assessment and prevention

Blind Spot Monitoring Using Deep Learning.

Enhancing Transportation Systems Via Deep Learning: A Survey

Scene Detection Classification and Tracking for Self-Driven Vehicle

Traffic Sign Recognition from Digital Images by Using Deep Learning

Research on the Application of Deep Learning In Autonomous Vehicles

A Survey of 3D Point Cloud and Deep Learning-Based Approaches for Scene Understanding in Autonomous Driving

Traffic-light Sign Recognition Using Capsule Network

Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review

An Empirical Evaluation of Deep Learning on Highway Driving

A Survey on Theories and Applications for Self-Driving Cars Based on Deep Learning Methods

Object Detection Using Deep Learning Methods in Traffic Scenarios

A survey to applications of deep learning in autonomous driving

Deep Learning for Autonomous Driving System

A Scene Understanding Network Based on Driving Scene

Research on the Application of Computer Vision Based on Deep Learning in Autonomous Driving Technology

Deep learning-based scene understanding for autonomous robots: a survey