Abstract:The multitarget vehicle tracking and motion state estimation are crucial for controlling the host vehicle accurately and preventing collisions. However, current multitarget tracking methods are inconvenient to deal with multivehicle issues due to the dynamically complex driving environment. Driving environment perception systems, as an indispensable component of intelligent vehicles, have the potential to solve this problem from the perspective of image processing. Thus, this study proposes a novel driving environment perception system of intelligent vehicles by using deep learning methods to track multitarget vehicles and estimate their motion states. Firstly, a panoramic segmentation neural network that supports end-to-end training is designed and implemented, which is composed of semantic segmentation and instance segmentation. A depth calculation model of the driving environment is established by adding a depth estimation branch to the feature extraction and fusion module of the panoramic segmentation network. These deep neural networks are trained and tested in the Mapillary Vistas Dataset and the Cityscapes Dataset, and the results showed that these methods performed well with high recognition accuracy. Then, Kalman filtering and Hungarian algorithm are used for the multitarget vehicle tracking and motion state estimation. The effectiveness of this method is tested by a simulation experiment, and results showed that the relative relation (i.e., relative speed and distance) between multiple vehicles can be estimated accurately. The findings of this study can contribute to the development of intelligent vehicles to alert drivers to possible danger, assist drivers’ decision-making, and improve traffic safety.

A Real-Time Traffic Environmental Perception Algorithm Fusing Stereo Vision and Deep Network.

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

Object Detection Using Multi-Sensor Fusion Based on Deep Learning

Detection-level Fusion for Multi-Object Perception in Dense Traffic Environment

3D object detection and state estimation method based on stereo vision and LIDAR fusion

A Vehicle Tracking Algorithm Combining Detector and Tracker

Multitarget Vehicle Tracking and Motion State Estimation Using a Novel Driving Environment Perception System of Intelligent Vehicles

Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

Collaborative processing and data optimization of environmental perception technologies for autonomous vehicles

Real-time tracking-by-detection Framework for Traffic Applications via Deep Learning based Convolutional Neural Network

Real-Time Evaluation of Perception Uncertainty and Validity Verification of Autonomous Driving

Real-time vehicle detection and tracking based one traffic scene analysis

Real time object detection using LiDAR and camera fusion for autonomous driving

Deep LiDAR-Radar-Visual Fusion for Object Detection in Urban Environments

Multi-Camera Object Fusion Tracking Model for Autonomous Driving.

A real-time dense stereo matching method for critical environment sensing in autonomous driving

Lightweight Real-Time Object Detection via Enhanced Global Perception and Intra-Layer Interaction for Complex Traffic Scenarios

Environment Perception Framework Fusing Multi-Object Tracking, Dynamic Occupancy Grid Maps and Digital Maps

AdvNet: Multi-Task Fusion of Object Detection and Semantic Segmentation

A Robust Real-Time Computing-based Environment Sensing System for Intelligent Vehicle

An Optimized Multi-sensor Fused Object Detection Method for Intelligent Vehicles.