Abstract:The autonomous driving technology based on deep reinforcement learning (DRL) has been confirmed as one of the most cutting-edge research fields worldwide. The agent is enabled to achieve the goal of making independent decisions by interacting with the environment and learning driving strategies based on the feedback from the environment. This technology has been widely used in end-to-end driving tasks. However, this field faces several challenges. First, developing real vehicles is expensive, time-consuming, and risky. To further expedite the testing, verification, and iteration of end-to-end deep reinforcement learning algorithms, a joint simulation development and validation platform was designed and implemented in this study based on VTD–CarSim and the Tensorflow deep learning framework, and research work was conducted based on this platform. Second, sparse reward signals can cause problems (e.g., a low-sample learning rate). It is imperative for the agent to be capable of navigating in an unfamiliar environment and driving safely under a wide variety of weather or lighting conditions. To address the problem of poor generalization ability of the agent to unknown scenarios, a deep deterministic policy gradient (DDPG) decision-making and planning method was proposed in this study in accordance with a multi-task fusion strategy. The main task based on DRL decision-making planning and the auxiliary task based on image semantic segmentation were cross-fused, and part of the network was shared with the main task to reduce the possibility of model overfitting and improve the generalization ability. As indicated by the experimental results, first, the joint simulation development and validation platform built in this study exhibited prominent versatility. Users were enabled to easily substitute any default module with customized algorithms and verify the effectiveness of new functions in enhancing overall performance using other default modules of the platform. Second, the deep reinforcement learning strategy based on multi-task fusion proposed in this study was competitive. Its performance was better than other DRL algorithms in certain tasks, which improved the generalization ability of the vehicle decision-making planning algorithm.

Camera and LiDAR Fusion for On-road Vehicle Tracking with Reinforcement Learning

A Long-focus Camera and Telemetry LiDAR Calibration for Rapid Transit Equipment.

Multi-Stage Residual Fusion Network for LIDAR-Camera Road Detection

Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

Radar Camera Fusion via Representation Learning in Autonomous Driving

Real time object detection using LiDAR and camera fusion for autonomous driving

Deep LiDAR-Radar-Visual Fusion for Object Detection in Urban Environments

Joint Optimization of Sensing, Decision-making and Motion-controlling for Autonomous Vehicles: A Deep Reinforcement Learning Approach

Reinforcement-Tracking: An Effective Trajectory Tracking and Navigation Method for Autonomous Urban Driving

Multi-Camera Object Fusion Tracking Model for Autonomous Driving.

Autonomous Driving using Residual Sensor Fusion and Deep Reinforcement Learning

Lvio-Fusion: A Self-adaptive Multi-sensor Fusion SLAM Framework Using Actor-critic Method

Exploiting Multi-Modal Fusion for Urban Autonomous Driving Using Latent Deep Reinforcement Learning

Multi-View Fusion of Sensor Data for Improved Perception and Prediction in Autonomous Driving

Multi-AUV Assisted Seamless Underwater Target Tracking Relying on Deep Learning and Reinforcement Learning

An Efficient Probabilistic Solution to Mapping Errors in LiDAR-Camera Fusion for Autonomous Vehicles

Vehicle Trajectory Estimation Based on Fusion of Visual Motion Features and Deep Learning

Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors

LFP: Efficient and Accurate End-to-End Lane-Level Planning via Camera-LiDAR Fusion

A Multi-Task Fusion Strategy-Based Decision-Making and Planning Method for Autonomous Driving Vehicles

Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection