The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review

Steffen Hagedorn,Marcel Hallgarten,Martin Stoll,Alexandru Condurache
2024-09-11
Abstract:Automated driving has the potential to revolutionize personal, public, and freight mobility. Beside accurately perceiving the environment, automated vehicles must plan a safe, comfortable, and efficient motion trajectory. To promote safety and progress, many works rely on modules that predict the future motion of surrounding traffic. Modular automated driving systems commonly handle prediction and planning as sequential, separate tasks. While this accounts for the influence of surrounding traffic on the ego vehicle, it fails to anticipate the reactions of traffic participants to the ego vehicle's behavior. Recent methods increasingly integrate prediction and planning in a joint or interdependent step to model bidirectional interactions. To date, a comprehensive overview of different integration principles is lacking. We systematically review state-of-the-art deep learning-based planning systems, and focus on how they integrate prediction. Different facets of the integration ranging from system architecture to high-level behavioral aspects are considered and related to each other. Moreover, we discuss the implications, strengths, and limitations of different integration principles. By pointing out research gaps, describing relevant future challenges, and highlighting trends in the research field, we identify promising directions for future research.
Robotics,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning,Multiagent Systems
What problem does this paper attempt to address?
The paper aims to address the issue of better integrating the prediction and planning modules in autonomous driving systems. Specifically: 1. **Existing Problems**: Traditional autonomous driving systems usually treat prediction (i.e., forecasting the future behavior of surrounding traffic participants) and planning (i.e., devising an appropriate driving trajectory for the vehicle itself) as two independent and sequential tasks. Although this approach considers the impact of surrounding traffic on the vehicle, it fails to account for the influence of the vehicle's behavior on other traffic participants, thus lacking bidirectional interaction. 2. **Objective**: This paper reviews the latest deep learning methods to explore how to more closely integrate prediction and planning tasks in autonomous driving systems. The authors argue that prediction and planning should not be viewed as sequential problems but should be interdependent and jointly optimized to achieve more efficient bidirectional interaction. The article focuses on analyzing the advantages and limitations of different integration principles and identifies gaps in the research as well as future research directions. 3. **Contribution**: The paper proposes a classification system to describe different integration methods of prediction and planning and discusses in detail the relationship between these classifications and system architecture, behavior characteristics, and safety. Additionally, it reveals the shortcomings in current research and, based on this, offers some suggestions for future research.