Advancing Autonomous Driving Safety Through LLM Enhanced Trajectory Prediction

Qian Cheng,Xinyu Jiao,Mengmeng Yang,Mingliang Yang,Kun Jiang,Diange Yang
DOI: https://doi.org/10.1007/978-3-031-70392-8_71
2024-01-01
Abstract:AbstractIn recent years, there has been remarkable progress in autonomous driving technology. To improve the safety of autonomous driving comprehensively, accurate predictions for all traffic agents are crucial. Typically, the graph neural network is widely employed for the trajectory prediction. To enhance the prediction accuracy rate, this paper utilizes a finetuned vision-to-language large model to extract driving intentions. With the well-designed prompt and the supervision of the specific dataset, the LLM (large language model) can analyze the current traffic condition and give the corresponding driving intention. This paper also combines the result of the LLM and the output of the traditional prediction model, and the future trajectory is modified with the driving intention, which can improve the final prediction accuracy. Finally, in the decision-making part, both the driving intention from the LLM and the trajectory from the traditional prediction model are considered in the boundary-based drivable area, and a safe planning path is then generated. According to the validation in the public motion forecasting dataset, this method has greatly improved the accuracy of the prediction and the safety of route planning.
What problem does this paper attempt to address?