Multi-Task Long-Range Urban Driving Based on Hierarchical Planning and Reinforcement Learning

Xiaoqin Zhang,Mouyu Wu,Huimin Ma,Tianyu Hu,Jian Yuan
DOI: https://doi.org/10.1109/itsc48978.2021.9564705
2021-01-01
Abstract:Multi-task long-range autonomous driving in urban areas is a challenging task. Traditional methods are not applicable to situations with high-dimensional observations. Current reinforcement learning (RL) based algorithms are able to train agents to utilize high-dimensional signals for vehicle control and accomplish short-range tasks. However, current RL methods do not work well on multi-task long-range urban driving tasks. In this paper, we propose a novel Hierarchical-planning and Spatial-encoding-based Multi-task Reinforcement Learning (HSMRL) model for the autonomous driving problem in urban areas. For long-range tasks, global routes are planned in real-time based on the A* algorithm, and short-range tasks are constructed based on the global routes, through which the short-range tasks are regarded as sub-tasks of long-range tasks. Then the short-range task information is efficiently encoded using spatial encoding, and the short-range tasks are accomplished by RL algorithms. We also prove that under our settings, multiple autonomous driving tasks are capable of training a shared policy. Experiments show that our method can stably accomplish the multi-task long-range driving tasks and performs significantly better than the state-of-the-art methods.
What problem does this paper attempt to address?