ObserveNet Control: A Vision-Dynamics Learning Approach to Predictive Control in Autonomous Vehicles

Cosmin Ginerica,Mihai Zaha,Florin Gogianu,Lucian Busoniu,Bogdan Trasnea,Sorin Grigorescu

DOI: https://doi.org/10.1109/LRA.2021.3096157

2021-07-19

Abstract:A key component in autonomous driving is the ability of the self-driving car to understand, track and predict the dynamics of the surrounding environment. Although there is significant work in the area of object detection, tracking and observations prediction, there is no prior work demonstrating that raw observations prediction can be used for motion planning and control. In this paper, we propose ObserveNet Control, which is a vision-dynamics approach to the predictive control problem of autonomous vehicles. Our method is composed of a: i) deep neural network able to confidently predict future sensory data on a time horizon of up to 10s and ii) a temporal planner designed to compute a safe vehicle state trajectory based on the predicted sensory data. Given the vehicle's historical state and sensing data in the form of Lidar point clouds, the method aims to learn the dynamics of the observed driving environment in a self-supervised manner, without the need to manually specify training labels. The experiments are performed both in simulation and real-life, using CARLA and RovisLab's AMTU mobile platform as a 1:4 scaled model of a car. We evaluate the capabilities of ObserveNet Control in aggressive driving contexts, such as overtaking maneuvers or side cut-off situations, while comparing the results with a baseline Dynamic Window Approach (DWA) and two state-of-the-art imitation learning systems, that is, Learning by Cheating (LBC) and World on Rails (WOR).

Robotics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use the predicted raw sensor data for motion planning and control in self - driving vehicles. Specifically, the paper proposes a method named ObserveNet Control. This method predicts sensor data within a future period through a deep neural network and calculates safe vehicle - state trajectories based on these predicted data. This method aims to overcome the problem of insufficient prediction of external environmental dynamic changes in traditional methods, especially in complex driving environments, such as overtaking or side - cutting situations. These scenarios have very high requirements for the controller, and the performance of traditional object - detection - and - tracking - based methods will decline significantly when the detection system fails. The ObserveNet Control method trains the network in a self - supervised learning manner without the need for manual annotation of training data, so it can adapt to different driving environments more flexibly.