Abstract:Safe and efficient autonomous driving maneuvers in an interactive and complex environment can be considerably challenging due to the unpredictable actions of other surrounding agents that may be cooperative or adversarial in their interactions with the ego vehicle. One of the state-of-the-art approaches is to apply Reinforcement Learning (RL) to learn a time-sequential driving policy, to execute proper control strategy or tracking trajectory in dynamic situations. However, direct application of RL algorithms is not satisfactorily enough to deal with the cases in the autonomous driving domain, mainly due to the complex driving environment and continuous action space. In this paper, we adopt Q-learning as our basic learning framework and design a unique format of the Q-function approximator that consists of neural networks to handle the continuous action space challenge. The learning model is present in a closed form of continuous control variables and trained in a simulation platform that we have developed with embedded properties of real-time vehicle interactions. The proposed algorithm avoids invoking an additional actor network that learns to take actions, as in actor-critic algorithms. At the same time, some prior knowledge of vehicle dynamics is also fed into the model to assist learning. We test our algorithm with a challenging use case - lane change maneuver, to verify the practicability and feasibility of the proposed approach. Results from accumulated rewards and vehicle performance show that RL vehicle agents successfully learn a safe, comfort and efficient driving policy as defined in the reward function.

Deep Reinforcement Learning with Embedded LQR Controllers

Open-Loop Motion Control of a Hydraulic Soft Robotic Arm Using Deep Reinforcement Learning

A Tour of Reinforcement Learning: The View from Continuous Control

Hierarchical Control of Multi-Agent Systems using Online Reinforcement Learning

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

A Reinforcement Learning Method for LQR Control Problem

A Deep Reinforcement Learning Control Method Guided by RBF-ARX Pseudo LQR

Specialized Deep Residual Policy Safe Reinforcement Learning-Based Controller for Complex and Continuous State-Action Spaces

Continuous control with deep reinforcement learning

Using Deep Reinforcement Learning for the Continuous Control of Robotic Arms

Automated Driving Maneuvers under Interactive Environment based on Deep Reinforcement Learning

Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments from Temporal Logic Specifications

Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers

Model-Based Reinforcement Learning In Continuous Environments Using Real-Time Constrained Optimization

Hierarchical Deep Reinforcement Learning for Continuous Action Control

A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation

Hybrid LMC: Hybrid Learning and Model-based Control for Wheeled Humanoid Robot via Ensemble Deep Reinforcement Learning

Reinforcement Learning for a Discrete-Time Linear-Quadratic Control Problem with an Application

Deep Incremental Model Based Reinforcement Learning: A One-Step Lookback Approach for Continuous Robotics Control

NEARL: Non-Explicit Action Reinforcement Learning for Robotic Control

Decomposed Deep Reinforcement Learning for Robotic Control