Abstract:Driving behavior modeling is of great importance for designing safe, smart, and personalized autonomous driving systems. In this paper, an internal reward function-based driving model that emulates the human's decision-making mechanism is utilized. To infer the reward function parameters from naturalistic human driving data, we propose a structural assumption about human driving behavior that focuses on discrete latent driving intentions. It converts the continuous behavior modeling problem to a discrete setting and thus makes maximum entropy inverse reinforcement learning (IRL) tractable to learn reward functions. Specifically, a polynomial trajectory sampler is adopted to generate candidate trajectories considering high-level intentions and approximate the partition function in the maximum entropy IRL framework. An environment model considering interactive behaviors among the ego and surrounding vehicles is built to better estimate the generated trajectories. The proposed method is applied to learn personalized reward functions for individual human drivers from the NGSIM highway driving dataset. The qualitative results demonstrate that the learned reward functions are able to explicitly express the preferences of different drivers and interpret their decisions. The quantitative results reveal that the learned reward functions are robust, which is manifested by only a marginal decline in proximity to the human driving trajectories when applying the reward function in the testing conditions. For the testing performance, the personalized modeling method outperforms the general modeling approach, significantly reducing the modeling errors in human likeness (a custom metric to gauge accuracy), and these two methods deliver better results compared to other baseline methods. Moreover, it is found that predicting the response actions of surrounding vehicles and incorporating their potential decelerations caused by the ego vehicle are critical in estima-ing the generated trajectories, and the accuracy of personalized planning using the learned reward functions relies on the accuracy of the forecasting model.

Human-in-the-Loop Behavior Modeling via an Integral Concurrent Adaptive Inverse Reinforcement Learning

Online Learning Human Behavior for a Class of Human-in-the-Loop Systems via Adaptive Inverse Optimal Control

Off-Dynamics Inverse Reinforcement Learning

Off-Dynamics Inverse Reinforcement Learning from Hetero-Domain

A Finite-Horizon Inverse Linear Quadratic Optimal Control Method for Human-in-the-Loop Behavior Learning

Learning Human Behavior in Shared Control: Adaptive Inverse Differential Game Approach

Composite Adaptive Online Inverse Optimal Control Approach to Human Behavior Learning

Convergence Analysis of an Incremental Approach to Online Inverse Reinforcement Learning

Human Behavior Learning for a Class of Nonlinear Human-in-the-loop Systems via Takagi-Sugeno Fuzzy Model

Iteration-dependent High-order Internal Model Based Iterative Learning Control for Continuous-time Nonlinear Systems

Human-Interactive Subgoal Supervision for Efficient Inverse Reinforcement Learning

Iteration-dependent High-order Internal Model Based Iterative Learning Control for Discrete-time Nonlinear Systems with Time-iteration-varying Parameter

Synthesis With Guaranteed Cost and Less Human Intervention for Human-in-the-Loop Control Systems

Data-based neural controls for an unknown continuous-time multi-input system with integral reinforcement

Hybrid Inverse Reinforcement Learning

Driving Behavior Modeling Using Naturalistic Human Driving Data With Inverse Reinforcement Learning

Unified Human-Robot-Environment Interaction Control in Contact-Rich Collaborative Manipulation Tasks Via Model-Based Reinforcement Learning

Towards Modeling and Influencing the Dynamics of Human Learning

Learning Variable Impedance Control via Inverse Reinforcement Learning for Force-Related Tasks

An Efficient Unified Approach Using Demonstrations for Inverse Reinforcement Learning

Inverse reinforcement learning by expert imitation for the stochastic linear-quadratic optimal control problem