Abstract:Driving behavior modeling is of great importance for designing safe, smart, and personalized autonomous driving systems. In this paper, an internal reward function-based driving model that emulates the human's decision-making mechanism is utilized. To infer the reward function parameters from naturalistic human driving data, we propose a structural assumption about human driving behavior that focuses on discrete latent driving intentions. It converts the continuous behavior modeling problem to a discrete setting and thus makes maximum entropy inverse reinforcement learning (IRL) tractable to learn reward functions. Specifically, a polynomial trajectory sampler is adopted to generate candidate trajectories considering high-level intentions and approximate the partition function in the maximum entropy IRL framework. An environment model considering interactive behaviors among the ego and surrounding vehicles is built to better estimate the generated trajectories. The proposed method is applied to learn personalized reward functions for individual human drivers from the NGSIM highway driving dataset. The qualitative results demonstrate that the learned reward functions are able to explicitly express the preferences of different drivers and interpret their decisions. The quantitative results reveal that the learned reward functions are robust, which is manifested by only a marginal decline in proximity to the human driving trajectories when applying the reward function in the testing conditions. For the testing performance, the personalized modeling method outperforms the general modeling approach, significantly reducing the modeling errors in human likeness (a custom metric to gauge accuracy), and these two methods deliver better results compared to other baseline methods. Moreover, it is found that predicting the response actions of surrounding vehicles and incorporating their potential decelerations caused by the ego vehicle are critical in estima-ing the generated trajectories, and the accuracy of personalized planning using the learned reward functions relies on the accuracy of the forecasting model.

Learning Human Rewards by Inferring Their Latent Intelligence Levels in Multi-Agent Games: A Theory-of-Mind Approach with Application to Driving Data

Expressing Diverse Human Driving Behavior with Probabilistic Rewards and Online Inference

Driving Behavior Modeling Using Naturalistic Human Driving Data With Inverse Reinforcement Learning

Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoning

Unveiling the latent dynamics in social cognition with multi-agent inverse reinforcement learning

Bounded Risk-Sensitive Markov Games: Forward Policy Design and Inverse Reward Learning with Iterative Reasoning and Cumulative Prospect Theory

Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learning

Human-Like Decision Making for Autonomous Vehicles at the Intersection Using Inverse Reinforcement Learning

Towards Learning from Implicit Human Reward: (extended Abstract)

Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning

Maximum Entropy Inverse Reinforcement Learning Based on Frenet Frame Sampling for Human-Like Autonomous Driving

Towards personalized human AI interaction - adapting the behavior of AI agents using neural signatures of subjective interest

Human-centric Reward Optimization for Reinforcement Learning-based Automated Driving using Large Language Models

Adaptively Shaping Reinforcement Learning Agents Via Human Reward

Analyzing the Suitability of Cost Functions for Explaining and Imitating Human Driving Behavior Based on Inverse Reinforcement Learning

Learning Latent Representations to Influence Multi-Agent Interaction

Game Theoretic Decision Making by Actively Learning Human Intentions Applied on Autonomous Driving

Learning to Incentivize Other Learning Agents

Learning from Active Human Involvement Through Proxy Value Propagation

High-Level Human Intention Learning for Cooperative Decision-Making

Modeling Theory of Mind in Multi-Agent Games Using Adaptive Feedback Control