ODGR: Online Dynamic Goal Recognition

Matan Shamir,Osher Elhadad,Matthew E. Taylor,Reuth Mirsky

2024-07-23

Abstract:Traditionally, Reinforcement Learning (RL) problems are aimed at optimization of the behavior of an agent. This paper proposes a novel take on RL, which is used to learn the policy of another agent, to allow real-time recognition of that agent's goals. Goal Recognition (GR) has traditionally been framed as a planning problem where one must recognize an agent's objectives based on its observed actions. Recent approaches have shown how reinforcement learning can be used as part of the GR pipeline, but are limited to recognizing predefined goals and lack scalability in domains with a large goal space. This paper formulates a novel problem, "Online Dynamic Goal Recognition" (ODGR), as a first step to address these limitations. Contributions include introducing the concept of dynamic goals into the standard GR problem definition, revisiting common approaches by reformulating them using ODGR, and demonstrating the feasibility of solving ODGR in a navigation domain using transfer learning. These novel formulations open the door for future extensions of existing transfer learning-based GR methods, which will be robust to changing and expansive real-time environments.

Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The paper attempts to address the problem of real-time identification of other agents' goals in dynamic environments. Specifically, the paper introduces a new problem—Online Dynamic Goal Recognition (ODGR), which aims to tackle the scalability and predefined goal set issues of existing Goal Recognition (GR) methods when faced with a large number of goals. Traditionally, Reinforcement Learning (RL) problems focus on optimizing the behavior policy of an agent; however, this paper leverages RL to learn the goal recognition policy of another agent, thereby achieving real-time goal recognition. The main contributions of the paper include: 1. **Conceptual Contribution**: Introducing the concept of dynamic goals and incorporating it into the standard definition of the goal recognition problem. 2. **Algorithmic Contribution**: Proposing a general algorithm that demonstrates the feasibility of solving the ODGR problem using transfer learning in a simple navigation domain. By introducing the concept of dynamic goals and demonstrating policy transfer methods in the navigation domain, the researchers show that GR methods can be extended to recognize and adapt to goals that change over time, thereby broadening the application scope of these methods in real-time dynamic environments.

ODGR: Online Dynamic Goal Recognition

Off-Dynamics Inverse Reinforcement Learning

Goal Recognition as Reinforcement Learning

Heuristic Online Goal Recognition in Continuous Domains

Real-Time Online Goal Recognition in Continuous Domains via Deep Reinforcement Learning

GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models

GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Driving

Goal Recognition as a Deep Learning Task: the GRNet Approach

Autonomous Learning and Navigation of Mobile Robots Based on Deep Reinforcement Learning

Data-Driven Goal Recognition Design for General Behavioral Agents

Goal-Driven Autonomous Exploration Through Deep Reinforcement Learning

Transformable Gaussian Reward Function for Socially-Aware Navigation with Deep Reinforcement Learning

Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning

GRAC: Self-Guided and Self-Regularized Actor-Critic

Multigoal Visual Navigation With Collision Avoidance via Deep Reinforcement Learning

Evaluation of Goal Recognition Systems on Unreliable Data and Uninspectable Agents

Transformable Gaussian Reward Function for Socially Aware Navigation Using Deep Reinforcement Learning

GRIT: Fast, Interpretable, and Verifiable Goal Recognition with Learned Decision Trees for Autonomous Driving

On-Robot Reinforcement Learning with Goal-Contrastive Rewards

Goal-Conditioned Reinforcement Learning: Problems and Solutions

TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations