Abstract:When robots interact with humans in homes, roads, or factories the human's behavior often changes in response to the robot. Non-stationary humans are challenging for robot learners: actions the robot has learned to coordinate with the original human may fail after the human adapts to the robot. In this paper we introduce an algorithmic formalism that enables robots (i.e., ego agents) to co-adapt alongside dynamic humans (i.e., other agents) using only the robot's low-level states, actions, and rewards. A core challenge is that humans not only react to the robot's behavior, but the way in which humans react inevitably changes both over time and between users. To deal with this challenge, our insight is that -- instead of building an exact model of the human -- robots can learn and reason over high-level representations of the human's policy and policy dynamics. Applying this insight we develop RILI: Robustly Influencing Latent Intent. RILI first embeds low-level robot observations into predictions of the human's latent strategy and strategy dynamics. Next, RILI harnesses these predictions to select actions that influence the adaptive human towards advantageous, high reward behaviors over repeated interactions. We demonstrate that -- given RILI's measured performance with users sampled from an underlying distribution -- we can probabilistically bound RILI's expected performance across new humans sampled from the same distribution. Our simulated experiments compare RILI to state-of-the-art representation and reinforcement learning baselines, and show that RILI better learns to coordinate with imperfect, noisy, and time-varying agents. Finally, we conduct two user studies where RILI co-adapts alongside actual humans in a game of tag and a tower-building task. See videos of our user studies here: <a class="link-external link-https" href="https://youtu.be/WYGO5amDXbQ" rel="external noopener nofollow">this https URL</a>

Subconscious Robotic Imitation Learning

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Task-Oriented Deep Reinforcement Learning for Robotic Skill Acquisition and Control

Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates

RIRL: A Recurrent Imitation and Reinforcement Learning Method for Long-Horizon Robotic Tasks

Imitation-Enhanced Reinforcement Learning with Privileged Smooth Transition for Hexapod Locomotion

Efficient Robot Skill Learning with Imitation from a Single Video for Contact-Rich Fabric Manipulation

Robotic Imitation of Human Actions

Imitation Learning via Simultaneous Optimization of Policies and Auxiliary Trajectories

A data-efficient goal-directed deep reinforcement learning method for robot visuomotor skill

RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effective

Accelerated Robot Learning via Human Brain Signals

Prompt, Plan, Perform: LLM-based Humanoid Control via Quantized Imitation Learning

SEIL: Simulation-augmented Equivariant Imitation Learning

CRIL: Continual Robot Imitation Learning via Generative and Prediction Model

One-shot sim-to-real transfer policy for robotic assembly via reinforcement learning with visual demonstration

Accelerating Self-Imitation Learning from Demonstrations via Policy Constraints and Q-Ensemble

Efficient Intrinsically Motivated Robotic Grasping with Learning-Adaptive Imagination in Latent Space

SLR: Learning Quadruped Locomotion without Privileged Information

Learning Latent Representations to Co-Adapt to Humans

RILe: Reinforced Imitation Learning