Abstract:When robots interact with humans in homes, roads, or factories the human's behavior often changes in response to the robot. Non-stationary humans are challenging for robot learners: actions the robot has learned to coordinate with the original human may fail after the human adapts to the robot. In this paper we introduce an algorithmic formalism that enables robots (i.e., ego agents) to co-adapt alongside dynamic humans (i.e., other agents) using only the robot's low-level states, actions, and rewards. A core challenge is that humans not only react to the robot's behavior, but the way in which humans react inevitably changes both over time and between users. To deal with this challenge, our insight is that -- instead of building an exact model of the human -- robots can learn and reason over high-level representations of the human's policy and policy dynamics. Applying this insight we develop RILI: Robustly Influencing Latent Intent. RILI first embeds low-level robot observations into predictions of the human's latent strategy and strategy dynamics. Next, RILI harnesses these predictions to select actions that influence the adaptive human towards advantageous, high reward behaviors over repeated interactions. We demonstrate that -- given RILI's measured performance with users sampled from an underlying distribution -- we can probabilistically bound RILI's expected performance across new humans sampled from the same distribution. Our simulated experiments compare RILI to state-of-the-art representation and reinforcement learning baselines, and show that RILI better learns to coordinate with imperfect, noisy, and time-varying agents. Finally, we conduct two user studies where RILI co-adapts alongside actual humans in a game of tag and a tower-building task. See videos of our user studies here: <a class="link-external link-https" href="https://youtu.be/WYGO5amDXbQ" rel="external noopener nofollow">this https URL</a>

Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning

Deep Reinforcement Learning Based Co-Optimization of Morphology and Gait for Small-Scale Legged Robot

Co-Imitation: Learning Design and Behaviour by Imitation

Environmental Adaptation of Robot Morphology and Control Through Real-World Evolution

Scalable Co-Optimization of Morphology and Control in Embodied Machines

On Designing a Learning Robot: Improving Morphology for Enhanced Task Performance and Learning

A comparison of controller architectures and learning mechanisms for arbitrary robot morphologies

Multi-embodiment Legged Robot Control As a Sequence Modeling Problem

Data-efficient Learning of Morphology and Controller for a Microrobot

Investigating Premature Convergence in Co-optimization of Morphology and Control in Evolved Virtual Soft Robots

Learning Latent Representations to Co-Adapt to Humans

Hitting the Gym: Reinforcement Learning Control of Exercise-Strengthened Biohybrid Robots in Simulation

Simultaneous Optimization of Discrete and Continuous Parameters Defining a Robot Morphology and Controller

The dynamic of body and brain co-evolution

From the Lab to the Desert: Fast Prototyping and Learning of Robot Locomotion

MorAL: Learning Morphologically Adaptive Locomotion Controller for Quadrupedal Robots on Challenging Terrains

Gait-learning with morphologically evolving robots generated by L-system

A Morphological Transfer-Based Multi-Fidelity Evolutionary Algorithm for Soft Robot Design

Adapt On-the-Go: Behavior Modulation for Single-Life Robot Deployment

Adjustbot: Bio-Inspired Quadruped Robot with Adjustable Posture and Undulated Body for Challenging Terradynamic Tasks

Evolutionary Morphology Towards Overconstrained Locomotion via Large-Scale, Multi-Terrain Deep Reinforcement Learning