Abstract:Building assistive interfaces for controlling robots through arbitrary, high-dimensional, noisy inputs (e.g., webcam images of eye gaze) can be challenging, especially when it involves inferring the user's desired action in the absence of a natural 'default' interface. Reinforcement learning from online user feedback on the system's performance presents a natural solution to this problem, and enables the interface to adapt to individual users. However, this approach tends to require a large amount of human-in-the-loop training data, especially when feedback is sparse. We propose a hierarchical solution that learns efficiently from sparse user feedback: we use offline pre-training to acquire a latent embedding space of useful, high-level robot behaviors, which, in turn, enables the system to focus on using online user feedback to learn a mapping from user inputs to desired high-level behaviors. The key insight is that access to a pre-trained policy enables the system to learn more from sparse rewards than a naïve RL algorithm: using the pre-trained policy, the system can make use of successful task executions to relabel, in hindsight, what the user actually meant to do during unsuccessful executions. We evaluate our method primarily through a user study with 12 participants who perform tasks in three simulated robotic manipulation domains using a webcam and their eye gaze: flipping light switches, opening a shelf door to reach objects inside, and rotating a valve. The results show that our method successfully learns to map 128-dimensional gaze features to 7-dimensional joint torques from sparse rewards in under 10 minutes of online training, and seamlessly helps users who employ different gaze strategies, while adapting to distributional shift in webcam inputs, tasks, and environments.

Fast Explicit-Input Assistance for Teleoperation in Clutter

Dexterous Manoeuvre Through Touch in a Cluttered Scene

Customized Handling of Unintended Interface Operation in Assistive Robots

Goal-predictive Robotic Teleoperation from Noisy Sensors.

Human–Robot Shared Control Based on Locally Weighted Intent Prediction for a Teleoperated Hydraulic Manipulator System

Bootstrapping Adaptive Human-Machine Interfaces with Offline Reinforcement Learning

An Intent-based Task-aware Shared Control Framework for Intuitive Hands Free Telemanipulation

Fast, Anytime Motion Planning for Prehensile Manipulation in Clutter

Assisting Manipulation and Grasping in Robot Teleoperation with Augmented Reality Visual Cues

2D Linear Time-Variant Controller for Human's Intention Detection for Reach-to-Grasp Trajectories in Novel Scenes

Seamless Robot Teleoperation: Intuitive Control Through Hand Gestures and Neural Network Decoding

Real-time Dexterous Telemanipulation with an End-Effect-Oriented Learning-based Approach

ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learning

Contact-Implicit Model Predictive Control for Dexterous In-hand Manipulation: A Long-Horizon and Robust Approach

Visual Imitation Made Easy

A2G: Leveraging Intuitive Physics for Force-Efficient Robotic Grasping

Experimental Assessment of Human-Robot Teaming for Multi-Step Remote Manipulation with Expert Operators

A Versatile Interaction Framework for Robot Programming Based on Hand Gestures and Poses

Language-Guided Category Push–Grasp Synergy Learning in Clutter by Efficiently Perceiving Object Manipulation Space

Contact-Aware Non-Prehensile Manipulation for Object Retrieval in Cluttered Environments

Intuitive Hand Teleoperation by Novice Operators Using a Continuous Teleoperation Subspace