Abstract:When robots enter everyday human environments, they need to understand their tasks and how they should perform those tasks. To encode these, reward functions, which specify the objective of a robot, are employed. However, designing reward functions can be extremely challenging for complex tasks and environments. A promising approach is to learn reward functions from humans. Recently, several robot learning works embrace this approach and leverage human demonstrations to learn the reward functions. Known as inverse reinforcement learning, this approach relies on a fundamental assumption: humans can provide near-optimal demonstrations to the robot. Unfortunately, this is rarely the case: human demonstrations to the robot are often suboptimal due to various reasons, e.g., difficulty of teleoperation, robot having high degrees of freedom, or humans' cognitive limitations. This thesis is an attempt towards learning reward functions from human users by using other, more reliable data modalities. Specifically, we study how reward functions can be learned using comparative feedback, in which the human user compares multiple robot trajectories instead of (or in addition to) providing demonstrations. To this end, we first propose various forms of comparative feedback, e.g., pairwise comparisons, best-of-many choices, rankings, scaled comparisons; and describe how a robot can use these various forms of human feedback to infer a reward function, which may be parametric or non-parametric. Next, we propose active learning techniques to enable the robot to ask for comparison feedback that optimizes for the expected information that will be gained from that user feedback. Finally, we demonstrate the applicability of our methods in a wide variety of domains, ranging from autonomous driving simulations to home robotics, from standard reinforcement learning benchmarks to lower-body exoskeletons.

Learning User Preferences for Trajectories from Brain Signals

Clustering user preferences for personalized teleoperation control schemes via trajectory similarity analysis

Improving User Experience in Preference-Based Optimization of Reward Functions for Assistive Robots

Unified Learning from Demonstrations, Corrections, and Preferences during Physical Human-Robot Interaction

Four-Way Classification of EEG Responses To Virtual Robot Navigation

PREDICT: Preference Reasoning by Evaluating Decomposed preferences Inferred from Candidate Trajectories

Brain Responses During Robot-Error Observation

EEG-Based Analysis of Brain Responses in Multi-Modal Human-Robot Interaction: Modulating Engagement

Intuitive Spatial Tactile Feedback for Better Awareness about Robot Trajectory during Human-Robot Collaboration

Towards Preference Learning for Autonomous Ground Robot Navigation Tasks

Preference Detection of the Humanoid Robot Face Based on EEG and Eye Movement

Analysis of User Preferences for Robot Motions in Immersive Telepresence

Spatially Small-scale Approach-avoidance Behaviors Allow Learning-free Machine Inference of Object Preferences in Human Minds

Trajectory Improvement and Reward Learning from Comparative Language Feedback

Learning Preferences for Interactive Autonomy

Augmenting robot intelligence via EEG signals to avoid trajectory planning mistakes of a smart wheelchair

Towards Inferring Users' Impressions of Robot Performance in Navigation Scenarios

Learning User Preferences via Reinforcement Learning with Spatial Interface Valuing

Planning Socially Expressive Mobile Robot Trajectories

New Heuristics for Interfacing Human Motor System using Brain Waves

"You Have Reached Your Destination": A Single Trial EEG Classification Study