Abstract:In this work we explore the use of reinforcement learning (RL) to help with human decision making, combining state-of-the-art RL algorithms with an application to prosthetics. Managing human-machine interaction is a problem of considerable scope, and the simplification of human-robot interfaces is especially important in the domains of biomedical technology and rehabilitation medicine. For example, amputees who control artificial limbs are often required to quickly switch between a number of control actions or modes of operation in order to operate their devices. We suggest that by learning to anticipate (predict) a user's behaviour, artificial limbs could take on an active role in a human's control decisions so as to reduce the burden on their users. Recently, we showed that RL in the form of general value functions (GVFs) could be used to accurately detect a user's control intent prior to their explicit control choices. In the present work, we explore the use of temporal-difference learning and GVFs to predict when users will switch their control influence between the different motor functions of a robot arm. Experiments were performed using a multi-function robot arm that was controlled by muscle signals from a user's body (similar to conventional artificial limb control). Our approach was able to acquire and maintain forecasts about a user's switching decisions in real time. It also provides an intuitive and reward-free way for users to correct or reinforce the decisions made by the machine learning system. We expect that when a system is certain enough about its predictions, it can begin to take over switching decisions from the user to streamline control and potentially decrease the time and effort needed to complete tasks. This preliminary study therefore suggests a way to naturally integrate human- and machine-based decision making systems.

Relating Human Error-Based Learning to Modern Deep RL Algorithms

What deep reinforcement learning tells us about human motor learning and vice-versa

Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Intervention

Trial without Error: Towards Safe Reinforcement Learning via Human Intervention

Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning

Stable and Safe Human-aligned Reinforcement Learning through Neural Ordinary Differential Equations

Discovering General Reinforcement Learning Algorithms with Adversarial Environment Design

Reinforcement Learning based Embodied Agents Modelling Human Users Through Interaction and Multi-Sensory Perception

Deep reinforcement learning from human preferences

Temporal-Difference Learning to Assist Human Decision Making during the Control of an Artificial Limb

The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback

Partial End-to-end Reinforcement Learning for Robustness Against Modelling Error in Autonomous Racing

An Invitation to Deep Reinforcement Learning

Learning to be Safe: Deep RL with a Safety Critic

Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actions

Principled Reinforcement Learning with Human Feedback from Pairwise or $K$-wise Comparisons

A Digital Twin Framework for Reinforcement Learning with Real-Time Self-Improvement via Human Assistive Teleoperation

Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHF

Direct and indirect reinforcement learning

Accelerating Reinforcement Learning using EEG-based implicit human feedback

A survey on model-based reinforcement learning