Abstract:The human brain uses selective attention to filter perceptual input so that only the components that are useful for behaviour are processed using its limited computational resources. We focus on one particular form of visual attention known as feature-based attention, which is concerned with identifying features of the visual input that are important for the current task regardless of their spatial location. Visual feature-based attention has been proposed to improve the efficiency of Reinforcement Learning (RL) by reducing the dimensionality of state representations and guiding learning towards relevant features. Despite achieving human level performance in complex perceptual-motor tasks, Deep RL algorithms have been consistently criticised for their poor efficiency and lack of flexibility. Visual feature-based attention therefore represents one option for addressing these criticisms. Nevertheless, it is still an open question how the brain is able to learn which features to attend to during RL. To help answer this question we propose a novel algorithm, termed Selective Particle Attention (SPA), which imbues a Deep RL agent with the ability to perform selective feature-based attention. SPA learns which combinations of features to attend to based on their bottom-up saliency and how accurately they predict future reward. We evaluate SPA on a multiple choice task and a 2D video game that both involve raw pixel input and dynamic changes to the task structure. We show various benefits of SPA over approaches that naively attend to either all or random subsets of features. Our results demonstrate (1) how visual feature-based attention in Deep RL models can improve their learning efficiency and ability to deal with sudden changes in task structure and (2) that particle filters may represent a viable computational account of how visual feature-based attention occurs in the brain.

Exploring Coarse-grained Pre-guided Attention to Assist Fine-grained Attention Reinforcement Learning Agents

Attention Guided Imitation Learning and Reinforcement Learning

An initial attempt of combining visual selective attention with deep reinforcement learning

Better Deep Visual Attention with Reinforcement Learning in Action Recognition.

Self-Attention for Deep Reinforcement Learning.

Attention or memory? Neurointerpretable agents in space and time

AGIL: Learning Attention from Human for Visuomotor Tasks

Multi-agent Reinforcement Learning with Multi-head Attention

Focus-Then-Decide: Segmentation-Assisted Reinforcement Learning

Understanding More about Human and Machine Attention in Deep Neural Networks

Learning to attend in a brain-inspired deep neural network

Where to Focus: Investigating Hierarchical Attention Relationship for Fine-Grained Visual Classification

Role‐based Attention in Deep Reinforcement Learning for Games

Attend Before you Act: Leveraging human visual attention for continual learning

Hierarchical Reinforcement Learning with Attention Reward

Selective Particle Attention: Visual Feature-Based Attention in Deep Reinforcement Learning

Human Vs Machine Attention in Neural Networks: A Comparative Study.

Towards Interpretable Reinforcement Learning Using Attention Augmented Agents

Meta Attention for Off-Policy Actor-Critic.

A Hierarchical Reinforcement Learning Algorithm Based on Attention Mechanism for UAV Autonomous Navigation

Optimizing Attention for Sequence Modeling Via Reinforcement Learning.