Abstract:Although previous studies point to qualitative similarities between working memory (WM) and attention, the degree to which these two constructs rely on shared neural mechanisms remains unknown. Focusing on one such potentially shared mechanism, we tested the hypothesis that selecting an item within WM utilizes similar neural mechanisms as selecting a visible item via a shift of attention. We used fMRI and machine learning to decode both the selection among items visually available and the selection among items stored in WM in human subjects (both sexes). Patterns of activity in visual, parietal, and to a lesser extent frontal cortex predicted the locations of the selected items. Critically, these patterns were strikingly interchangeable; classifiers trained on data during attentional selection predicted selection from WM, and classifiers trained on data during selection from memory predicted attentional selection. Using models of voxel receptive fields, we visualized topographic population activity that revealed gain enhancements at the locations of the externally and internally selected items. Our results suggest that selecting among perceived items and selecting among items in WM share a common mechanism. This common mechanism, analogous to a shift of spatial attention, controls the relative gains of neural populations that encode behaviorally relevant information. SIGNIFICANCE STATEMENT How we allocate our attention to external stimuli that we see and to internal representations of stimuli stored in memory might rely on a common mechanism. Supporting this hypothesis, we demonstrated that not only could patterns of human brain activity predict which items were selected during perception and memory, but that these patterns were interchangeable during external and internal selection. Additionally, this generalized selection mechanism operates by changes in the gains of the neural populations both encoding attended sensory representations and storing relevant memory representations.

Attention or memory? Neurointerpretable agents in space and time

S2RL: Do We Really Need to Perceive All States in Deep Multi-Agent Reinforcement Learning?

Attention Schema in Neural Agents

Towards Interpretable Reinforcement Learning Using Attention Augmented Agents

Self-Attention for Deep Reinforcement Learning.

Learning to attend in a brain-inspired deep neural network

Neural architecture impact on identifying temporally extended Reinforcement Learning tasks

Computational characterization of the role of an attention schema in controlling visuospatial attention

Hard Attention Control By Mutual Information Maximization

Common Neural Mechanisms Control Attention and Working Memory

A Design of Global Workspace Model with Attention: Simulations of Attentional Blink and Lag-1 Sparing

Attention, Short-Term Memory, and Action Selection: A Unifying Theory

The attention schema theory in a neural network agent: Controlling visuospatial attention using a descriptive model of attention

Exploring Coarse-grained Pre-guided Attention to Assist Fine-grained Attention Reinforcement Learning Agents

A Neural Network Model of Visual Attention Integrating Biased Competition and Reinforcement Learning

Inverse Attention Agent for Multi-Agent System

Dynamic Interaction between Reinforcement Learning and Attention in Multidimensional Environments

Visual Attention in Imaginative Agents

Selective Particle Attention: Visual Feature-Based Attention in Deep Reinforcement Learning

Object Based Attention Through Internal Gating

An initial attempt of combining visual selective attention with deep reinforcement learning