Abstract:People deploy top-down, goal-directed attention to accomplish tasks, such as finding lost keys. By tuning the visual system to relevant information sources, object recognition can become more efficient (a benefit) and more biased toward the target (a potential cost). Motivated by selective attention in categorisation models, we developed a goal-directed attention mechanism that can process naturalistic (photographic) stimuli. Our attention mechanism can be incorporated into any existing deep convolutional neural network (DCNNs). The processing stages in DCNNs have been related to ventral visual stream. In that light, our attentional mechanism incorporates top-down influences from prefrontal cortex (PFC) to support goal-directed behaviour. Akin to how attention weights in categorisation models warp representational spaces, we introduce a layer of attention weights to the mid-level of a DCNN that amplify or attenuate activity to further a goal. We evaluated the attentional mechanism using photographic stimuli, varying the attentional target. We found that increasing goal-directed attention has benefits (increasing hit rates) and costs (increasing false alarm rates). At a moderate level, attention improves sensitivity (i.e., increases $d^\prime$) at only a moderate increase in bias for tasks involving standard images, blended images, and natural adversarial images chosen to fool DCNNs. These results suggest that goal-directed attention can reconfigure general-purpose DCNNs to better suit the current task goal, much like PFC modulates activity along the ventral stream. In addition to being more parsimonious and brain consistent, the mid-level attention approach performed better than a standard machine learning approach for transfer learning, namely retraining the final network layer to accommodate the new task.

FAN: Focused Attention Networks

A Novel Block-wise Attention Method: Focus-and-Association Networks

Focusing Attention: Towards Accurate Text Recognition in Natural Images

Focus Your Attention: A Bidirectional Focal Attention Network for Image-Text Matching

Focal Attention Networks: optimising attention for biomedical image segmentation

Deep Converged Network for Attention

An Attention Module for Convolutional Neural Networks

FcaNet: Frequency Channel Attention Networks

Visual Attention Consistency for Human Attribute Recognition

Spatial Attention Contrastive Network for Scene Text Recognition

Fully Attentional Network for Semantic Segmentation

Fine-grained Image Recognition Via Attention Interaction and Counterfactual Attention Network

Efficient Attention Network: Accelerate Attention by Searching Where to Plug

Learning 1D Causal Visual Representation with De-focus Attention Networks

Attention Graph: Learning Effective Visual Features for Large-Scale Image Classification

Attention-aware concentrated network for saliency prediction

Loss-Based Attention for Interpreting Image-Level Prediction of Convolutional Neural Networks.

The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks

Attentional Feature Fusion

Explicit Attention Network for Face Alignment.

LMA: Lightweight Mixed-Domain Attention for Efficient Network Design