Abstract:Capacity limitations in visual tasks can be observed when the number of task-related objects increases. An influential idea is that such capacity limitations are determined by competition at the neural level: two objects that are encoded by shared neural populations interfere more in behavior (e.g., visual search) than two objects encoded by separate neural populations. However, the neural representational similarity of objects varies across brain regions and across time, raising the question of where and when competition determines task performance. Furthermore, it is unclear whether the association between neural representational similarity and task performance is common or unique across tasks. Here, we used neural representational similarity derived from fMRI, MEG, and deep neural networks (DNN) to predict performance on two visual search tasks involving the same objects and requiring the same responses but differing in instructions: cued visual search and oddball visual search. Separate groups of human participants (both sexes) viewed the individual objects in neuroimaging experiments to establish the neural representational similarity between those objects. Results showed that performance on both search tasks could be predicted by neural representational similarity throughout the visual system (fMRI), from 80 msec after onset (MEG), and in all DNN layers. Stepwise regression analysis, however, revealed task-specific associations, with unique variability in oddball search performance predicted by early/posterior neural similarity, and unique variability in cued search task performance predicted by late/anterior neural similarity. These results reveal that capacity limitations in superficially similar visual search tasks may reflect competition at different stages of visual processing. Significance Statement Visual search for target objects is slowed down by the presence of distractors, but not all distractors are equally distracting – the more similar a distractor is to the target, the more it slows down search. Here, we used fMRI, MEG, and a deep neural network to reveal where, when, and how neural similarity between targets and distractors predicts visual search performance across two search tasks: oddball visual search (locating the different-looking object) and cued visual search (locating the cued object). Results also revealed brain regions, time points, and feature levels that predicted task-unique performance. These results provide a neural basis for similarity theories of visual search and show that this neural basis differs across visual search tasks.

Predicting cued and oddball visual search performance from fMRI, MEG, and DNN neural representational similarity

Predicting cued and oddball visual search performance from neural representational similarity

Visual Target Detection in a Distracting Background Relies on Neural Encoding of Both Visual Targets and Background.

Visual search and real-image similarity: An empirical assessment through the lens of deep learning

Brain Network Reorganization During Visual Search Task Revealed by a Network Analysis of Fixation-Related Potential.

Differential involvement of EEG oscillatory components in sameness vs. spatial-relation visual reasoning tasks

Overlapping neural representations for selection in attention and working memory

Skills and cautiousness predict performance in difficult search

Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation

Neural representation of targets and distractors during object individuation and identification

Neural Basis of Interaction Between Target Presence and Display Homogeneity in Visual Search: an Fmri Study

No difference in prior representations of what to attend and what to ignore

Electrophysiological correlates of visual memory search

Task-specific vision models explain task-specific areas of visual cortex

Differential brain mechanisms for processing distracting information in task‐relevant and ‐irrelevant dimensions in visual search

Guided Visual Search is associated with a feature-based priority map in early visual cortex

Finding Shared Decodable Concepts and their Negations in the Brain

A Neurobiological Evaluation Metric for Neural Network Model Search

Predicting Visual Attention and Distraction During Visual Search Using Convolutional Neural Networks

Attention to Detail: Why Considering Task Demands Is Essential for Single-Trial Analysis of BOLD Correlates of the Visual P1 and N1

Target-distractor similarity predicts visual search efficiency but only for highly similar features