Abstract:Capacity limitations in visual tasks can be observed when the number of task-related objects increases. An influential idea is that such capacity limitations are determined by competition at the neural level: two objects that are encoded by shared neural populations interfere more in behavior (e.g., visual search) than two objects encoded by separate neural populations. However, the neural representational similarity of objects varies across brain regions and across time, raising the question of where and when competition determines task performance. Furthermore, it is unclear whether the association between neural representational similarity and task performance is common or unique across tasks. Here, we used neural representational similarity derived from fMRI, MEG, and deep neural networks (DNN) to predict performance on two visual search tasks involving the same objects and requiring the same responses but differing in instructions: cued visual search and oddball visual search. Separate groups of human participants (both sexes) viewed the individual objects in neuroimaging experiments to establish the neural representational similarity between those objects. Results showed that performance on both search tasks could be predicted by neural representational similarity throughout the visual system (fMRI), from 80 msec after onset (MEG), and in all DNN layers. Stepwise regression analysis, however, revealed task-specific associations, with unique variability in oddball search performance predicted by early/posterior neural similarity, and unique variability in cued search task performance predicted by late/anterior neural similarity. These results reveal that capacity limitations in superficially similar visual search tasks may reflect competition at different stages of visual processing. Significance Statement Visual search for target objects is slowed down by the presence of distractors, but not all distractors are equally distracting – the more similar a distractor is to the target, the more it slows down search. Here, we used fMRI, MEG, and a deep neural network to reveal where, when, and how neural similarity between targets and distractors predicts visual search performance across two search tasks: oddball visual search (locating the different-looking object) and cued visual search (locating the cued object). Results also revealed brain regions, time points, and feature levels that predicted task-unique performance. These results provide a neural basis for similarity theories of visual search and show that this neural basis differs across visual search tasks.

Reliability and Generalizability of Similarity-Based Fusion of MEG and fMRI Data in Human Ventral and Dorsal Visual Streams

Similarity-Based Fusion of MEG and fMRI Reveals Spatio-Temporal Dynamics in Human Cortex During Visual Object Recognition

Similarity-based fMRI-MEG fusion reveals hierarchical organisation within the brain's semantic system

Hierarchical organization of the human ventral visual streams revealed with magnetoencephalography

Simultaneous EEG-fMRI: Trial level spatio-temporal fusion for hierarchically reliable information discovery

Hierarchical organisation of the human ventral visual streams revealed with magnetoencephalography

Functional Source Separation for EEG-fMRI Fusion: Application to Steady-State Visual Evoked Potentials

Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Models

Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI

Reliability of fMRI time series: Similarity of neural processing during movie viewing

Towards Neural Foundation Models for Vision: Aligning EEG, MEG, and fMRI Representations for Decoding, Encoding, and Modality Conversion

Consistency and similarity of MEG- and fMRI-signal time courses during movie viewing

A multimodal approach connecting cortical and behavioural responses to the visual continuity illusion

Multisensory integration of dynamic emotional faces and voices: method for simultaneous EEG-fMRI measurements

MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data

Predicting cued and oddball visual search performance from fMRI, MEG, and DNN neural representational similarity

NeuroCine: Decoding Vivid Video Sequences from Human Brain Activties

Invariant inter-subject relational structures in the human visual cortex

Fusing Eeg And Fmri Based On A Bottom-Up Model: Inferring Activation And Effective Connectivity In Neural Masses

Eeg/Fmri Fusion Based on Independent Component Analysis: Integration of Data-Driven and Model-Driven Methods

Investigating critical brain area for EEG-based binocular color fusion and rivalry with EEGNet