Abstract:Abstract Deep convolutional neural networks (DCNNs) are able to partially predict brain activity during object categorization tasks, but factors contributing to this predictive power are not fully understood. Our study aimed to investigate the factors contributing to the predictive power of DCNNs in object categorization tasks. We compared the activity of four DCNN architectures with EEG recordings obtained from 62 human participants during an object categorization task. Previous physiological studies on object categorization have highlighted the importance of figure-ground segregation—the ability to distinguish objects from their backgrounds. Therefore, we investigated whether figure-ground segregation could explain the predictive power of DCNNs. Using a stimulus set consisting of identical target objects embedded in different backgrounds, we examined the influence of object background versus object category within both EEG and DCNN activity. Crucially, the recombination of naturalistic objects and experimentally controlled backgrounds creates a challenging and naturalistic task, while retaining experimental control. Our results showed that early EEG activity (< 100 msec) and early DCNN layers represent object background rather than object category. We also found that the ability of DCNNs to predict EEG activity is primarily influenced by how both systems process object backgrounds, rather than object categories. We demonstrated the role of figure-ground segregation as a potential prerequisite for recognition of object features, by contrasting the activations of trained and untrained (i.e., random weights) DCNNs. These findings suggest that both human visual cortex and DCNNs prioritize the segregation of object backgrounds and target objects to perform object categorization. Altogether, our study provides new insights into the mechanisms underlying object categorization as we demonstrated that both human visual cortex and DCNNs care deeply about object background.

A Comparison of Statistical Learning of Naturalistic Textures Between DCNNs and the Human Visual Hierarchy

Comparing object recognition in humans and deep convolutional neural networks -- An eye tracking study

Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation

Visual Statistical Learning of Naturalistic Textures.

Revealing Fine Structures of the Retinal Receptive Field by Deep-Learning Networks

Human Visual Pathways for Action Recognition versus Deep Convolutional Neural Networks: Representation Correspondence in Late but Not Early Layers

Noise reduction as a unified mechanism of perceptual learning in humans, macaques, and convolutional neural networks

Human Visual Cortex and Deep Convolutional Neural Network Care Deeply about Object Background

Comparison Against Task Driven Artificial Neural Networks Reveals Functional Organization of Mouse Visual Cortex

Dissociable Neural Representations of Adversarially Perturbed Images in Convolutional Neural Networks and the Human Brain

A Visual Cortex-Attentive Deep Convolutional Neural Network for Digital Image Design

Deep Neural Networks predict Hierarchical Spatio-temporal Cortical Dynamics of Human Visual Object Recognition

Incremental Learning and Self-Attention Mechanisms Improve Neural System Identification

Guiding Visual Attention in Deep Convolutional Neural Networks Based on Human Eye Movements

Hierarchical Sparse Coding of Objects in Deep Convolutional Neural Networks

Deep Continuous Networks

Comparative Analysis between Vision Transformers and CNNs from the view of Neuroscience

Complex Properties of Training Stimuli Affect Brain Alignment in a Deep Network Model of Mouse Visual Cortex

Convolutional neural networks for vision neuroscience: significance, developments, and outstanding issues

Deep Spiking Neural Networks with High Representation Similarity Model Visual Pathways of Macaque and Mouse

Training for object recognition with increasing spatial frequency: A comparison of deep learning with human vision