Abstract:A key requirement for the development of effective learning representations is their evaluation and comparison to representations we know to be effective. In natural sensory domains, the community has viewed the brain as a source of inspiration and as an implicit benchmark for success. However, it has not been possible to directly test representational learning algorithms directly against the representations contained in neural systems. Here, we propose a new benchmark for visual representations on which we have directly tested the neural representation in multiple visual cortical areas in macaque (utilizing data from [Majaj et al., 2012]), and on which any computer vision algorithm that produces a feature space can be tested. The benchmark measures the effectiveness of the neural or machine representation by computing the classification loss on the ordered eigendecomposition of a kernel matrix [Montavon et al., 2011]. In our analysis we find that the neural representation in visual area IT is superior to visual area V4. In our analysis of representational learning algorithms, we find that three-layer models approach the representational performance of V4 and the algorithm in [Le et al., 2012] surpasses the performance of V4. Impressively, we find that a recent supervised algorithm [Krizhevsky et al., 2012] achieves performance comparable to that of IT for an intermediate level of image variation difficulty, and surpasses IT at a higher difficulty level. We believe this result represents a major milestone: it is the first learning algorithm we have found that exceeds our current estimate of IT representation performance. We hope that this benchmark will assist the community in matching the representational performance of visual cortex and will serve as an initial rallying point for further correspondence between representations derived in brains and machines.

Exploration and Comparison of Deep Learning Architectures to Predict Brain Response to Realistic Pictures

Generalizability analysis of deep learning predictions of human brain responses to augmented and semantically novel visual stimuli

The Algonauts Project 2023 Challenge: How the Human Brain Makes Sense of Natural Scenes

Decoding human brain activity with deep learning

Deep Semantic Architecture with discriminative feature visualization for neuroimage analysis

The Algonauts Project 2023 Challenge: UARK-UAlbany Team Solution

Unidirectional brain-computer interface: Artificial neural network encoding natural images to fMRI response in the visual cortex

Achieving More Human Brain-Like Vision via Human EEG Representational Alignment

The Algonauts Project: A Platform for Communication between the Sciences of Biological and Artificial Intelligence

Relating CNNs with brain: Challenges and findings

The Neural Representation Benchmark and its Evaluation on Brain and Machine

NEURO-DRAM: a 3D recurrent visual attention model for interpretable neuroimaging classification

Brain-like emergent properties in deep networks: impact of network architecture, datasets and training

Deep Recurrent Encoder: A scalable end-to-end network to model brain signals

Deep image reconstruction from human brain activity

Deep Natural Image Reconstruction from Human Brain Activity Based on Conditional Progressively Growing Generative Adversarial Networks

Probing neural representations of scene perception in a hippocampally dependent task using artificial neural networks

Alzheimer’s disease detection from magnetic resonance imaging: a deep learning perspective

Exploring the potential of representation and transfer learning for anatomical neuroimaging: Application to psychiatry

Systematic comparison of 3D Deep learning and classical machine learning explanations for Alzheimer's Disease detection

Contrastive Representation Learning for Whole Brain Cytoarchitectonic Mapping in Histological Human Brain Sections