Abstract:Abstract Convolutional neural networks trained on object recognition derive inspiration from the neural architecture of the visual system in primates, and have been used as models of the feedforward computation performed in the primate ventral stream. In contrast to the deep hierarchical organization of primates, the visual system of the mouse has a shallower arrangement. Since mice and primates are both capable of visually guided behavior, this raises questions about the role of architecture in neural computation. In this work, we introduce a novel framework for building a biologically constrained convolutional neural network model of the mouse visual cortex. The architecture and structural parameters of the network are derived from experimental measurements, specifically the 100-micrometer resolution interareal connectome, the estimates of numbers of neurons in each area and cortical layer, and the statistics of connections between cortical layers. This network is constructed to support detailed task-optimized models of mouse visual cortex, with neural populations that can be compared to specific corresponding populations in the mouse brain. Using a well-studied image classification task as our working example, we demonstrate the computational capability of this mouse-sized network. Given its relatively small size, MouseNet achieves roughly 2/3rds the performance level on ImageNet as VGG16. In combination with the large scale Allen Brain Observatory Visual Coding dataset, we use representational similarity analysis to quantify the extent to which MouseNet recapitulates the neural representation in mouse visual cortex. Importantly, we provide evidence that optimizing for task performance does not improve similarity to the corresponding biological system beyond a certain point. We demonstrate that the distributions of some physiological quantities are closer to the observed distributions in the mouse brain after task training. We encourage the use of the MouseNet architecture by making the code freely available. Author summary Task-driven deep neural networks have shown great potential in predicting functional responses of biological neurons. Nevertheless, they are not precise biological analogues, raising questions about how they should be interpreted. Here, we build new deep neural network models of the mouse visual cortex (MouseNet) that are biologically constrained in detail, not only in terms of the basic structure of their connectivity, but also in terms of the count and hence density of neurons within each area, and the spatial extent of their projections. Equipped with the MouseNet model, we can address key questions about mesoscale brain architecture and its role in task learning and performance.We ask, and provide a first set of answers, to: What is the performance of a mouse brain-sized – and mouse brain-structured – model on benchmark image classification tasks? How does the training of a network on this task affect the functional properties of specified layers within the biologically constrained architecture – both overall, and in comparison with recorded function of mouse neurons? We anticipate much future work on allied questions, and the development of more sophisticated models in both mouse and other species, based on the freely available MouseNet model and code which we develop and provide here.

A Rich Source of Labels for Deep Network Models of the Primate Dorsal Visual Stream

Exploring the Brain-like Properties of Deep Neural Networks: A Neural Encoding Perspective

Prune and distill: similar reformatting of image information along rat visual cortex and deep neural networks

Scaling Laws for Task-Optimized Models of the Primate Visual Ventral Stream

A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs

CNN MouseNet: A biologically constrained convolutional neural network model for mouse visual cortex

Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

Explaining V1 Properties with a Biologically Constrained Deep Learning Architecture

Convolutional neural networks for vision neuroscience: significance, developments, and outstanding issues

Complex Properties of Training Stimuli Affect Brain Alignment in a Deep Network Model of Mouse Visual Cortex

Vision CNNs trained to estimate spatial latents learned similar ventral-stream-aligned representations

Explaining face representation in the primate brain using different computational models

MouseNet: A biologically constrained convolutional neural network model for the mouse visual cortex

Revealing Fine Structures of the Retinal Receptive Field by Deep-Learning Networks

Online Discriminative Sparse Coding As a Systems-level Model of the Primate's Dorsal and Ventral Pathways

Saliency Prediction on Omnidirectional Images with Brain-Like Shallow Neural Network.

Biologically Inspired Visual Model With Preliminary Cognition and Active Attention Adjustment

Comparison Against Task Driven Artificial Neural Networks Reveals Functional Organization of Mouse Visual Cortex

Brain Decodes Deep Nets

Multimodal Deep Learning Model Unveils Behavioral Dynamics of V1 Activity in Freely Moving Mice

Do Topographic Deep ANN Models of the Primate Ventral Stream Predict the Perceptual Effects of Direct IT Cortical Interventions?