Abstract:Although modern imaging technologies allow us to study connectivity between two distinct brain regions in-vivo, an in-depth understanding of how anatomical structure supports brain function and how spontaneous functional fluctuations emerge remarkable cognition is still elusive. Meanwhile, tremendous efforts have been made in the realm of machine learning to establish the nonlinear mapping between neuroimaging data and phenotypic traits. However, the absence of neuroscience insight in the current approaches poses significant challenges in understanding cognitive behavior from transient neural activities. To address this challenge, we put the spotlight on the coupling mechanism of structural connectivity (SC) and functional connectivity (FC) by formulating such network neuroscience question into an expressive graph representation learning problem for high-order topology. Specifically, we introduce the concept of topological detour to characterize how a ubiquitous instance of FC (direct link) is supported by neural pathways (detour) physically wired by SC, which forms a cyclic loop interacted by brain structure and function. In the cliché of machine learning, the multi-hop detour pathway underlying SC-FC coupling allows us to devise a novel multi-head self-attention mechanism within Transformer to capture multi-modal feature representation from paired graphs of SC and FC. Taken together, we propose a biological-inspired deep model, coined as NeuroPath, to find putative connectomic feature representations from the unprecedented amount of neuroimages, which can be plugged into various downstream applications such as task recognition and disease diagnosis. We have evaluated NeuroPath on large-scale public datasets including HCP and UK Biobank under supervised and zero-shot learning, where the state-of-the-art performance by our NeuroPath indicates great potential in network neuroscience.

Brain-inspired Dual-Pathway Neural Network Architecture and Its Generalization Analysis

Brain Cognition-Inspired Dual-Pathway CNN Architecture for Image Classification

A Dual-Stream Neural Network Explains the Functional Segregation of Dorsal and Ventral Visual Pathways in Human Brains

Representations and generalization in artificial and brain neural networks

A Neural Network Model With Gap Junction for Topological Detection

Determinantal Point Process Attention Over Grid Cell Code Supports Out of Distribution Generalization

CP-CNN: Core-Periphery Principle Guided Convolutional Neural Network

A Deep Convolutional Neural Network for Topology Optimization with Perceptible Generalization Ability

Multiscale Brain-Like Neural Network for Saliency Prediction on Omnidirectional Images

Experimental Observations of the Topology of Convolutional Neural Network Activations

Dissociable Neural Representations of Adversarially Perturbed Images in Convolutional Neural Networks and the Human Brain

Introducing topography in convolutional neural networks

NeuroPath: A Neural Pathway Transformer for Joining the Dots of Human Connectomes

Emergent Neural Network Mechanisms for Generalization to Objects in Novel Orientations

Exploring the Brain-like Properties of Deep Neural Networks: A Neural Encoding Perspective

Exploring Brain Effective Connectivity in Visual Perception Using a Hierarchical Correlation Network.

Brain inspired Robust Vision using Convolutional Neural Networks with Feedback

Topographic Property of Backpropagation Artificial Neural Network: from Human Functional Connectivity Network to Artificial Neural Network

Analogy-Detail Networks for Object Recognition

Local lateral connectivity is sufficient for replicating cortex-like topographical organization in deep neural networks