Abstract:Naturalistic stimuli, including movie, music, and speech, have been increasingly applied in the research of neuroimaging. Relative to a resting-state or single-task state, naturalistic stimuli can evoke more intense brain activities and have been proved to possess higher test-retest reliability, suggesting greater potential to study adaptive human brain function. In the current research, naturalistic functional magnetic resonance imaging (N-fMRI) has been a powerful tool to record brain states under naturalistic stimuli, and many efforts have been devoted to study the high-level semantic features from spatial or temporal representations via N-fMRI. However, integrating both spatial and temporal characteristics of brain activities for better interpreting the patterns under naturalistic stimuli is still underexplored. In this work, a novel hybrid learning framework that comprehensively investigates both the spatial (via Predictive Model) and the temporal [via convolutional neural network (CNN) model] characteristics of the brain is proposed. Specifically, to focus on certain relevant regions from the whole brain, regions of significance (ROS), which contain common spatial activation characteristics across individuals, are selected via the Predictive Model. Further, voxels of significance (VOS), whose signals contain significant temporal characteristics under naturalistic stimuli, are interpreted via one-dimensional CNN (1D-CNN) model. In this article, our proposed framework is applied onto the N-fMRI data during naturalistic classical/pop/speech audios stimuli. The promising performance is achieved via the Predictive Model to differentiate the different audio categories. Especially for distinguishing the classic and speech audios, the accuracy of classification is up to 92%. Moreover, spatial ROS and VOS are effectively obtained. Besides, temporal characteristics of the high-level semantic features are investigated on the frequency domain via convolution kernels of 1D-CNN model, and we effectively bridge the "semantic gap" between high-level semantic features of N-fMRI and low-level acoustic features of naturalistic audios in the frequency domain. Our results provide novel insights on characterizing spatiotemporal patterns of brain activities via N-fMRI and effectively explore the high-level semantic features under naturalistic stimuli, which will further benefit the understanding of the brain working mechanism and the advance of naturalistic stimuli clinical application.

Frequency-specific Functional Difference Between Gyri and Sulci in Naturalistic Paradigm Fmri

Exploring Functional Difference Between Gyri and Sulci via Region-Specific 1D Convolutional Neural Networks.

Deep Learning Models Unveiled Functional Difference Between Cortical Gyri and Sulci

Modeling functional difference between gyri and sulci within intrinsic connectivity networks

Gyral-sulcal Contrast in Intrinsic Functional Brain Networks Across Task Performances

Exploring Intrinsic Functional Differences of Gyri, Sulci and 2-Hinge, 3-Hinge Joints on Cerebral Cortex.

Modeling Functional Dynamics of Cortical Gyri and Sulci.

Elucidating Functional Differences Between Cortical Gyri and Sulci Via Sparse Representation HCP Grayordinate Fmri Data

Regularity and Variability of Functional Brain Connectivity Characteristics Between Gyri and Sulci under Naturalistic Stimulus.

The Cerebral Cortex is Bisectionally Segregated into Two Fundamentally Different Functional Units of Gyri and Sulci.

Exploring Gyro-Sulcal Functional Connectivity Differences Across Task Domains Via Anatomy-Guided Spatio-Temporal Graph Convolutional Networks

A Functional Model of Cortical Gyri and Sulci

Functional Subdivisions of the Cerebellum in Naturalistic Paradigm Functional Magnetic Resonance Imaging

Anatomy-Guided Spatio-Temporal Graph Convolutional Networks (Ag-Stgcns) for Modeling Functional Connectivity Between Gyri and Sulci Across Multiple Task Domains.

Hierarchical functional differences between gyri and sulci at different scales

Modeling refined differences of cortical folding patterns via spatial, morphological, and temporal fusion representations

Cortical Morphological Networks Differ Between Gyri and Sulci

Structure-Function Relationship in Electrical and Hemodynamic Brain Networks: Insights from EEG and fNIRS during Rest and Task States

Fundamental Functional Differences Between Gyri and Sulci: Implications for Brain Function, Cognition, and Behavior.

Lateral frontoparietal functional connectivity based on individual sulcal morphology

A hybrid learning framework for fine-grained interpretation of brain spatiotemporal patterns during naturalistic functional magnetic resonance imaging