Abstract:A novel music/speech analysis framework via integration of audio content and functional brain response.Brain response features are derived from fMRI data when participants are listening to the music/speech.A novel regression algorithm named ITGP is proposed to improve the quality in predicting functional brain response. Effective analysis of music/speech data such as clustering, retrieval, and classification has received significant attention in recent years. Traditional methods mainly rely on the low-level acoustic features derived from digital audio stream, and the accuracy of these methods is limited by the well-known semantic gap. To alleviate this problem, we propose a novel framework for music/speech clustering, retrieval, and classification by integrating the low-level acoustic features derived from audio content with the functional magnetic resonance imaging (fMRI) measured features that represent the brain's functional response when subjects are listening to the music/speech excerpts. First, the brain networks and regions of interest (ROIs) involved in the comprehension of audio stimuli, such as the auditory, emotion, attention, and working memory systems, are located by a new approach named dense individualized and common connectivity-based cortical landmarks (DICCCOLs). Then the functional connectivity matrix measuring the similarity between the fMRI signals of different ROIs is adopted to represent the brain's comprehension of audio semantics. Afterwards, we propose an improved twin Gaussian process (ITGP) model based on self-training to predict the fMRI-measured features of testing data without fMRI scanning. Finally, multi-view learning algorithms are proposed to integrate acoustic features with fMRI-measured features for music/speech clustering, retrieval, and classification, respectively. The experimental results demonstrate the superiority of our proposed work in comparison with existing methods and suggest the advantage of integrating functional brain responses via fMRI data for music/speech analysis.

A New Method for Detecting Brain Activities from Fmri Dataset

Detecting Brain Activations by Constrained Non-Negative Matrix Factorization from Task-Related BOLD Fmri

Clustered Cnmf for Fmri Data Analysis

Fmri Time Series Analysis Based on Stationary Wavelet and Spectrum Analysis

Multi-Dimensional Delay-Correlation MUSIC: A New Method to Extract Multi-Sources of EEGs

[A Method Based on Independent Component Analysis for Processing Fmri Data].

Data-driven analysis of functional brain interactions during free listening to music and speech

A Frequency Signal Method for Fmri Data Analysis

Clustering Analysis for Fmri Dataset Based on ISODATA Algorithm

Synchronization-based approach for detecting functional activation of brain.

Detecting Epileptic Activities from resting fMRI Time-course by using PCA algorithm

Music/speech Classification Using High-Level Features Derived from Fmri Brain Imaging.

Identifying Activated Brain Regions from Fmri Data Based on Blind Non-Negative Component Separation Technique.

Coherent EEG Sources Localization in Transformation Data Space

Delay Correlation Subspace Decomposition Algorithm and Its Application in fMRI

Adaptive Independent Subspace Analysis of Brain Magnetic Resonance Imaging Data

A practical model-based segmentation approach for improved activation detection in single-subject functional Magnetic Resonance Imaging studies

A delay sub-space decomposition algorithm of electroencephalography sources localization

Analysis of music/speech via integration of audio content and functional brain response.

A Method Based on Regression Analysis for Detecting Functional Connectivity of Human Brain

A Novel Unified SPM-ICA-PCA Method for Detecting Epileptic Activities in Resting-State fMRI