Abstract:Fusing complementary information from different modalities can lead to the discovery of more accurate diagnostic biomarkers for psychiatric disorders. However, biomarker discovery through data fusion is challenging since it requires extracting interpretable and reproducible patterns from data sets, consisting of shared/unshared patterns and of different orders. For example, multi-channel electroencephalography (EEG) signals from multiple subjects can be represented as a third-order tensor with modes: subject, time, and channel, while functional magnetic resonance imaging (fMRI) data may be in the form of subject by voxel matrices. Traditional data fusion methods rearrange higher-order tensors, such as EEG, as matrices to use matrix factorization-based approaches. In contrast, fusion methods based on coupled matrix and tensor factorizations (CMTF) exploit the potential multi-way structure of higher-order tensors. The CMTF approach has been shown to capture underlying patterns more accurately without imposing strong constraints on the latent neural patterns, i.e., biomarkers. In this paper, EEG, fMRI, and structural MRI (sMRI) data collected during an auditory oddball task (AOD) from a group of subjects consisting of patients with schizophrenia and healthy controls, are arranged as matrices and higher-order tensors coupled along the subject mode, and jointly analyzed using structure-revealing CMTF methods [also known as advanced CMTF (ACMTF)] focusing on unique identification of underlying patterns in the presence of shared/unshared patterns. We demonstrate that joint analysis of the EEG tensor and fMRI matrix using ACMTF reveals significant and biologically meaningful components in terms of differentiating between patients with schizophrenia and healthy controls while also providing spatial patterns with high resolution and improving the clustering performance compared to the analysis of only the EEG tensor. We also show that these patterns are reproducible, and study reproducibility for different model parameters. In comparison to the joint independent component analysis (jICA) data fusion approach, ACMTF provides easier interpretation of EEG data by revealing a single summary map of the topography for each component. Furthermore, fusion of sMRI data with EEG and fMRI through an ACMTF model provides structural patterns; however, we also show that when fusing data sets from multiple modalities, hence of very different nature, preprocessing plays a crucial role.

Tensor decompositions and data fusion in epileptic electroencephalography and functional magnetic resonance imaging data

Tensor Decomposition of Large-scale Clinical EEGs Reveals Interpretable Patterns of Brain Physiology

Early soft and flexible fusion of EEG and fMRI via tensor decompositions

Augmenting interictal mapping with neurovascular coupling biomarkers by structured factorization of epileptic EEG and fMRI data

Early soft and flexible fusion of electroencephalography and functional magnetic resonance imaging via double coupled matrix tensor factorization for multisubject group analysis

Multi-channel EEG epileptic spike detection by a new method of tensor decomposition

Advanced framework for epilepsy detection through image-based EEG signal analysis

Tensor-driven extraction of developmental features from varying paediatric EEG datasets

Functional Source Separation for EEG-fMRI Fusion: Application to Steady-State Visual Evoked Potentials

Tensor Analysis and Fusion of Multimodal Brain Images

A DF-SSA analytical framework for revealing variations in multidimensional EEG features of epileptic seizures

A Graph Fourier Transform Based Bidirectional Long Short-Term Memory Neural Network for Electrophysiological Source Imaging

Unraveling Diagnostic Biomarkers of Schizophrenia Through Structure-Revealing Fusion of Multi-Modal Neuroimaging Data

Simultaneous EEG-fMRI: Trial level spatio-temporal fusion for hierarchically reliable information discovery

A Review of EEG-based Localization of Epileptic Seizure Foci: Common Points with Multimodal Fusion of Brain Data

Multi-Dimensional Framework for EEG Signal Processing and Denoising Through Tensor-based Architecture

Identification of Interictal Epileptic Networks from Dense-EEG

Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals

Two-Dimensional Temporal Clustering Analysis for Patients with Epilepsy: Detecting Epilepsy-Related Information in EEG-fMRI Concordant, Discordant and Spike-Less Patients

A New Generation of Brain-Computer Interfaces Driven by Discovery of Latent EEG-fMRI Linkages Using Tensor Decomposition

Prediction of Epilepsy Based on Tensor Decomposition and Functional Brain Network