Abstract:Multimodal functional neuroimaging by integrating functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) has the promise of recovering brain activities with high spatiotemporal resolution, which is crucial for neuroscience research and clinical diagnosis. However, the misalignment of the localizations between fMRI and EEG activities may degrade the accuracy of the fMRI-constrained EEG source imaging (ESI) technique. To leverage the complementary spatiotemporal resolution of fMRI and EEG in a data-driven fashion, we propose an asymmetric approach for EEG/fMRI fusion, termed fMRI-informed source imaging based on spatiotemporal basis functions (fMRI-SI-STBF). fMRI-SI-STBF employs the covariance components (CCs) derived from clusters defined by fMRI and EEG signals as spatial priors within the empirical Bayesian framework. Additionally, fMRI-SI-STBF represents the current source matrix as a linear combination of several unknown temporal basis functions (TBFs) by matrix decomposition. The relative contribution of each of the fMRI-informed and EEG-informed CCs, as well as the number and profiles of the TBFs, are all automatically determined based on the EEG data using variational Bayesian inference. Our results demonstrate that fMRI-SI-STBF can effectively utilize valid fMRI information for ESI and is robust to invalid fMRI priors. This robustness is essential for practical ESI since the validity of fMRI priors is often unclear considering that fMRI is an indirect measure of neural activity. Moreover, fMRI-SI-STBF can achieve performance improvement by incorporating temporal constraints compared to methods that use spatial constraints only. For the numerical simulations, fMRI-SI-STBF reconstructs the source extents, locations and time courses more accurately than existing EEG-fMRI ESI methods (i.e., fwMNE, fMRI-SI-SBF) and ESI methods without fMRI priors (i.e., wMNE, LORETA, SBL, SI-STBF, SI-SBF), indicated by the smaller spatial dispersion (average SD <math><5</math> mm), distance of localization error (average DLE <math><2</math> mm), shape error (average SE <math><0.9</math>) and larger model evidence values.

Integration of Simultaneous Fmri and EEG Source Localization in Emotional Decision Problems

Gender Differences in the Processing of Standard Emotional Visual Stimuli: Integrating ERP and Fmri Results

Single-Trial Eeg-Informed Fmri Analysis of Emotional Decision Problems in Hot Executive Function

Multisensory integration of dynamic emotional faces and voices: method for simultaneous EEG-fMRI measurements

Feature Fusion Analysis of Simultaneously Recorded EEG-fMRI in Emotion Cognitive Reappraisal

Source localization and functional network analysis in emotion cognitive reappraisal with EEG-fMRI integration

A Comparison Between Eeg Source Localization and Fmri During the Processing of Emotional Visual Stimuli

A study of neural mechanism in emotion regulation by simultaneous recording of EEG and fMRI based on ICA

Emotion Recognition Based on a EEG–fNIRS Hybrid Brain Network in the Source Space

Enhancing EEG-Based Decision-Making Performance Prediction by Maximizing Mutual Information Between Emotion and Decision-Relevant Features

Evaluation of Brain Source Localization Methods Based on Test-Retest Reliability with Multiple Session EEG Data.

Integration of EEG/MEG with MRI and Fmri in Functional Neuroimaging

Feedback-related Brain Activity in Individual Decision: Evidence from a Gambling EEG Study.

Exploring Neural Mechanisms of Reward Processing Using Coupled Matrix Tensor Factorization: A Simultaneous EEG–fMRI Investigation

Differential effects of specific emotions on spatial decision-making: evidence from cross-frequency functionally independent brain networks

Deep EEG source localization via EMD-based fMRI high spatial frequency

fMRI-SI-STBF: An fMRI-informed Bayesian electromagnetic spatio-temporal extended source imaging

A Realistic Framework for Investigating Decision Making in the Brain With High Spatiotemporal Resolution Using Simultaneous EEG/fMRI and Joint ICA

Predicting individual decision-making responses based on the functional connectivity of resting-state EEG.

Simultaneous EEG-fMRI: Trial level spatio-temporal fusion for hierarchically reliable information discovery

Investigating EEG-based functional connectivity patterns for multimodal emotion recognition