Abstract:Since multimodal emotion classification in different human states has rarely been studied, this paper explores the emotional mechanisms of the brain functional connectivity networks after emotional stimulation. We devise a multimodal emotion classification method fusing a brain functional connectivity network based on electroencephalography (EEG) and eye gaze (ECFCEG) to study emotional mechanisms. First, the nonlinear phase lag index (PLI) and phase-locked value (PLV) are calculated to construct the multiband brain functional connectivity networks, which are then converted into binary brain networks, and the seven features of the binary brain networks are extracted. At the same time, the features of the eye gaze signals are extracted. Then, a fusion algorithm called kernel canonical correlation analysis, based on feature level and randomization (FRKCCA), is executed for feature-level fusion (FLF) of brain functional connectivity networks and eye gaze. Finally, support vector machines (SVMs) are utilized to classify positive and negative emotions in multiple frequency bands with single modal features and multimodal features. The experimental results demonstrate that multimodal complementary representation properties can effectively improve the accuracy of emotion classification, achieving a classification accuracy of 91.32 +/- 1.81%. The classification accuracy of pupil diameter in the valence dimension is higher than that of additional features. In addition, the average emotion classification effect of the valence dimension is preferable to that of arousal. Our findings demonstrate that the brain functional connectivity networks of the right brain exhibit a deficiency. In particular, the information processing ability of the right temporal (RT) and right posterior (RP) regions is weak in the low frequency after emotional stimulation; Conversely, phase synchronization of the brain functional connectivity networks based on PLI is stronger than that of PLV.

A Bayesian incorporated linear non-Gaussian acyclic model for multiple directed graph estimation to study brain emotion circuit development in adolescence

A Bayesian incorporated linear non-Gaussian acyclic model for multiple directed graph estimation to study brain emotion circuit development in adolescence

Modeling Brain Functional Dynamics Via Hidden Markov Models

Development of human emotion circuits investigated using a Big-Data analytic approach: Stability, reliability, and robustness.

Functional network estimation using multigraph learning with application to brain maturation study

Brain Maturation Study during Adolescence Using Graph Laplacian Learning Based Fourier Transform

Cross-sectional exploration of brain functional connectivity in the triadic development model of adolescents

EEG-Based Emotion Recognition Using Partial Directed Coherence Dense Graph Propagation

The role of functional emotion circuits in distinct dimensions of psychopathology in youth

Brain Network Manifold Learned by Cognition-Inspired Graph Embedding Model for Emotion Recognition

Causal inference of brain connectivity from fMRI with $ψ$-Learning Incorporated Linear non-Gaussian Acyclic Model ($ψ$-LiNGAM)

Inference-based statistical network analysis uncovers star-like brain functional architectures for internalizing psychopathology in children

Bayesian subtyping for multi-state brain functional connectome with application on adolescent brain cognition

A Cortical Folding Pattern-Guided Model of Intrinsic Functional Brain Networks in Emotion Processing

Immature Integration and Segregation of Emotion-Related Brain Circuitry in Young Children

Functional hypergraph uncovers novel covariant structures over neurodevelopment

Neural Correlates of Incidental and Directed Facial Emotion Processing in Adolescents and Adults.

Functional Connectivity Predicts Individual Development of Inhibitory Control during Adolescence

Emotion Recognition Using Dynamic Graph Metrics Based on Functional Integration and Separation of EEG Brain Network

Using Real-Time Fmri to Influence Effective Connectivity in the Developing Emotion Regulation Network

Multimodal Emotion Classification Method and Analysis of Brain Functional Connectivity Networks