Abstract:Depression is a prevalent mental disorder affecting a significant portion of the global population, leading to considerable disability and contributing to the overall burden of disease. Consequently, designing efficient and robust automated methods for depression detection has become imperative. Recently, deep learning methods, especially multimodal fusion methods, have been increasingly used in computer-aided depression detection. Importantly, individuals with depression and those without respond differently to various emotional stimuli, providing valuable information for detecting depression. Building on these observations, we propose an intra- and inter-emotional stimulus transformer-based fusion model to effectively extract depression-related features. The intra-emotional stimulus fusion framework aims to prioritize different modalities, capitalizing on their diversity and complementarity for depression detection. The inter-emotional stimulus model maps each emotional stimulus onto both invariant and specific subspaces using individual invariant and specific encoders. The emotional stimulus-invariant subspace facilitates efficient information sharing and integration across different emotional stimulus categories, while the emotional stimulus specific subspace seeks to enhance diversity and capture the distinct characteristics of individual emotional stimulus categories. Our proposed intra- and inter-emotional stimulus fusion model effectively integrates multimodal data under various emotional stimulus categories, providing a comprehensive representation that allows accurate task predictions in the context of depression detection. We evaluate the proposed model on the Chinese Soochow University students dataset, and the results outperform state-of-the-art models in terms of concordance correlation coefficient (CCC), root mean squared error (RMSE) and accuracy.

Toward Sensing Emotions With Deep Visual Analysis: A Long-Term Psychological Modeling Approach

An Intra- and Inter-Emotion Transformer-Based Fusion Model with Homogeneous and Diverse Constraints Using Multi-Emotional Audiovisual Features for Depression Detection.

A Conceptual Framework of Cognitive-Affective Theory of Mind: Towards a Precision Identification of Mental Disorders

Multi-Modal and Multi-Task Depression Detection with Sentiment Assistance

Automatic Depression Prediction Via Cross-Modal Attention-Based Multi-Modal Fusion in Social Networks

Dynamic Facial Features in Positive-Emotional Speech for Identification of Depressive Tendencies

Emotion-based Modeling of Mental Disorders on Social Media

Faces of the Mind: Unveiling Mental Health States Through Facial Expressions in 11,427 Adolescents

A Neural Predictive Model of Negative Emotions for COVID-19

Enhancing depression detection: A multimodal approach with text extension and content fusion

An Intelligent Depression Detection Model Based On Multimodal Fusion Technology

A measurement method for mental health based on dynamic multimodal feature recognition

A multi-featured expression recognition model incorporating attention mechanism and object detection structure for psychological problem diagnosis

Toward Multimodal Modeling of Emotional Expressiveness

Construction of an Intelligent Evaluation Model of Mental Health Based on Big Data

Multimodal Sensing for Depression Risk Detection: Integrating Audio, Video, and Text Data

A Mental Health Assessment Model of College Students Using Intelligent Technology

A Cross-subject and Cross-modal Model for Multimodal Emotion Recognition

Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing Approaches

Emotion recognition based on brain-like multimodal hierarchical perception

Real-time emotional health detection using fine-tuned transfer networks with multimodal fusion