Abstract:Depression is a prevalent mental disorder affecting a significant portion of the global population, leading to considerable disability and contributing to the overall burden of disease. Consequently, designing efficient and robust automated methods for depression detection has become imperative. Recently, deep learning methods, especially multimodal fusion methods, have been increasingly used in computer-aided depression detection. Importantly, individuals with depression and those without respond differently to various emotional stimuli, providing valuable information for detecting depression. Building on these observations, we propose an intra- and inter-emotional stimulus transformer-based fusion model to effectively extract depression-related features. The intra-emotional stimulus fusion framework aims to prioritize different modalities, capitalizing on their diversity and complementarity for depression detection. The inter-emotional stimulus model maps each emotional stimulus onto both invariant and specific subspaces using individual invariant and specific encoders. The emotional stimulus-invariant subspace facilitates efficient information sharing and integration across different emotional stimulus categories, while the emotional stimulus specific subspace seeks to enhance diversity and capture the distinct characteristics of individual emotional stimulus categories. Our proposed intra- and inter-emotional stimulus fusion model effectively integrates multimodal data under various emotional stimulus categories, providing a comprehensive representation that allows accurate task predictions in the context of depression detection. We evaluate the proposed model on the Chinese Soochow University students dataset, and the results outperform state-of-the-art models in terms of concordance correlation coefficient (CCC), root mean squared error (RMSE) and accuracy.

Combining Multiple Sensor Features for Stress Detection Using Combinatorial Fusion

Sensor Feature Selection and Combination for Stress Identification Using Combinatorial Fusion

Feature Selection and Combination for Stress Identification Using Correlation and Diversity

An Intra- and Inter-Emotion Transformer-Based Fusion Model with Homogeneous and Diverse Constraints Using Multi-Emotional Audiovisual Features for Depression Detection.

Multimodal Hierarchical CNN Feature Fusion for Stress Detection

Multimodal Stress Detection Using Facial Landmarks and Biometric Signals

A Mental Stress Classification Method Based on Feature Fusion Using Physiological Signals

Evaluating Feature Selection for Stress Identification

Dynamic Alignment and Fusion of Multimodal Physiological Patterns for Stress Recognition

An Investigation of Decision Analytic Methodologies for Stress Identification

Early Detection of Mental Stress Using Advanced Neuroimaging and Artificial Intelligence

A Multimodal Late Fusion Framework for Physiological Sensor and Audio-Signal-Based Stress Detection: An Experimental Study and Public Dataset

SELF-CARE: Selective Fusion with Context-Aware Low-Power Edge Computing for Stress Detection

AFCNet: A Fusion Method of Multi-Sensor Data Based on Adaptive Feature Complementation

Driver stress detection via multimodal fusion using attention-based CNN-LSTM

Shuffled ECA-Net for stress detection from multimodal wearable sensor data

Multi-level Stress Assessment Using Multi-domain Fusion of ECG Signal

Multi-modal Interactive Fusion Method for Detecting Teenagers Psychological Stress.

Stress Detection using Context-Aware Sensor Fusion from Wearable Devices

A Method for Stress Detection Based on FCM Algorithm

Attention-Block Deep Learning Based Features Fusion in Wearable Social Sensor for Mental Wellbeing Evaluations