Abstract:Nowadays, depression heavily affects humans' physical and mental health. Depression occurs due to changes in mood, loss of interest, and stress, which leads to self-harm events and suicide. Thus analyzing depression is very important to reduce suicidal acts. In recent years, automatic depression evaluation has been developed in computer vision technology. Several models were investigated for depression analysis, but they are limited only to video and audio data analysis. In this paper, hybrid Artificial Intelligence (AI) based Multi-modal depression analysis was proposed in which the severity of depression from multi-modal data such as video, audio and text descriptors are extracted. Initially, the proposed approach estimates the Patient Health Questionnaire (PHQ) depression scale by a hybrid framework Residual Network based Deep Neural Network (D-ResNet), which computes the PHQ-8 score from video and audio features. Then, Paragraph Vector Kernel Extreme Learning Machine (PV-KELM) is developed to infer the mental and physical states of the individuals related to the psychoanalytic features of depression. It recognizes the absence (or) presence of the measured psychoanalytic symptoms. Finally, the estimated PHQ-8 score and psychoanalytic symptoms are extracted from the Residual Network based Deep Neural Network and the Paragraph Vector based Kernel Extreme Learning Machine, which is fed together into the ensemble classifier. In the ensemble classifier, three classifiers are used, namely Support Vector Machine (SVM), Naive-Bayes (NB), and Decision Tree (DT) classifier, to classify whether the individual is depressed or not. The proposed approach is implemented in PYTHON software, and the experiments will be carried out using the Distress Analysis Interview Corpus-Wizard of -OZ interview depression dataset. By using the proposed approach, the accuracy, precision, recall, F-measure, RMSE, MAE, JSD and Contextual similarity obtained are 0.89, 0.86, 0.86 and 0.86, 0.373, 0.35, 0.355 and 0.689 respectively. Our proposed approach has been compared with the state-of-the-art approaches, and the performance result shows the efficiency of the proposed approach.

Multimodal, multiview and multitasking depression detection framework endorsed with auxiliary sentiment polarity and emotion detection

Multi-Modal and Multi-Task Depression Detection with Sentiment Assistance

Automatic Depression Prediction Via Cross-Modal Attention-Based Multi-Modal Fusion in Social Networks

An Intra- and Inter-Emotion Transformer-Based Fusion Model with Homogeneous and Diverse Constraints Using Multi-Emotional Audiovisual Features for Depression Detection.

Multimodal Depression Severity Prediction from medical bio-markers using Machine Learning Tools and Technologies

SenseMood: Depression Detection on Social Media

A novel multi-modal depression detection approach based on mobile crowd sensing and task-based mechanisms

A Multimodal Approach for Detection and Assessment of Depression Using Text, Audio and Video

A Hybrid BERT-CNN Approach for Depression Detection on Social Media Using Multimodal Data

Multi-Task Learning for Jointly Detecting Depression and Emotion.

Classification of Depression on social media using Distant Supervision

Real Time Multimodal Depression Analysis

Enhancing depression detection: A multimodal approach with text extension and content fusion

An ensemble approach to detect depression from social media platform: E-CLS

Textual-dominated Multimodal Depression Detection

D-ResNet-PVKELM: deep neural network and paragraph vector based kernel extreme machine learning model for multimodal depression analysis

Cooperative Multimodal Approach to Depression Detection in Twitter

Depression Detection Via Harvesting Social Media: A Multimodal Dictionary Learning Solution

Multimodal mental health analysis in social media

Fusing Visual, Textual and Connectivity Clues for Studying Mental Health

Reading Between the Frames: Multi-Modal Depression Detection in Videos from Non-Verbal Cues