Abstract:Nowadays, depression heavily affects humans' physical and mental health. Depression occurs due to changes in mood, loss of interest, and stress, which leads to self-harm events and suicide. Thus analyzing depression is very important to reduce suicidal acts. In recent years, automatic depression evaluation has been developed in computer vision technology. Several models were investigated for depression analysis, but they are limited only to video and audio data analysis. In this paper, hybrid Artificial Intelligence (AI) based Multi-modal depression analysis was proposed in which the severity of depression from multi-modal data such as video, audio and text descriptors are extracted. Initially, the proposed approach estimates the Patient Health Questionnaire (PHQ) depression scale by a hybrid framework Residual Network based Deep Neural Network (D-ResNet), which computes the PHQ-8 score from video and audio features. Then, Paragraph Vector Kernel Extreme Learning Machine (PV-KELM) is developed to infer the mental and physical states of the individuals related to the psychoanalytic features of depression. It recognizes the absence (or) presence of the measured psychoanalytic symptoms. Finally, the estimated PHQ-8 score and psychoanalytic symptoms are extracted from the Residual Network based Deep Neural Network and the Paragraph Vector based Kernel Extreme Learning Machine, which is fed together into the ensemble classifier. In the ensemble classifier, three classifiers are used, namely Support Vector Machine (SVM), Naive-Bayes (NB), and Decision Tree (DT) classifier, to classify whether the individual is depressed or not. The proposed approach is implemented in PYTHON software, and the experiments will be carried out using the Distress Analysis Interview Corpus-Wizard of -OZ interview depression dataset. By using the proposed approach, the accuracy, precision, recall, F-measure, RMSE, MAE, JSD and Contextual similarity obtained are 0.89, 0.86, 0.86 and 0.86, 0.373, 0.35, 0.355 and 0.689 respectively. Our proposed approach has been compared with the state-of-the-art approaches, and the performance result shows the efficiency of the proposed approach.

Automated Major Depressive Disorder (AMDD) Detection using Audio-Visual Feature Discriptor and CNN

Hybrid Network Feature Extraction for Depression Assessment from Speech

Automatic Assessment of Depression from Speech Via a Hierarchical Attention Transfer Network and Attention Autoencoders

Towards Automatic Depression Detection: A BiLSTM/1D CNN-Based Model

Depression Scale Recognition from Audio, Visual and Text Analysis

Depression detection using cascaded attention based deep learning framework using speech data

A Multimodal Approach for Detection and Assessment of Depression Using Text, Audio and Video

Multi-level Attention network using text, audio and video for Depression Prediction

End-to-end multimodal system for depression detection from online recordings

Automatic Detection of Depression from Stratified Samples of Audio Data

Attention-Based Acoustic Feature Fusion Network for Depression Detection

MFCC-based Recurrent Neural Network for automatic clinical depression recognition and assessment from speech

Automatic recognition of depression based on audio and video: A review

Automated depression analysis using convolutional neural networks from speech

Depression Detection and Analysis using Large Language Models on Textual and Audio-Visual Modalities

D-ResNet-PVKELM: deep neural network and paragraph vector based kernel extreme machine learning model for multimodal depression analysis

Real Time Multimodal Depression Analysis

Multimodal Sensing for Depression Risk Detection: Integrating Audio, Video, and Text Data

Decision support system for major depression detection using spectrogram and convolution neural network with EEG signals

A Novel Audio-Visual Information Fusion System for Mental Disorders Detection

DETECTION OF DEPRESSION USING ACOUSTIC AND TREATMENT RESPONSE PREDICTION USING MACHINE LEARNING