[Clinical aspects and prognosis of Legionella infections].

H. Lode,H. Grothe,H. Schäfer,B. Kemmerich,G. Ruckdeschel,H. Ehret

Abstract:

What problem does this paper attempt to address?

Automatic Assessment of Depression from Speech Via a Hierarchical Attention Transfer Network and Attention Autoencoders

Ziping Zhao,Zhongtian Bao,Zixing Zhang,Jun Deng,Nicholas Cummins,Haishuai Wang,Jianhua Tao,Bjoern Schuller

DOI: https://doi.org/10.1109/jstsp.2019.2955012

IF: 7.695

2019-01-01

IEEE Journal of Selected Topics in Signal Processing

Abstract:Early interventions in mental health conditions such as Major Depressive Disorder (MDD) are critical to improved health outcomes, as they can help reduce the burden of the disease. As the efficient diagnosis of depression severity is therefore highly desirable, the use of behavioural cues such as speech characteristics in diagnosis is attracting increasing interest in the field of quantitative mental health research. However, despite the widespread use of machine learning methods in the depression analysis community, the lack of adequate labelled data has become a bottleneck preventing the broader application of techniques such as deep learning. Accordingly, we herein describe a deep learning approach that combines unsupervised learning, knowledge transfer and hierarchical attention for the task of speech-based depression severity measurement. Our novel approach, a Hierarchical Attention Transfer Network (HATN), uses hierarchical attention autoencoders to learn attention from a source task, followed by speech recognition, and then transfers this knowledge into a depression analysis system. Experiments based on the depression sub-challenge dataset of the Audio/Visual Emotion Challenge (AVEC) 2017 demonstrate the effectiveness of our proposed model. On the test set, our technique outperformed other speech-based systems presented in the literature, achieving a Root Mean Square Error (RMSE) of 5.51 and a Mean Absolute Error (MAE) of 4.20 on a Patient Health Questionnaire (PHQ)-8 scale [0, 24]. To the best of our knowledge, these scores represent the best-known speech results on the AVEC 2017 depression corpus to date.
An Intra- and Inter-Emotion Transformer-Based Fusion Model with Homogeneous and Diverse Constraints Using Multi-Emotional Audiovisual Features for Depression Detection.

Shiyu Teng,Jiaqing Liu,Yue Huang,Shurong Chai,Tomoko Tateyama,Xinyin Huang,Lanfen Lin,Yen-Wei Chen

DOI: https://doi.org/10.1587/transinf.2023hcp0006

2024-01-01

IEICE Transactions on Information and Systems

Abstract:Depression is a prevalent mental disorder affecting a significant portion of the global population, leading to considerable disability and contributing to the overall burden of disease. Consequently, designing efficient and robust automated methods for depression detection has become imperative. Recently, deep learning methods, especially multimodal fusion methods, have been increasingly used in computer-aided depression detection. Importantly, individuals with depression and those without respond differently to various emotional stimuli, providing valuable information for detecting depression. Building on these observations, we propose an intra- and inter-emotional stimulus transformer-based fusion model to effectively extract depression-related features. The intra-emotional stimulus fusion framework aims to prioritize different modalities, capitalizing on their diversity and complementarity for depression detection. The inter-emotional stimulus model maps each emotional stimulus onto both invariant and specific subspaces using individual invariant and specific encoders. The emotional stimulus-invariant subspace facilitates efficient information sharing and integration across different emotional stimulus categories, while the emotional stimulus specific subspace seeks to enhance diversity and capture the distinct characteristics of individual emotional stimulus categories. Our proposed intra- and inter-emotional stimulus fusion model effectively integrates multimodal data under various emotional stimulus categories, providing a comprehensive representation that allows accurate task predictions in the context of depression detection. We evaluate the proposed model on the Chinese Soochow University students dataset, and the results outperform state-of-the-art models in terms of concordance correlation coefficient (CCC), root mean squared error (RMSE) and accuracy.
Hierarchical Attention Transfer Networks for Depression Assessment from Speech

Ziping Zhao,Zhongtian Bao,Zixing Zhang,Nicholas Cummins,Haishuai Wang,Bjoern Schuller

DOI: https://doi.org/10.1109/icassp40776.2020.9053207

2020-01-01

Abstract:A growing area of mental health research is the search for speech-based objective markers for conditions such as depression. However, when combined with machine learning, this search can be challenging due to a limited amount of annotated training data. In this paper, we propose a novel crosstask approach which transfers attention mechanisms from speech recognition to aid depression severity measurement. This transfer is applied in a two-level hierarchical network which mirrors the natural hierarchical structure of speech. Experiments based on the Distress Analysis Interview Corpus - Wizard of Oz (DAIC-WOZ) dataset, as used in the 2017 Audio/Visual Emotion Challenge, demonstrate the effectiveness of our Hierarchical Attention Transfer Network. On the development set, the proposed approach achieves a root mean square error (RMSE) of 3.85, and a mean absolute error (MAE) of 2.99, on a Patient Health Questionnaire (PHQ)-8 scale [0], [24], while on the test set, it achieves an RMSE of 5.66 and an MAE of 4.28. To the best of our knowledge, these scores represent the best-known speech-only results to date on this corpus.
MFCC-based Recurrent Neural Network for automatic clinical depression recognition and assessment from speech

Emna Rejaibi,Ali Komaty,Fabrice Meriaudeau,Said Agrebi,Alice Othmani

DOI: https://doi.org/10.1016/j.bspc.2021.103107

IF: 5.1

2022-01-01

Biomedical Signal Processing and Control

Abstract:Clinical depression or Major Depressive Disorder (MDD) is a common and serious medical illness. In this paper, a deep Recurrent Neural Network-based framework is presented to detect depression and to predict its severity level from speech. Low-level and high-level audio features are extracted from audio recordings to predict the 24 scores of the Patient Health Questionnaire and the binary class of depression diagnosis. To overcome the problem of the small size of Speech Depression Recognition (SDR) datasets, expanding training labels and transferred features are considered. The proposed approach outperforms the state-of-art approaches on the DAIC-WOZ database with an overall accuracy of 76.27% and a root mean square error of 0.4 in assessing depression, while a root mean square error of 0.168 is achieved in predicting the depression severity levels. The proposed framework has several advantages (fastness, non-invasiveness, and non-intrusion), which makes it convenient for real-time applications. The performances of the proposed approach are evaluated under a multi-modal and a multi-features experiments. MFCC based high-level features hold relevant information related to depression. Yet, adding visual action units and different other acoustic features further boosts the classification results by 20% and 10% to reach an accuracy of 95.6% and 86%, respectively. Considering visual-facial modality needs to be carefully studied as it sparks patient privacy concerns while adding more acoustic features increases the computation time.

engineering, biomedical
Multimodal Measurement of Depression Using Deep Learning Models

Le Yang,Dongmei Jiang,Xiaohan Xia,Ercheng Pei,Meshia Cédric Oveneke,Hichem Sahli

DOI: https://doi.org/10.1145/3133944.3133948

2017-10-23

Abstract:This paper addresses multi-modal depression analysis. We propose a multi-modal fusion framework composed of deep convolutional neural network (DCNN) and deep neural network (DNN) models. Our framework considers audio, video and text streams. For each modality, handcrafted feature descriptors are input into a DCNN to learn high-level global features with compact dynamic information, then the learned features are fed to a DNN to predict the PHQ-8 scores. For multi-modal fusion, the estimated PHQ-8 scores from the three modalities are integrated in a DNN to obtain the final PHQ-8 score. Moreover, in this work, we propose new feature descriptors for text and video. For the text descriptors, we select the participant»s answers to the questions associated with psychoanalytic aspects of depression, such as sleep disorder, and make use of the Paragraph Vector (PV) to learn the distributed representations of these sentences. For the video descriptors, we propose a new global descriptor, the Histogram of Displacement Range (HDR), calculated directly from the facial landmarks to measure their displacements and speed. Experiments have been carried out on the AVEC2017 depression sub-challenge dataset. The obtained results show that the proposed depression recognition framework obtains very promising accuracy, with the root mean square error (RMSE) as 4.653, mean absolute error (MAE) as 3.980 on the development set, and RMSE as 5.974, MAE as 5.163 on the test set.
End-to-end multimodal system for depression detection from online recordings

Mateusz Kowalewski,Maciej Stroinski,Kamil Kwarciak,Volodymyr Laptiev,Daria Hemmerling

DOI: https://doi.org/10.1109/EMBC40787.2023.10340782

Abstract:Depression is one of the most occurring civilizational diseases. In this paper, we propose a new approach for detecting depression through the analysis of social media content using face analysis, emotion recognition neural networks, and speech processing. We utilized audio-visual analysis and acquired more than 605 features in the time domain. Those are fed to machine learning and deep learning models for depression classification. Our approach outperforms the other state-of-the-art models, achieving the F1-score 0.77. The results have the potential to provide valuable insights for mental health professionals, offer early detection and intervention, and serve as a resource for individuals seeking help with their mental health. This study enables real-time analysis and represents a significant advancement in mental health and technology and has the potential to impact society.Clinical relevance-The system aims to provide a fast and accurate way to detect depression in individuals through online recordings. The use of multimodal information (e.g. audio, image) enhances the performance of the non-verbal behavioral analysis. The end-to-end system reduces the need for manual analysis by mental health professionals and increases the efficiency of depression screening. The system can potentially help identify individuals who are at risk for depression, enabling early intervention and treatment. The results from the system can complement traditional assessments and support mental health professionals in making a diagnosis. The system can be used in real-time processing, f.e. during online calls, and provide objective measurements summarizing the overall behavior based on computer vision and audio analysis.
AudVowelConsNet: A phoneme-level based deep CNN architecture for clinical depression diagnosis

Muhammad Muzammel,Hanan Salam,Yann Hoffmann,Mohamed Chetouani,Alice Othmani

DOI: https://doi.org/10.1016/j.mlwa.2020.100005

2020-12-01

Machine Learning with Applications

Abstract:Depression is a common and serious mood disorder that negatively affects the patient’s capacity of functioning normally in daily tasks. Speech is proven to be a vigorous tool in depression diagnosis. Research in psychiatry concentrated on performing fine-grained analysis on word-level speech components contributing to the manifestation of depression in speech and revealed significant variations at the phoneme-level in depressed speech. On the other hand, research in Machine Learning-based automatic recognition of depression from speech focused on the exploration of various acoustic features for the detection of depression and its severity level. Few have focused on incorporating phoneme-level speech components in automatic assessment systems. In this paper, we propose an Artificial Intelligence (AI) based application for clinical depression recognition and assessment from speech. We investigate the acoustic characteristics of phoneme units, specifically vowels and consonants for depression recognition via Deep Learning. We present and compare three spectrogram-based Deep Neural Network architectures, trained on phoneme consonant and vowel units and their fusion respectively. Our experiments show that the deep learned consonant-based acoustic characteristics lead to better recognition results than vowel-based ones. The fusion of vowel and consonant speech characteristics through a deep network significantly outperforms the single space networks as well as the state-of-art deep learning approaches on the DAIC-WOZ database.

English Else
Design of polydiacetylene-phospholipid supramolecules for enhanced stability and sensitivity.

D. Kang,Ho-Sup Jung,Jiseok Lee,Sungbaek Seo,Jinsang Kim,Keesung Kim,K. Suh

DOI: https://doi.org/10.1021/la300863d

IF: 3.9

2012-05-02

Langmuir

Abstract:We present polydiacetylene (PDA) liposome assemblies with various phospholipids that have different headgroup charges and phase transition temperatures (T(m)). 10,12-Pentacosadiynoic acid (PCDA)-epoxy was used as a base PDA monomer and the insertion of highly charged phospholipids resulted in notable changes in the size of liposome and reduction of the aggregation of PDA liposome. Among the various phospholipids, the phospholipid with a moderate T(m) demonstrated enhanced stability and sensitivity, as measured by the size and zeta potential over storage time, thermochoromic response, and transmission electron microscopy images. By combining these results, we were able to detect immunologically an antibody of bovine viral diarrhea virus over a wide dynamic range of 0.001 to 100 μg/mL.
Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121

Musyyab Yousufi,Robertas Damaševičius,Rytis Maskeliūnas

DOI: https://doi.org/10.3390/brainsci14101018

IF: 3.333

2024-10-16

Brain Sciences

Abstract:Background/Objectives: This study investigates the classification of Major Depressive Disorder (MDD) using electroencephalography (EEG) Short-Time Fourier-Transform (STFT) spectrograms and audio Mel-spectrogram data of 52 subjects. The objective is to develop a multimodal classification model that integrates audio and EEG data to accurately identify depressive tendencies. Methods: We utilized the Multimodal open dataset for Mental Disorder Analysis (MODMA) and trained a pre-trained Densenet121 model using transfer learning. Features from both the EEG and audio modalities were extracted and concatenated before being passed through the final classification layer. Additionally, an ablation study was conducted on both datasets separately. Results: The proposed multimodal classification model demonstrated superior performance compared to existing methods, achieving an Accuracy of 97.53%, Precision of 98.20%, F1 Score of 97.76%, and Recall of 97.32%. A confusion matrix was also used to evaluate the model's effectiveness. Conclusions: The paper presents a robust multimodal classification approach that outperforms state-of-the-art methods with potential application in clinical diagnostics for depression assessment.

neurosciences
The Verbal and Non Verbal Signals of Depression -- Combining Acoustics, Text and Visuals for Estimating Depression Level

Syed Arbaaz Qureshi,Mohammed Hasanuzzaman,Sriparna Saha,Gaël Dias

DOI: https://doi.org/10.48550/arXiv.1904.07656

2019-04-03

Abstract:Depression is a serious medical condition that is suffered by a large number of people around the world. It significantly affects the way one feels, causing a persistent lowering of mood. In this paper, we propose a novel attention-based deep neural network which facilitates the fusion of various modalities. We use this network to regress the depression level. Acoustic, text and visual modalities have been used to train our proposed network. Various experiments have been carried out on the benchmark dataset, namely, Distress Analysis Interview Corpus - a Wizard of Oz (DAIC-WOZ). From the results, we empirically justify that the fusion of all three modalities helps in giving the most accurate estimation of depression level. Our proposed approach outperforms the state-of-the-art by 7.17% on root mean squared error (RMSE) and 8.08% on mean absolute error (MAE).

Computers and Society,Artificial Intelligence,Machine Learning
Automated depression analysis using convolutional neural networks from speech

Lang He,Cui Cao

DOI: https://doi.org/10.1016/j.jbi.2018.05.007

IF: 8

2018-07-01

Journal of Biomedical Informatics

Abstract:To help clinicians to efficiently diagnose the severity of a person's depression, the affective computing community and the artificial intelligence field have shown a growing interest in designing automated systems. The speech features have useful information for the diagnosis of depression. However, manually designing and domain knowledge are still important for the selection of the feature, which makes the process labor consuming and subjective. In recent years, deep-learned features based on neural networks have shown superior performance to hand-crafted features in various areas. In this paper, to overcome the difficulties mentioned above, we propose a combination of hand-crafted and deep-learned features which can effectively measure the severity of depression from speech. In the proposed method, Deep Convolutional Neural Networks (DCNN) are firstly built to learn deep-learned features from spectrograms and raw speech waveforms. Then we manually extract the state-of-the-art texture descriptors named median robust extended local binary patterns (MRELBP) from spectrograms. To capture the complementary information within the hand-crafted features and deep-learned features, we propose joint fine-tuning layers to combine the raw and spectrogram DCNN to boost the depression recognition performance. Moreover, to address the problems with small samples, a data augmentation method was proposed. Experiments conducted on AVEC2013 and AVEC2014 depression databases show that our approach is robust and effective for the diagnosis of depression when compared to state-of-the-art audio-based methods.

medical informatics,computer science, interdisciplinary applications
Multimodal Physiological Signals Representation Learning via Multiscale Contrasting for Depression Recognition

Kai Shao,Rui Wang,Yixue Hao,Long Hu,Min Chen,Hans Arno Jacobsen

2024-06-26

Abstract:Depression recognition based on physiological signals such as functional near-infrared spectroscopy (fNIRS) and electroencephalogram (EEG) has made considerable progress. However, most existing studies ignore the complementarity and semantic consistency of multimodal physiological signals under the same stimulation task in complex spatio-temporal patterns. In this paper, we introduce a multimodal physiological signals representation learning framework using Siamese architecture via multiscale contrasting for depression recognition (MRLMC). First, fNIRS and EEG are transformed into different but correlated data based on a time-domain data augmentation strategy. Then, we design a spatio-temporal contrasting module to learn the representation of fNIRS and EEG through weight-sharing multiscale spatio-temporal convolution. Furthermore, to enhance the learning of semantic representation associated with stimulation tasks, a semantic consistency contrast module is proposed, aiming to maximize the semantic similarity of fNIRS and EEG. Extensive experiments on publicly available and self-collected multimodal physiological signals datasets indicate that MRLMC outperforms the state-of-the-art models. Moreover, our proposed framework is capable of transferring to multimodal time series downstream tasks.

Machine Learning,Artificial Intelligence
Depression Status Estimation by Deep Learning based Hybrid Multi-Modal Fusion Model

Hrithwik Shalu,Harikrishnan P,Hari Sankar CN,Akash Das,Saptarshi Majumder,Arnhav Datar,Subin Mathew MS,Anugyan Das,Juned Kadiwala

DOI: https://doi.org/10.48550/arXiv.2011.14966

IF: 5.414

2020-11-30

Machine Learning

Abstract:Preliminary detection of mild depression could immensely help in effective treatment of the common mental health disorder. Due to the lack of proper awareness and the ample mix of stigmas and misconceptions present within the society, mental health status estimation has become a truly difficult task. Due to the immense variations in character level traits from person to person, traditional deep learning methods fail to generalize in a real world setting. In our study we aim to create a human allied AI workflow which could efficiently adapt to specific users and effectively perform in real world scenarios. We propose a Hybrid deep learning approach that combines the essence of one shot learning, classical supervised deep learning methods and human allied interactions for adaptation. In order to capture maximum information and make efficient diagnosis video, audio, and text modalities are utilized. Our Hybrid Fusion model achieved a high accuracy of 96.3% on the Dataset; and attained an AUC of 0.9682 which proves its robustness in discriminating classes in complex real-world scenarios making sure that no cases of mild depression are missed during diagnosis. The proposed method is deployed in a cloud-based smartphone application for robust testing. With user-specific adaptations and state of the art methodologies, we present a state-of-the-art model with user friendly experience.
DepMamba: Progressive Fusion Mamba for Multimodal Depression Detection

Jiaxin Ye,Junping Zhang,Hongming Shan

2024-09-24

Abstract:Depression is a common mental disorder that affects millions of people worldwide. Although promising, current multimodal methods hinge on aligned or aggregated multimodal fusion, suffering two significant limitations: (i) inefficient long-range temporal modeling, and (ii) sub-optimal multimodal fusion between intermodal fusion and intramodal processing. In this paper, we propose an audio-visual progressive fusion Mamba for multimodal depression detection, termed DepMamba. DepMamba features two core designs: hierarchical contextual modeling and progressive multimodal fusion. On the one hand, hierarchical modeling introduces convolution neural networks and Mamba to extract the local-to-global features within long-range sequences. On the other hand, the progressive fusion first presents a multimodal collaborative State Space Model (SSM) extracting intermodal and intramodal information for each modality, and then utilizes a multimodal enhanced SSM for modality cohesion. Extensive experimental results on two large-scale depression datasets demonstrate the superior performance of our DepMamba over existing state-of-the-art methods. Code is available at <a class="link-external link-https" href="https://github.com/Jiaxin-Ye/DepMamba" rel="external noopener nofollow">this https URL</a>.

Computers and Society,Computer Vision and Pattern Recognition
Multimodal Sensing for Depression Risk Detection: Integrating Audio, Video, and Text Data

Zhang,Zhang,Ni,Wei,Yang,Jin,Huang,Liang,Zhang,Li,Ding,Zhang,Wang

DOI: https://doi.org/10.3390/s24123714

IF: 3.9

2024-06-08

Sensors

Abstract:Depression is a major psychological disorder with a growing impact worldwide. Traditional methods for detecting the risk of depression, predominantly reliant on psychiatric evaluations and self-assessment questionnaires, are often criticized for their inefficiency and lack of objectivity. Advancements in deep learning have paved the way for innovations in depression risk detection methods that fuse multimodal data. This paper introduces a novel framework, the Audio, Video, and Text Fusion-Three Branch Network (AVTF-TBN), designed to amalgamate auditory, visual, and textual cues for a comprehensive analysis of depression risk. Our approach encompasses three dedicated branches—Audio Branch, Video Branch, and Text Branch—each responsible for extracting salient features from the corresponding modality. These features are subsequently fused through a multimodal fusion (MMF) module, yielding a robust feature vector that feeds into a predictive modeling layer. To further our research, we devised an emotion elicitation paradigm based on two distinct tasks—reading and interviewing—implemented to gather a rich, sensor-based depression risk detection dataset. The sensory equipment, such as cameras, captures subtle facial expressions and vocal characteristics essential for our analysis. The research thoroughly investigates the data generated by varying emotional stimuli and evaluates the contribution of different tasks to emotion evocation. During the experiment, the AVTF-TBN model has the best performance when the data from the two tasks are simultaneously used for detection, where the F1 Score is 0.78, Precision is 0.76, and Recall is 0.81. Our experimental results confirm the validity of the paradigm and demonstrate the efficacy of the AVTF-TBN model in detecting depression risk, showcasing the crucial role of sensor-based data in mental health detection.

engineering, electrical & electronic,instruments & instrumentation,chemistry, analytical
Multimodal temporal machine learning for Bipolar Disorder and Depression Recognition

Francesco Ceccarelli,Marwa Mahmoud

DOI: https://doi.org/10.1007/s10044-021-01001-y

IF: 2.307

2021-06-18

Pattern Analysis and Applications

Abstract:Mental disorder is a serious public health concern that affects the life of millions of people throughout the world. Early diagnosis is essential to ensure timely treatment and to improve the well-being of those affected by a mental disorder. In this paper, we present a novel multimodal framework to perform mental disorder recognition from videos. The proposed approach employs a combination of audio, video and textual modalities. Using recurrent neural network architectures, we incorporate the temporal information in the learning process and model the dynamic evolution of the features extracted for each patient. For multimodal fusion, we propose an efficient late fusion strategy based on a simple feed-forward neural network that we call <i>adaptive nonlinear judge classifier</i>. We evaluate the proposed framework on two mental disorder datasets. On both, the experimental results demonstrate that the proposed framework outperforms the state-of-the-art approaches. We also study the importance of each modality for mental disorder recognition and infer interesting conclusions about the temporal nature of each modality. Our findings demonstrate that careful consideration of the temporal evolution of each modality is of crucial importance to accurately perform mental disorder recognition.

computer science, artificial intelligence
Transformer-based multimodal feature enhancement networks for multimodal depression detection integrating video, audio and remote photoplethysmograph signals

Huiting Fan,Xingnan Zhang,Yingying Xu,Jiangxiong Fang,Shiqing Zhang,Xiaoming Zhao,Jun Yu

DOI: https://doi.org/10.1016/j.inffus.2023.102161

IF: 18.6

2023-12-02

Information Fusion

Abstract:Depression stands as one of the most widespread psychological disorders and has garnered increasing attention. Currently, how to effectively achieve automatic multimodal depression detection for assisting doctors in early diagnosis of depression, has become an important and challenging issue. To address this issue, this work proposes Transformer-based feature enhancement networks for multimodal depression detection. The proposed method effectively integrates three modalities including video, audio and remote photoplethysmographic (rPPG) signals for multimodal depression detection, in which the rPPG modality is introduced as an additional modality for enhancing the effectiveness of multimodal depression detection. The proposed method consists of three key steps: multimodal feature extraction for video, audio and rPPG modalities, Transformer-based multimodal feature enhancement (TMFE), and graph fusion networks (GFN) based multimodal fusion and depression prediction. More specially, in the stage of multimodal feature extraction, for video and audio modalities we employ deep convolutional neural networks (CNN) to extract the corresponding high-level video and audio features, respectively. For rPPG modality, we adopt a short-time end-to-end rPPG estimation framework to extract the rPPG signal values.The TMFE module stacks multiple Transformers such as the inter-modal, intra-modal, and tri-modal Transformers to jointly capture the dynamics and relationships within and between modalities for each time-step of input sequences. The GFN module is designed to effectively fuse the obtained feature representations from different modalities while maintaining the interactions between them simultaneously. Finally, the obtained shared feature representations of all modalities are fed into a multilayer perceptrons (MLP) network to implement final depression detection tasks. Extensive experiments are conducted on two public datasets such as AVEC2013 and AVEC2014, and experimental results demonstrate the validity of the proposed method on depression detection tasks.

computer science, artificial intelligence, theory & methods
Multi-level Attention network using text, audio and video for Depression Prediction

Anupama Ray,Siddharth Kumar,Rutvik Reddy,Prerana Mukherjee,Ritu Garg

DOI: https://doi.org/10.48550/arXiv.1909.01417

2019-09-03

Computer Vision and Pattern Recognition

Abstract:Depression has been the leading cause of mental-health illness worldwide. Major depressive disorder (MDD), is a common mental health disorder that affects both psychologically as well as physically which could lead to loss of lives. Due to the lack of diagnostic tests and subjectivity involved in detecting depression, there is a growing interest in using behavioural cues to automate depression diagnosis and stage prediction. The absence of labelled behavioural datasets for such problems and the huge amount of variations possible in behaviour makes the problem more challenging. This paper presents a novel multi-level attention based network for multi-modal depression prediction that fuses features from audio, video and text modalities while learning the intra and inter modality relevance. The multi-level attention reinforces overall learning by selecting the most influential features within each modality for the decision making. We perform exhaustive experimentation to create different regression models for audio, video and text modalities. Several fusions models with different configurations are constructed to understand the impact of each feature and modality. We outperform the current baseline by 17.52% in terms of root mean squared error.
Prediction of Depression Severity Based on the Prosodic and Semantic Features with Bidirectional LSTM and Time Distributed CNN

Kaining Mao,Wei Zhang,Deborah Baofeng Wang,Ang Li,Rongqi Jiao,Yanhui Zhu,Bin Wu,Tiansheng Zheng,Lei Qian,Wei Lyu,Minjie Ye,Jie Chen

DOI: https://doi.org/10.48550/arXiv.2202.12456

IF: 6.4588

2022-02-25

Human-Computer Interaction

Abstract:Depression is increasingly impacting individuals both physically and psychologically worldwide. It has become a global major public health problem and attracts attention from various research fields. Traditionally, the diagnosis of depression is formulated through semi-structured interviews and supplementary questionnaires, which makes the diagnosis heavily relying on physicians experience and is subject to bias. Mental health monitoring and cloud-based remote diagnosis can be implemented through an automated depression diagnosis system. In this article, we propose an attention-based multimodality speech and text representation for depression prediction. Our model is trained to estimate the depression severity of participants using the Distress Analysis Interview Corpus-Wizard of Oz (DAIC-WOZ) dataset. For the audio modality, we use the collaborative voice analysis repository (COVAREP) features provided by the dataset and employ a Bidirectional Long Short-Term Memory Network (Bi-LSTM) followed by a Time-distributed Convolutional Neural Network (T-CNN). For the text modality, we use global vectors for word representation (GloVe) to perform word embeddings and the embeddings are fed into the Bi-LSTM network. Results show that both audio and text models perform well on the depression severity estimation task, with best sequence level F1 score of 0.9870 and patient-level F1 score of 0.9074 for the audio model over five classes (healthy, mild, moderate, moderately severe, and severe), as well as sequence level F1 score of 0.9709 and patient-level F1 score of 0.9245 for the text model over five classes. Results are similar for the multimodality fused model, with the highest F1 score of 0.9580 on the patient-level depression detection task over five classes. Experiments show statistically significant improvements over previous works.
Major Depressive Disorder Classification Based on Different Convolutional Neural Network Models: Deep Learning Approach

Caglar Uyulan,Türker Tekin Ergüzel,Huseyin Unubol,Merve Cebi,Gokben Hizli Sayar,Nevzat Tarhan,Mahdi Nezhad Asad

DOI: https://doi.org/10.1177/1550059420916634

IF: 2.046

2020-06-03

Clinical EEG and Neuroscience

Abstract:The human brain is characterized by complex structural, functional connections that integrate unique cognitive characteristics. There is a fundamental hurdle for the evaluation of both structural and functional connections of the brain and the effects in the diagnosis and treatment of neurodegenerative diseases. Currently, there is no clinically specific diagnostic biomarker capable of confirming the diagnosis of major depressive disorder (MDD). Therefore, exploring translational biomarkers of mood disorders based on deep learning (DL) has valuable potential with its recently underlined promising outcomes. In this article, an electroencephalography (EEG)-based diagnosis model for MDD is built through advanced computational neuroscience methodology coupled with a deep convolutional neural network (CNN) approach. EEG recordings are analyzed by modeling 3 different deep CNN structure, namely, ResNet-50, MobileNet, Inception-v3, in order to dichotomize MDD patients and healthy controls. EEG data are collected for 4 main frequency bands (Δ, θ, α, and β, accompanying spatial resolution with location information by collecting data from 19 electrodes. Following the pre-processing step, different DL architectures were employed to underline discrimination performance by comparing classification accuracies. The classification performance of models based on location data, MobileNet architecture generated 89.33% and 92.66% classification accuracy. As to the frequency bands, delta frequency band outperformed compared to other bands with 90.22% predictive accuracy and area under curve (AUC) value of 0.9 for ResNet-50 architecture. The main contribution of the study is the delineation of distinctive spatial and temporal features using various DL architectures to dichotomize 46 MDD subjects from 46 healthy subjects. Exploring translational biomarkers of mood disorders based on DL perspective is the main focus of this study and, though it is challenging, with its promising potential to improve our understanding of the psychiatric disorders, computational methods are highly worthy for the diagnosis process and valuable in terms of both speed and accuracy compared with classical approaches.

psychology,clinical neurology,psychiatry,neurosciences,neuroimaging

[Clinical aspects and prognosis of Legionella infections].

Automatic Assessment of Depression from Speech Via a Hierarchical Attention Transfer Network and Attention Autoencoders

An Intra- and Inter-Emotion Transformer-Based Fusion Model with Homogeneous and Diverse Constraints Using Multi-Emotional Audiovisual Features for Depression Detection.

Hierarchical Attention Transfer Networks for Depression Assessment from Speech

MFCC-based Recurrent Neural Network for automatic clinical depression recognition and assessment from speech

Multimodal Measurement of Depression Using Deep Learning Models

End-to-end multimodal system for depression detection from online recordings

AudVowelConsNet: A phoneme-level based deep CNN architecture for clinical depression diagnosis

Design of polydiacetylene-phospholipid supramolecules for enhanced stability and sensitivity.

Multimodal Fusion of EEG and Audio Spectrogram for Major Depressive Disorder Recognition Using Modified DenseNet121

The Verbal and Non Verbal Signals of Depression -- Combining Acoustics, Text and Visuals for Estimating Depression Level

Automated depression analysis using convolutional neural networks from speech

Multimodal Physiological Signals Representation Learning via Multiscale Contrasting for Depression Recognition

Depression Status Estimation by Deep Learning based Hybrid Multi-Modal Fusion Model

DepMamba: Progressive Fusion Mamba for Multimodal Depression Detection

Multimodal Sensing for Depression Risk Detection: Integrating Audio, Video, and Text Data

Multimodal temporal machine learning for Bipolar Disorder and Depression Recognition

Transformer-based multimodal feature enhancement networks for multimodal depression detection integrating video, audio and remote photoplethysmograph signals

Multi-level Attention network using text, audio and video for Depression Prediction

Prediction of Depression Severity Based on the Prosodic and Semantic Features with Bidirectional LSTM and Time Distributed CNN

Major Depressive Disorder Classification Based on Different Convolutional Neural Network Models: Deep Learning Approach