Abstract:Background: While speech analysis holds promise for mental health assessment, research often focuses on single symptoms, despite symptom co-occurrences and interactions. In addition, predictive models in mental health do not properly assess speech-based systems' limitations, such as uncertainty, or fairness for a safe clinical deployment. Objective: We investigated the predictive potential of mobile-collected speech data for detecting and estimating depression, anxiety, fatigue, and insomnia, focusing on other factors than mere accuracy, in the general population. Methods: We included n=865 healthy adults and recorded their answers regarding their perceived mental and sleep states. We asked how they felt and if they had slept well lately. Clinically validated questionnaires measuring depression, anxiety, insomnia, and fatigue severity were also used. We developed a novel speech and machine learning pipeline involving voice activity detection, feature extraction, and model training. We automatically analyzed participants' speech with a fully ML automatic pipeline to capture speech variability. Then, we modelled speech with pretrained deep learning models that were pre-trained on a large open free database and we selected the best one on the validation set. Based on the best speech modelling approach, we evaluated clinical threshold detection, individual score prediction, model uncertainty estimation, and performance fairness across demographics (age, sex, education). We employed a train-validation-test split for all evaluations: to develop our models, select the best ones and assess the generalizability of held-out data. Results: The best model was WhisperM with a max pooling, and oversampling method. Our methods achieved good detection performance for all symptoms, depression (PHQ-9 AUC= 0.76F1=0.49, BDI AUC=0.78, F1=0,65), anxiety (GAD-7 F1=0.50, AUC=0.77) insomnia (AIS AUC=0.73, F1=0.62), and fatigue (MFI Total Score F1=0.88, AUC=0.68). These strengths were maintained for depression detection with BDI and Fatigue for abstention rates for uncertain cases (Risk-Coverage AUCs < 0.4). Individual symptom scores were predicted with good accuracy (Correlations were all significant, with Pearson strengths between 0.31 and 0.49). Fairness analysis revealed that models were consistent for sex (average Disparity Ratio (DR) = 0.86), to a lesser extent for education level (average Disparity Ratio (DR) = 0.47) and worse for age groups (average Disparity Ratio (DR) = 0.33). Conclusions: This study demonstrates the potential of speech-based systems for multifaceted mental health assessment in the general population, not only for detecting clinical thresholds but also for estimating their severity. Addressing fairness and incorporating uncertainty estimation with selective classification are key contributions that can enhance the clinical utility and responsible implementation of such systems. This approach offers promise for more accurate and nuanced mental health assessments, benefiting both patients and clinicians.

Automatic Depression Detection of Mobile-Based Text-dependent Speech Signals Using a Deep CNN Approach: A Prospective Cohort Study (Preprint)

Automatic Depression Detection Using Smartphone-Based Text-Dependent Speech Signals: Deep Convolutional Neural Network Approach

Automatic Assessment of Depression from Speech Via a Hierarchical Attention Transfer Network and Attention Autoencoders

Hybrid Network Feature Extraction for Depression Assessment from Speech

Automated speech analysis for risk detection of depression, anxiety, insomnia, and fatigue: Algorithm Development and Validation Study

A deep learning-based model for detecting depression in senior population

MFCC-based Recurrent Neural Network for automatic clinical depression recognition and assessment from speech

Detection of Minor and Major Depression through Voice as a Biomarker Using Machine Learning

Automated speech-based screening of depression using deep convolutional neural networks

End-to-end multimodal system for depression detection from online recordings

Automated depression analysis using convolutional neural networks from speech

Depression detection using cascaded attention based deep learning framework using speech data

Multi-modal deep learning system for depression and anxiety detection

Automatic Detection of Depression from Stratified Samples of Audio Data

Validation of Machine Learning-Based Assessment of Major Depressive Disorder from Paralinguistic Speech Characteristics in Routine Care

Identifying depression-related topics in smartphone-collected free-response speech recordings using an automatic speech recognition system and a deep learning topic model

Automated voice biomarkers for depression symptoms using an online cross‐sectional data collection initiative

Development of depression detection algorithm using text scripts of routine psychiatric interview

Automatic Detection of Depression in Speech Using Ensemble Convolutional Neural Networks

Speech-based Clinical Depression Screening: An Empirical Study

Automatic recognition of depression based on audio and video: A review