Abstract:Background: The ability to automatically detect anxiety disorders from speech could be useful as a screening tool for an anxiety disorder. Prior studies have shown that individual words in textual transcripts of speech have an association with anxiety severity. Transformer-based neural networks are models that have been recently shown to have powerful predictive capabilities based on the context of more than one input word. Transformers detect linguistic patterns and can be separately trained to make specific predictions based on these patterns. Objective: This study aimed to determine whether a transformer-based language model can be used to screen for generalized anxiety disorder from impromptu speech transcripts. Methods: A total of 2000 participants provided an impromptu speech sample in response to a modified version of the Trier Social Stress Test (TSST). They also completed the Generalized Anxiety Disorder 7-item (GAD-7) scale. A transformer-based neural network model (pretrained on large textual corpora) was fine-tuned on the speech transcripts and the GAD-7 to predict whether a participant was above or below a screening threshold of the GAD-7. We reported the area under the receiver operating characteristic curve (AUROC) on the test data and compared the results with a baseline logistic regression model using the Linguistic Inquiry and Word Count (LIWC) features as input. Using the integrated gradient method to determine specific words that strongly affect the predictions, we inferred specific linguistic patterns that influence the predictions. Results: The baseline LIWC-based logistic regression model had an AUROC value of 0.58. The fine-tuned transformer model achieved an AUROC value of 0.64. Specific words that were often implicated in the predictions were also dependent on the context. For example, the first-person singular pronoun "I" influenced toward an anxious prediction 88% of the time and a nonanxious prediction 12% of the time, depending on the context. Silent pauses in speech, also often implicated in predictions, influenced toward an anxious prediction 20% of the time and a nonanxious prediction 80% of the time. Conclusions: There is evidence that a transformer-based neural network model has increased predictive power compared with the single word-based LIWC model. We also showed that the use of specific words in a specific context-a linguistic pattern-is part of the reason for the better prediction. This suggests that such transformer-based models could play a useful role in anxiety screening systems.

Natural language analyzed with AI-based transformers predict traditional subjective well-being measures approaching the theoretical upper limits in accuracy

Question-based computational language approach outperforms rating scales in quantifying emotional states

Beyond rating scales: With targeted evaluation, large language models are poised for psychological assessment

The AI‐IP: Minimizing the guesswork of personality scale item development through artificial intelligence

AI-Enhanced Mental Health Diagnosis: Leveraging Transformers for Early Detection of Depression Tendency in Textual Data

Field-testing items using artificial intelligence: Natural language processing with transformers

Evaluating the Efficacy of AI-Based Interactive Assessments Using Large Language Models for Depression Screening

Personal Resilience Can Be Well Estimated from Heart Rate Variability and Paralinguistic Features during Human-Robot Conversations

Assessment and manipulation of latent constructs in pre-trained language models using psychometric scales

Predicting Generalized Anxiety Disorder From Impromptu Speech Transcripts Using Context-Aware Transformer-Based Neural Networks: Model Evaluation Study

Predicting life satisfaction using machine learning and explainable AI

PREDICTING INDIVIDUAL WELL-BEING THROUGH THE LANGUAGE OF SOCIAL MEDIA

A step-by-step Researcher's Guide to the use of an AI-based transformer in epidemiology: an exploratory analysis of ChatGPT using the STROBE checklist for observational studies

Conversational assessment using artificial intelligence is as clinically useful as depression scales and preferred by users

Field-Testing Multiple-Choice Questions With AI Examinees: English Grammar Items

Transformers in health: a systematic review on architectures for longitudinal data analysis

Asynchronously Embedding Psychological Test Questions into Human–Robot Conversations for User Profiling

Measuring the menu, not the food: "psychometric" data may instead measure "lingometrics" (and miss its greatest potential)

Pushing on Personality Detection from Verbal Behavior: A Transformer Meets Text Contours of Psycholinguistic Features

ALBA: Adaptive Language-based Assessments for Mental Health

How are We Doing Today? Using Natural Speech Analysis to Assess Older Adults' Subjective Well-Being