Developing Clinically Relevant Decision Tree Models Using Machine-Learning to Predict Psychotherapy Success in Late Life Depression

Jason Gandelman,Scott Mackin,Duygu Tosun-Turgut,Craig Nelson
DOI: https://doi.org/10.1016/j.jagp.2024.01.217
IF: 7.996
2024-02-21
American Journal of Geriatric Psychiatry
Abstract:Introduction Late Life Depression (LLD) is characterized by low remission rates using currently available treatments. Therefore, it is crucial for clinicians to have personalized tools to better identify older adults who may have a greater chance of success when deciding on a given treatment. In particular, psychotherapy is an important and under-utilized treatment option for older adults who may not tolerate medications or have drug interactions that limit pharmacotherapy. In this study, we aimed to use machine-learning methods to identify pre-treatment characteristics that are predictive of psychotherapy treatment success (i.e. Depression Remission after psychotherapy treatment). Methods We analyzed data from a previously described 12-week trial of weekly Problem Solving Therapy (PST) in 96 participants with Major Depressive Disorder who were 65 years or older (Kassel et al. 2022). Using machine-learning methods, we examined 20 pre-treatment characteristics of interest across four categories: 1) Demographic Factors, 2) Depression History, 3) Depression Symptom Characteristics, and 4) Cognitive Test Scores. The Gradient Boosted Method (GBM) was used to determine the relative importance of each characteristic as a possible predictor, which guided development of specific, short (depth=3) Decision Tree models to predict Depression Remission (17-item Hamilton Depression Score < 7). Given sample size limitations, these models were trained on the same sample to which they were applied for prediction. Results Three Decision Tree models were significant at the p<0.001 threshold: Age and Depression History, pre-treatment Episodic Memory Function, and All 20 Pre-treatment Characteristics Available (the resulting optimal model only utilized 5 of the 20 characteristics across the 4 categories of available pre-treatment characteristics). Amongst the three significant models: Classification Accuracy was 73.6-77.4% and AUC was 0.763-0.789. These 3 models offered high Specificity (82.8-95.1%) and Negative Predictive Value (75.3-82.8%) and relatively lower Sensitivity (34.5-63.3%) and Positive Predictive Value (61.5-76.9%). Age and Depression History (See Figure 1) was the simplest of the three significant models with only five Leaves (i.e. Terminal Nodes). In contrast, models derived from the pre-treatment Cognitive Domains of Executive Function, Processing Speed, and Language Function were each not significantly predictive of Depression Remission at the p<0.001 threshold. Models derived from pre-treatment Depression Symptom Characteristics were also not significantly predictive at this threshold. Conclusions We have derived an Age and Depression History Decision Tree model (Figure 1) that would only require assessment of 3 simple pre-treatment characteristics to improve prediction of psychotherapy success: correctly classifying about three quarters of the participants in our sample. The pre-treatment Episodic Memory Function Decision Tree model had a similar classification rate but does require cognitive testing. Both models were superior to models derived only from other Cognitive Domains, including models of Executive Function. However, validation of these models is needed and will require testing their predictive value in larger, more representative samples. These results demonstrate a novel machine-learning based prediction strategy that is simple for clinicians to use and thus may hold promise in personalizing difficult treatment decisions for LLD.
psychiatry,geriatrics & gerontology,gerontology
What problem does this paper attempt to address?