Development and Validation of a Deep-Learning Model for Differential Treatment Benefit Prediction for Adults with Major Depressive Disorder Deployed in the Artificial Intelligence in Depression Medication Enhancement (AIDME) Study

David Benrimoh,Caitrin Armstrong,Joseph Mehltretter,Robert Fratila,Kelly Perlman,Sonia Israel,Adam Kapelner,Sagar V. Parikh,Jordan F. Karp,Katherine Heller,Gustavo Turecki

2024-06-07

Abstract:INTRODUCTION: The pharmacological treatment of Major Depressive Disorder (MDD) relies on a trial-and-error approach. We introduce an artificial intelligence (AI) model aiming to personalize treatment and improve outcomes, which was deployed in the Artificial Intelligence in Depression Medication Enhancement (AIDME) Study. OBJECTIVES: 1) Develop a model capable of predicting probabilities of remission across multiple pharmacological treatments for adults with at least moderate major depression. 2) Validate model predictions and examine them for amplification of harmful biases. METHODS: Data from previous clinical trials of antidepressant medications were standardized into a common framework and included 9,042 adults with moderate to severe major depression. Feature selection retained 25 clinical and demographic variables. Using Bayesian optimization, a deep learning model was trained on the training set, refined using the validation set, and tested once on the held-out test set. RESULTS: In the evaluation on the held-out test set, the model demonstrated achieved an AUC of 0.65. The model outperformed a null model on the test set (p = 0.01). The model demonstrated clinical utility, achieving an absolute improvement in population remission rate in hypothetical and actual improvement testing. While the model did identify one drug (escitalopram) as generally outperforming the other drugs (consistent with the input data), there was otherwise significant variation in drug rankings. On bias testing, the model did not amplify potentially harmful biases. CONCLUSIONS: We demonstrate the first model capable of predicting outcomes for 10 different treatment options for patients with MDD, intended to be used at or near the start of treatment to personalize treatment. The model was put into clinical practice during the AIDME randomized controlled trial whose results are reported separately.

Neurons and Cognition,Machine Learning

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

This paper aims to address the issue of personalized selection in the pharmacological treatment of Major Depressive Disorder (MDD). Currently, the drug treatment for MDD mainly relies on a trial-and-error approach, which not only prolongs the patient's course of illness but also leads to poor treatment outcomes. To solve this problem, the research team developed and validated a deep learning-based model to predict the probability of remission in adult patients with moderate to severe MDD under various drug treatment regimens.

### Specific Objectives

1. **Develop the Model**: Construct a model capable of predicting the probability of remission under various drug treatment regimens.
2. **Validate the Model**: Validate the predictive performance of the model and check whether it amplifies harmful biases.

### Background and Motivation

- **Socioeconomic Burden of MDD**: Over 300 million people worldwide are affected by MDD, with an annual socioeconomic burden of $326.2 billion in the United States alone.
- **Limitations of Existing Treatments**: Although there are multiple effective treatments for MDD, only about one-third of patients achieve remission with the first treatment they try. Therefore, many patients have to undergo a "trial-and-error" process, which prolongs the course of illness and leads to poorer treatment outcomes.
- **Need for Personalized Treatment**: To improve treatment outcomes and save time and costs, a tool is needed to help doctors personalize treatment choices in clinical decision-making without requiring complex imaging or genetic testing.

### Methods

- **Data Sources**: Data from multiple clinical trials of antidepressants were obtained from the NIMH, collaborating researchers, and pharmaceutical companies' open science platforms.
- **Data Processing**: The data were standardized and divided into training, validation, and test sets. The deep learning model was trained using Bayesian optimization and finally tested on the test set.
- **Feature Selection**: 25 clinical and demographic variables were selected as input features.
- **Model Evaluation**: The predictive performance of the model was evaluated, including AUC scores, calibration errors, and potential bias checks.

### Results

- **Model Performance**: The AUC score on the test set was 0.65, significantly better than the random model (p=0.01).
- **Clinical Utility**: In hypothetical analyses, the model increased the population remission rate from 43.15% to 53.99%, and from 43.21% to 55.08% in conservative analyses.
- **Drug Preference**: The model most frequently predicted Escitalopram to have the highest remission rate, consistent with input data and previous meta-analysis results.
- **Bias Assessment**: The model did not amplify any potential harmful biases, and sensitivity analysis results were consistent with findings in the literature.

### Conclusion

This study demonstrates the first model capable of predicting the effects of 10 different treatment regimens, aimed at initial treatment selection for MDD patients to better personalize treatment. The model has been deployed in the AID-ME study, and its results will be reported in subsequent reports. This research provides methods for developing and validating AI models applicable to clinical practice, marking an important advancement in the use of predictive tools in mental health clinical practice.