Personalized Multimorbidity Management for Patients with Type 2 Diabetes Using Reinforcement Learning of Electronic Health Records

Hua Zheng,Ilya O. Ryzhov,Wei Xie,Judy Zhong
DOI: https://doi.org/10.48550/arXiv.2011.02287
2020-10-29
Abstract:Comorbid chronic conditions are common among people with type 2 diabetes. We developed an Artificial Intelligence algorithm, based on Reinforcement Learning (RL), for personalized diabetes and multi-morbidity management with strong potential to improve health outcomes relative to current clinical practice. In this paper, we modeled glycemia, blood pressure and cardiovascular disease (CVD) risk as health outcomes using a retrospective cohort of 16,665 patients with type 2 diabetes from New York University Langone Health ambulatory care electronic health records in 2009 to 2017. We trained a RL prescription algorithm that recommends a treatment regimen optimizing patients' cumulative health outcomes using their individual characteristics and medical history at each encounter. The RL recommendations were evaluated on an independent subset of patients. The results demonstrate that the proposed personalized reinforcement learning prescriptive framework for type 2 diabetes yielded high concordance with clinicians' prescriptions and substantial improvements in glycemia, blood pressure, cardiovascular disease risk outcomes.
Computers and Society,Machine Learning
What problem does this paper attempt to address?
The problem this paper attempts to address is: how to use Reinforcement Learning (RL) algorithms, based on Electronic Health Records (EHR), to provide personalized comorbidity management plans for patients with type 2 diabetes, in order to improve their health outcomes such as blood glucose, blood pressure, and cardiovascular disease risk. Specifically, the paper focuses on the following aspects: 1. **Comorbidity Management**: Patients with type 2 diabetes often have other chronic conditions such as hypertension and cardiovascular disease. These comorbidities make treatment more complex, and existing clinical guidelines usually target single diseases, failing to effectively address comorbid conditions. 2. **Personalized Treatment Recommendations**: Traditional treatment plans are often based on population data and lack consideration of individual differences. The paper proposes a reinforcement learning-based algorithm that can dynamically recommend the optimal treatment plan based on each patient's individual characteristics and medical history. 3. **Improving Health Outcomes**: By optimizing indicators such as blood glucose, blood pressure, and cardiovascular disease risk, the goal is to improve patients' health status and reduce the incidence of uncontrolled blood glucose, hypertension, and high cardiovascular disease risk. The paper conducts a retrospective cohort study using electronic health record data from the NYU Langone Health Center to train and evaluate multiple reinforcement learning algorithms, including those that optimize blood glucose, blood pressure, and cardiovascular disease risk individually, as well as a comorbidity management algorithm that jointly optimizes these three indicators. The study results indicate that these algorithms are consistent with clinical prescriptions in most cases and significantly improve patient health outcomes in cases of inconsistency.