Abstract:Patients suffering from chronic diseases often receive interventions after they are discharged from hospitals for their health management. The intervention decisions can be made sequentially during the course of disease progression and treatment process, and timely and proper interventions can lead to better health condition and quality of life and reduce the rate of re-hospitalization. Moreover, to determine effective intervention plans, personalized care should be considered. However, in many cases, intervention plans are not tailored to patients’ health conditions or dynamically adjusted based on their status changes due to the lack of adequate analytical methods. To address this issue, we introduce a causal Bayesian network-based Markov decision process (CNMDP) model to dynamically evaluate the health status and effectiveness of interventions for each patient and update the corresponding intervention strategy. In such a model, a causal Bayesian network is integrated in order to predict the effect of interventions in each patient’s health status. The resulting risk at each decision point is considered in a Markov decision process model by examining diverse factors of a patient to make a personalized optimal decision on intervention planning to minimize the risk of health deterioration or readmission. To illustrate the applicability of the model and develop a specific formulation, an intervention decision model for chronic obstructive pulmonary disease (COPD) patients is presented using the proposed method. Such a model can be applied to other diseases as well to design personalized, prompt, and effective intervention plans. Note to Practitioners—Substantial efforts have been devoted to treatment and care management of chronic diseases. Particularly, there has been an initiative to design appropriate follow-up plans and interventions for postdischarge care to reduce hospital readmissions. However, many intervention plans are not patient specific and do not change with patient conditions. To mitigate such an limitation, based on identified risk factors and causal relationships, a causal Bayesian network model is introduced to predict readmission risk and evaluate intervention effectiveness. The results are then embedded into a Markov decision process to derive optimal intervention plan to minimize readmission risk. Such a process is repeated dynamically to evaluate health status and update intervention strategies. Using chronic obstructive pulmonary disease (COPD) as an example of chronic disease, a case study at a community hospital is presented to illustrate the applicability of the model.

Use of a Markov Decision Process Model for Treatment Selection in an Asymptomatic Disease with Consideration of Risk Sensitivity

Decision-Making for Preventive Interventions in Asymptomatic Patients

Finding Optimal Cancer Treatment using Markov Decision Process to Improve Overall Health and Quality of Life

Markov models for clinical decision‐making in radiation oncology: A systematic review

A Markov decision process approach to optimizing cancer therapy using multiple modalities

Markov Decision Process Modeling for Multi-stage Optimization of Intervention and Treatment Strategies in Breast Cancer

Dynamic treatment selection and modification for personalised blood pressure therapy using a Markov decision process model: a cost-effectiveness analysis

Personalized Dynamic Treatment Regimes in Continuous Time: A Bayesian Approach for Optimizing Clinical Decisions with Timing

Artificial Intelligence Framework for Simulating Clinical Decision-Making: A Markov Decision Process Approach

A Causal Network-Based Markov Decision Process Model for Intervention Planning

A Promising Approach to Optimizing Sequential Treatment Decisions for Depression: Markov Decision Process

Decision Curve Analysis: a Technical Note

Cost-Effectiveness of Personalized Policies for Implementing Organ-at-Risk Sparing Adaptive Radiation Therapy in Head and Neck Cancer: A Markov Decision Process Approach

Decision curve analysis for personalized treatment choice between multiple options

Deciding when to intervene: a Markov decision process approach

Cost-effectiveness analysis of treatments for chronic disease: using R to incorporate time dependency of treatment response

Integrating decision modeling and machine learning to inform treatment stratification

Data-Driven Allocation of Preventive Care with Application to Diabetes Mellitus Type II

Monte Carlo decision curve analysis using aggregate data

Learning Cost-Effective Treatment Regimes using Markov Decision Processes

Probabilistic Prediction for Binary Treatment Choice: with focus on personalized medicine