Abstract:Accurate models of clinical actions and their impacts on disease progression are critical for estimating personalized optimal dynamic treatment regimes (DTRs) in medical/health research, especially in managing chronic conditions. Traditional statistical methods for DTRs usually focus on estimating the optimal treatment or dosage at each given medical intervention, but overlook the important question of “when this intervention should happen.” We fill this gap by developing a two-step Bayesian approach to optimize clinical decisions with timing. In the first step, we build a generative model for a sequence of medical interventions—which are discrete events in continuous time—with a marked temporal point process (MTPP) where the mark is the assigned treatment or dosage. Then this clinical action model is embedded into a Bayesian joint framework where the other components model clinical observations including longitudinal medical measurements and time-to-event data conditional on treatment histories. In the second step, we propose a policy gradient method to learn the personalized optimal clinical decision that maximizes the patient survival by interacting the MTPP with the model on clinical observations while accounting for uncertainties in clinical observations learned from the posterior inference of the Bayesian joint model in the first step. A signature application of the proposed approach is to schedule follow-up visitations and assign a dosage at each visitation for patients after kidney transplantation. We evaluate our approach with comparison to alternative methods on both simulated and real-world datasets. In our experiments, the personalized decisions made by the proposed method are clinically useful: they are interpretable and successfully help improve patient survival.

Linear Mixed Models for Comparing Dynamic Treatment Regimens on a Longitudinal Outcome in Sequentially Randomized Trials

Design and analysis considerations for comparing dynamic treatment regimens with binary outcomes from sequential multiple assignment randomized trials

Sample size considerations for comparing dynamic treatment regimens in a sequential multiple-assignment randomized trial with a continuous longitudinal outcome

Comparing cluster-level dynamic treatment regimens using sequential, multiple assignment, randomized trials: Regression estimation and sample size considerations

Sample size estimation for comparing dynamic treatment regimens in a SMART: A Monte Carlo-based approach and case study with longitudinal overdispersed count outcomes

A generalized outcome-adaptive sequential multiple assignment randomized trial design

A novel approach to assess dynamic treatment regimes embedded in a SMART with an ordinal outcome

Efficient and Robust Approaches for Analysis of SMARTs: Illustration using the ADAPT-R Trial

Robust Hybrid Learning for Estimating Personalized Dynamic Treatment Regimens

Incorporating Participants' Welfare into Sequential Multiple Assignment Randomized Trials

Doubly-Robust Dynamic Treatment Regimen Estimation for Binary Outcomes

Interim monitoring in sequential multiple assignment randomized trials

Set-valued dynamic treatment regimes for competing outcomes

Reinforcement Learning in Clinical Medicine: a Method to Optimize Dynamic Treatment Regime over Time.

Incorporating participants’ welfare into sequential multiple assignment randomized trials

A Generalized Logrank-type Test for Comparison of Treatment Regimes in Sequential Multiple Assignment Randomized Trials

Interim Monitoring of Sequential Multiple Assignment Randomized Trials Using Partial Information

Estimating individualized treatment rules in longitudinal studies with covariate-driven observation times

Personalized Dynamic Treatment Regimes in Continuous Time: A Bayesian Approach for Optimizing Clinical Decisions with Timing

Adaptive Randomization Methods for Sequential Multiple Assignment Randomized Trials (SMARTs) via Thompson Sampling