Abstract:Clinical trials involving multiple treatments utilize randomization of the treatment assignments to enable the evaluation of treatment efficacies in an unbiased manner. Such evaluation is performed in post hoc studies that usually use supervised-learning methods that rely on large amounts of data collected in a randomized fashion. That approach often proves to be suboptimal in that some participants may suffer and even die as a result of having not received the most appropriate treatments during the trial. Reinforcement-learning methods improve the situation by making it possible to learn the treatment efficacies dynamically during the course of the trial, and to adapt treatment assignments accordingly. Recent efforts using \textit{multi-arm bandits}, a type of reinforcement-learning methods, have focused on maximizing clinical outcomes for a population that was assumed to be homogeneous. However, those approaches have failed to account for the variability among participants that is becoming increasingly evident as a result of recent clinical-trial-based studies. We present a contextual-bandit-based online treatment optimization algorithm that, in choosing treatments for new participants in the study, takes into account not only the maximization of the clinical outcomes but also the patient characteristics. We evaluated our algorithm using a real clinical trial dataset from the International Stroke Trial. The results of our retrospective analysis indicate that the proposed approach performs significantly better than either a random assignment of treatments (the current gold standard) or a multi-arm-bandit-based approach, providing substantial gains in the percentage of participants who are assigned the most suitable treatments. The contextual-bandit and multi-arm bandit approaches provide 72.63% and 64.34% gains, respectively, compared to a random assignment.

Contextual Multi-Armed Bandits for Causal Marketing

Optimising Individual-Treatment-Effect Using Bandits

How causal machine learning can leverage marketing strategies: Assessing and improving the performance of a coupon campaign

Causal Contextual Bandits with Adaptive Context

Heterogeneous Causal Learning for Effectiveness Optimization in User Marketing

A Causal Approach for Business Optimization: Application on an Online Marketplace

Regret Analysis of Bandit Problems with Causal Background Knowledge

Efficient Real-world Testing of Causal Decision Making via Bayesian Experimental Design for Contextual Optimisation

Selectively Contextual Bandits

Regret Analysis of Causal Bandit Problems

Causally Abstracted Multi-armed Bandits

A Contextual-bandit-based Approach for Informed Decision-making in Clinical Trials

Causal Feature Selection Method for Contextual Multi-Armed Bandits in Recommender System

Robustly Improving Bandit Algorithms with Confounded and Selection Biased Offline Data: A Causal Approach

Post-Contextual-Bandit Inference

Deep Contextual Multi-armed Bandits

Doubly High-Dimensional Contextual Bandits: An Interpretable Model for Joint Assortment-Pricing

Contextual Multi-Armed Bandit for Email Layout Recommendation

AutoML for Contextual Bandits

Online Causal Inference for Advertising in Real-Time Bidding Auctions

Causal Bandits with General Causal Models and Interventions