Abstract:Conducting randomized experiments in education settings raises the question of how we can use machine learning techniques to improve educational interventions. Using Multi-Armed Bandits (MAB) algorithms like Thompson Sampling (TS) in adaptive experiments can increase students' chances of obtaining better outcomes by increasing the probability of assignment to the most optimal condition (arm), even before an intervention completes. This is an advantage over traditional A/B testing, which may allocate an equal number of students to both optimal and non-optimal conditions. The problem is the exploration-exploitation trade-off. Even though adaptive policies aim to collect enough information to allocate more students to better arms reliably, past work shows that this may not be enough exploration to draw reliable conclusions about whether arms differ. Hence, it is of interest to provide additional uniform random (UR) exploration throughout the experiment. This paper shows a real-world adaptive experiment on how students engage with instructors' weekly email reminders to build their time management habits. Our metric of interest is open email rates which tracks the arms represented by different subject lines. These are delivered following different allocation algorithms: UR, TS, and what we identified as TS† - which combines both TS and UR rewards to update its priors. We highlight problems with these adaptive algorithms - such as possible exploitation of an arm when there is no significant difference - and address their causes and consequences. Future directions includes studying situations where the early choice of the optimal arm is not ideal and how adaptive algorithms can address them.

Contextual Multi-Armed Bandit for Email Layout Recommendation

Latent Contextual Bandits and their Application to Personalized Recommendations for New Users

A Contextual-Bandit Approach to Personalized News Article Recommendation

Contextual Multi-Armed Bandits for Causal Marketing

Neural Contextual Bandits for Personalized Recommendation

Selectively Contextual Bandits

Achieving User-Side Fairness in Contextual Bandits

Simulated Contextual Bandits for Personalization Tasks from Recommendation Datasets

Online Evaluation of Audiences for Targeted Advertising via Bandit Experiments

Learning Personalized Page Content Ranking Using Customer Representation

Increasing Students' Engagement to Reminder Emails Through Multi-Armed Bandits

Deep Contextual Multi-armed Bandits

Contextual Dependent Click Bandit Algorithm For Web Recommendation

Jump Starting Bandits with LLM-Generated Prior Knowledge

A Field Test of Bandit Algorithms for Recommendations: Understanding the Validity of Assumptions on Human Preferences in Multi-armed Bandits

Learning Contextual Bandits in a Non-stationary Environment

E-commerce in Your Inbox: Product Recommendations at Scale

Design Principles of Robust Multi-Armed Bandit Framework in Video Recommendations

Contextual Bandit Applications in Customer Support Bot

Contextual Bandit Approach-based Recommendation System for Personalized Web-based Services

Incentivising Exploration and Recommendations for Contextual Bandits with Payments