Abstract:Deep reinforcement learning (DRL) has opened up many unprecedented opportunities in revolutionizing the digital marketing field. In this study, we designed a DRL-based personalized targeting strategy in a sequential setting. We show that the strategy is able to address three important challenges of sequential targeting: (1) forward looking (balancing between a firm’s current revenue and future revenues), (2) earning while learning (maximizing profits while continuously learning through exploration-exploitation), and (3) scalability (coping with a high-dimensional state and policy space). We illustrate this through a novel design of a DRL-based artificial intelligence (AI) agent. To better adapt DRL to complex consumer behavior dimensions, we proposed a quantization-based uncertainty learning heuristic for efficient exploration-exploitation. Our policy evaluation results through simulation suggest that the proposed DRL agent generates 26.75% more long-term revenues than can the non-DRL approaches on average and learns 76.92% faster than the second fastest model among all benchmarks. Further, in order to better understand the potential underlying mechanisms, we conducted multiple interpretability analyses to explain the patterns of learned optimal policy at both the individual and population levels. Our findings provide important managerial-relevant and theory-consistent insights. For instance, consecutive price promotions at the beginning can capture price-sensitive consumers’ immediate attention, whereas carefully spaced nonpromotional “cooldown” periods between price promotions can allow consumers to adjust their reference points. Additionally, consideration of future revenues is necessary from a long-term horizon, but weighing the future too much can also dampen revenues. In addition, analyses of heterogeneous treatment effects suggest that the optimal promotion sequence pattern highly varies across the consumer engagement stages. Overall, our study results demonstrate DRL’s potential to optimize these strategies’ combination to maximize long-term revenues. This paper was accepted by Kartik Hosanagar, information systems. Supplemental Material: The data files and online appendix are available at https://doi.org/10.1287/mnsc.2022.4621 .

Beyond Ads: Sequential Decision-Making Algorithms in Law and Public Policy

The Principles and Limits of Algorithm-in-the-Loop Decision Making

Algorithmic Decision Making in the Presence of Unmeasured Confounding

Machine Decisions and Human Consequences

HUMAN DECISIONS AND MACHINE PREDICTIONS

Fairness in Learning-Based Sequential Decision Algorithms: A Survey

Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies

Unpacking the Black Box: Regulating Algorithmic Decisions

Algorithms and Decision-Making in the Public Sector

A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare

How do "technical" design-choices made when building algorithmic decision-making tools for criminal justice authorities create constitutional dangers? Part II

Bridging the gap: Towards an Expanded Toolkit for AI-driven Decision-Making in the Public Sector

Choices, Risks, and Reward Reports: Charting Public Policy for Reinforcement Learning Systems

Rethinking Administrative Law for Algorithmic Decision Making

Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems

Challenging the Human-in-the-loop in Algorithmic Decision-making

Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction

The Legal Duty to Search for Less Discriminatory Algorithms

Procedural Fairness as Stepping Stone for Successful Implementation of Algorithmic Decision-Making in Public Administration: Review and Outlook

Deep Reinforcement Learning for Sequential Targeting

Legitimate Power, Illegitimate Automation: The problem of ignoring legitimacy in automated decision systems