Algorithmic Assistance with Recommendation-Dependent Preferences

Bryce McLaughlin,Jann Spiess

2024-01-20

Abstract:When an algorithm provides risk assessments, we typically think of them as helpful inputs to human decisions, such as when risk scores are presented to judges or doctors. However, a decision-maker may not only react to the information provided by the algorithm. The decision-maker may also view the algorithmic recommendation as a default action, making it costly for them to deviate, such as when a judge is reluctant to overrule a high-risk assessment for a defendant or a doctor fears the consequences of deviating from recommended procedures. To address such unintended consequences of algorithmic assistance, we propose a principal-agent model of joint human-machine decision-making. Within this model, we consider the effect and design of algorithmic recommendations when they affect choices not just by shifting beliefs, but also by altering preferences. We motivate this assumption from institutional factors, such as a desire to avoid audits, as well as from well-established models in behavioral science that predict loss aversion relative to a reference point, which here is set by the algorithm. We show that recommendation-dependent preferences create inefficiencies where the decision-maker is overly responsive to the recommendation. As a potential remedy, we discuss algorithms that strategically withhold recommendations, and show how they can improve the quality of final decisions.

Machine Learning,Human-Computer Interaction,General Economics

What problem does this paper attempt to address?

The paper attempts to address the issue of unintended consequences in algorithm-assisted decision-making. Specifically, when algorithms provide risk assessments or recommendations, decision-makers (such as judges or doctors) not only adjust their beliefs based on the information provided by the algorithm but also treat the algorithm's recommendation as the default action, which increases the cost of deviating from the recommendation. For example, judges may be reluctant to overturn a high-risk assessment, and doctors may avoid certain tests due to fear of the consequences of deviating from the recommended procedure. This reliance on recommendations leads to inefficient decision-making, as decision-makers overly depend on algorithmic recommendations without fully utilizing their private information.

To address this issue, the authors propose a principal-agent model to analyze the effects and design of algorithmic recommendations in human-machine joint decision-making. In this model, the authors consider the situation where algorithmic recommendations influence decisions not only by changing beliefs but also by altering preferences. The authors argue that recommendation dependence leads to overreaction to recommendations by decision-makers, resulting in efficiency losses. As a potential solution, the authors discuss strategically withholding recommendations and demonstrate how these algorithms can improve the quality of final decisions by reducing recommendation dependence.