Abstract:Decision support systems for classification tasks are predominantly designed to predict the value of the ground truth labels. However, since their predictions are not perfect, these systems also need to make human experts understand when and how to use these predictions to update their own predictions. Unfortunately, this has been proven challenging. In this context, it has been recently argued that an alternative type of decision support systems may circumvent this challenge. Rather than providing a single label prediction, these systems provide a set of label prediction values constructed using a conformal predictor, namely a prediction set, and forcefully ask experts to predict a label value from the prediction set. However, the design and evaluation of these systems have so far relied on stylized expert models, questioning their promise. In this paper, we revisit the design of this type of systems from the perspective of online learning and develop a methodology that does not require, nor assumes, an expert model. Our methodology leverages the nested structure of the prediction sets provided by any conformal predictor and a natural counterfactual monotonicity assumption to achieve an exponential improvement in regret in comparison to vanilla bandit algorithms. We conduct a large-scale human subject study ($n = 2{,}751$) to compare our methodology to several competitive baselines. The results show that, for decision support systems based on prediction sets, limiting experts' level of agency leads to greater performance than allowing experts to always exercise their own agency. We have made available the data gathered in our human subject study as well as an open source implementation of our system at <a class="link-external link-https" href="https://github.com/Networks-Learning/counterfactual-prediction-sets" rel="external noopener nofollow">this https URL</a>.

Machine Intelligence for Individualized Decision Making Under a Counterfactual World: A Rejoinder

Individualized Decision-Making Under Partial Identification: Three Perspectives, Two Optimality Results, and One Paradox

Decision Theory for Treatment Choice Problems with Partial Identification

Clinical decision making under uncertainty: a bootstrapped counterfactual inference approach

Policy Learning with Asymmetric Counterfactual Utilities

Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems

On Robustness of Individualized Decision Rules

Probabilistic Prediction for Binary Treatment Choice: with focus on personalized medicine

Relative rationality: Is machine rationality subjective?

Algorithmic Decision Making in the Presence of Unmeasured Confounding

A Bandit Model for Human-Machine Decision Making with Private Information and Opacity

Rejoinder to "Perspectives on `harm' in personalized medicine -- an alternative perspective"

Conformal Counterfactual Inference under Hidden Confounding

Uncertain Machine Ethical Decisions Using Hypothetical Retrospection

Rational Choice and Artificial Intelligence

Decision-aid or Controller? Steering Human Decision Makers with Algorithms

On the (non-) reliance on algorithms—A decision-theoretic account

Designing Decision Support Systems Using Counterfactual Prediction Sets

Optimal decision making and matching are tied through diminishing returns

Apparently Irrational Choice as Optimal Sequential Decision Making

A Subjective Model of Human Decision Making Based on Quantum Decision Theory