Abstract:Whenever a binary classifier is used to provide decision support, it typically provides both a label prediction and a confidence value. Then, the decision maker is supposed to use the confidence value to calibrate how much to trust the prediction. In this context, it has been often argued that the confidence value should correspond to a well calibrated estimate of the probability that the predicted label matches the ground truth label. However, multiple lines of empirical evidence suggest that decision makers have difficulties at developing a good sense on when to trust a prediction using these confidence values. In this paper, our goal is first to understand why and then investigate how to construct more useful confidence values. We first argue that, for a broad class of utility functions, there exist data distributions for which a rational decision maker is, in general, unlikely to discover the optimal decision policy using the above confidence values -- an optimal decision maker would need to sometimes place more (less) trust on predictions with lower (higher) confidence values. However, we then show that, if the confidence values satisfy a natural alignment property with respect to the decision maker's confidence on her own predictions, there always exists an optimal decision policy under which the level of trust the decision maker would need to place on predictions is monotone on the confidence values, facilitating its discoverability. Further, we show that multicalibration with respect to the decision maker's confidence on her own predictions is a sufficient condition for alignment. Experiments on four different AI-assisted decision making tasks where a classifier provides decision support to real human experts validate our theoretical results and suggest that alignment may lead to better decisions.

Analytical Figures of Merit in Univariate, Multivariate, and Multiway Calibration: What Have We Learned? What Do We Still Need to Learn?

On Multivariate Calibration Problems

Multivariate curve resolution-alternating least-squares and second-order advantage in first-order calibration. A systematic characterisation for three-component analytical systems

When is Multicalibration Post-Processing Necessary?

Reassessing How to Compare and Improve the Calibration of Machine Learning Models

Multivariate Calibrations with Auxiliary Information

Calibration modeling by partial least-squares and principal component regression and its optimization using an improved leverage correction for prediction testing

Subspace Partial Least Squares Model for Multivariate Spectroscopic Calibration

Calibrating chemical multisensory devices for real world applications: An in-depth comparison of quantitative Machine Learning approaches

Representational Ethical Model Calibration

A Unifying Theory of Distance from Calibration

The Calibration Gap between Model and Human Confidence in Large Language Models

Multicalibration for Confidence Scoring in LLMs

Human-Aligned Calibration for AI-Assisted Decision Making

Three-way Mixed Effect ANOVA to Estimate MRMC Limits of Agreement

Confidence-Aware Multi-Field Model Calibration

Is calibration a fairness requirement? An argument from the point of view of moral philosophy and decision theory

Towards Fair and Calibrated Models

Calibrated Multivariate Regression with Application to Neural Semantic Basis Discovery

How Flawed Is ECE? An Analysis via Logit Smoothing

Calibrate: Interactive Analysis of Probabilistic Model Output