Abstract:Scoring rules for eliciting expert predictions of random variables are usually developed assuming that experts derive utility only from the quality of their predictions (e.g., score awarded by the rule, or payoff in a prediction market). We study a more realistic setting in which (a) the principal is a decision maker and will take a decision based on the expert's prediction; and (b) the expert has an inherent interest in the decision. For example, in a corporate decision market, the expert may derive different levels of utility from the actions taken by her manager. As a consequence the expert will usually have an incentive to misreport her forecast to influence the choice of the decision maker if typical scoring rules are used. We develop a general model for this setting and introduce the concept of a compensation rule. When combined with the expert's inherent utility for decisions, a compensation rule induces a net scoring rule that behaves like a normal scoring rule. Assuming full knowledge of expert utility, we provide a complete characterization of all (strictly) proper compensation rules. We then analyze the situation where the expert's utility function is not fully known to the decision maker. We show bounds on: (a) expert incentive to misreport; (b) the degree to which an expert will misreport; and (c) decision maker loss in utility due to such uncertainty. These bounds depend in natural ways on the degree of uncertainty, the local degree of convexity of net scoring function, and natural properties of the decision maker's utility function. They also suggest optimization procedures for the design of compensation rules. Finally, we briefly discuss the use of compensation rules as market scoring rules for self-interested experts in a prediction market.

Proxy Forecasting to Avoid Stochastic Decision Rules in Decision Markets

Eliciting Forecasts from Self-interested Experts: Scoring Rules for Decision Makers

Parametric Prediction from Parametric Agents

Learning from Delayed Outcomes via Proxies with Applications to Recommender Systems

Crowdsourced Outcome Determination in Prediction Markets

Strategy-Proof Prediction Markets

Timely Information from Prediction Markets

Proof of principle for a self-governing prediction and forecasting reward algorithm

Strategy-Proof Incentives for Predictions

Joint Scoring Rules: Zero-Sum Competition Avoids Performative Prediction

Self-Resolving Prediction Markets for Unverifiable Outcomes

Persuasion, Delegation, and Private Information in Algorithm-Assisted Decisions

Right Decisions from Wrong Predictions: A Mechanism Design Alternative to Individual Calibration

Progress in Absorber R&D 2: Windows

Bayesian social learning with decision making in multiple rounds

A decision-theoretic model for a principal-agent collaborative learning problem

On Manipulation in Prediction Markets When Participants Influence Outcomes Directly

A Market for Trading Forecasts: A Wagering Mechanism

Estimating and Incentivizing Imperfect-Knowledge Agents with Hidden Rewards

Improving Agent Decision Payoffs via a New Framework of Opponent Modeling

Social Learning with Heterogeneous Agents and Sequential Decision Making