Abstract:We study the problem of making predictions of an adversarially chosen high-dimensional state that are unbiased subject to an arbitrary collection of conditioning events, with the goal of tailoring these events to downstream decision makers. We give efficient algorithms for solving this problem, as well as a number of applications that stem from choosing an appropriate set of conditioning events. For example, we can efficiently make predictions targeted at polynomially many decision makers, giving each of them optimal swap regret if they best-respond to our predictions. We generalize this to online combinatorial optimization, where the decision makers have a very large action space, to give the first algorithms offering polynomially many decision makers no regret on polynomially many subsequences that may depend on their actions and the context. We apply these results to get efficient no-subsequence-regret algorithms in extensive-form games (EFGs), yielding a new family of regret guarantees for EFGs that generalizes some existing EFG regret notions, e.g. regret to informed causal deviations, and is generally incomparable to other known such notions. Next, we develop a novel transparent alternative to conformal prediction for building valid online adversarial multiclass prediction sets. We produce class scores that downstream algorithms can use for producing valid-coverage prediction sets, as if these scores were the true conditional class probabilities. We show this implies strong conditional validity guarantees including set-size-conditional and multigroup-fair coverage for polynomially many downstream prediction sets. Moreover, our class scores can be guaranteed to have improved $L_2$ loss, cross-entropy loss, and generally any Bregman loss, compared to any collection of benchmark models, yielding a high-dimensional real-valued version of omniprediction.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is how to generate unbiased prediction results when making predictions in high - dimensional states, and these prediction results need to meet the requirements of any set of conditional events. Specifically, the authors focus on how to efficiently generate predictions for high - dimensional states in an adversarial environment (that is, the state is selected by the adversary) so that these predictions are unbiased under a given series of conditional events. In addition, the paper also explores how these predictions can be combined with the decision - making processes of downstream decision - makers to achieve goals such as swap regret - free, type regret - free, and subsequence regret - free. ### Main Research Questions 1. **Unbiasedness of High - Dimensional Predictions**: How to generate unbiased prediction results in high - dimensional states, and these prediction results need to remain unbiased under a series of conditional events. 2. **Combination of Prediction and Decision - making**: How to combine these unbiased prediction results with the decision - making processes of downstream decision - makers to achieve various forms of regret - free (such as swap regret - free, type regret - free, and subsequence regret - free). 3. **Applications in Online Combinatorial Optimization and Extensive - form Games**: How to apply the above - mentioned methods to online combinatorial optimization problems and extensive - form games to achieve subsequence regret guarantees. 4. **Uncertainty Quantification**: How to generate prediction sets with transparent coverage guarantees, and these prediction sets can be used for uncertainty quantification in multi - class classification and multi - label classification. ### Specific Research Contents - **Unbiased Prediction Algorithm**: The authors propose an efficient algorithm that can generate unbiased prediction results within polynomial time, and these prediction results remain unbiased under a given series of conditional events. The time complexity of the algorithm has a polynomial relationship with the dimension $d$, the number of rounds $T$, and the number of conditional events $|E|$. - **Relationship between Prediction and Decision - making**: By connecting the prediction with the optimal response strategies of downstream decision - makers, the authors show how to generate unbiased prediction results so that downstream decision - makers can achieve swap regret - free and type regret - free when making optimal responses to these predictions. - **Online Combinatorial Optimization and Extensive - form Games**: The authors apply these methods to online combinatorial optimization problems and extensive - form games, showing how to generate prediction results without subsequence regret, which is of great significance in practical applications. - **Uncertainty Quantification**: The authors propose a scoring - free method to generate prediction sets, which have transparent coverage guarantees and can be used for uncertainty quantification in multi - class classification and multi - label classification. ### Application Examples - **Swap Regret - free for Multiple Decision - makers**: By generating unbiased prediction results, multiple decision - makers can achieve swap regret - free when making optimal responses to these predictions. - **Online Combinatorial Optimization**: In online combinatorial optimization problems, generate prediction results without subsequence regret, so that decision - makers can still achieve regret - free when facing a large number of possible actions. - **Extensive - form Games**: In extensive - form games, generate prediction results without subsequence regret, so that game participants can still achieve regret - free when facing complex decision trees. - **Uncertainty Quantification**: Generate prediction sets with transparent coverage guarantees, and these prediction sets can be used for uncertainty quantification in multi - class classification and multi - label classification. ### Conclusion The paper solves the problem of generating unbiased predictions in high - dimensional states by proposing an efficient unbiased prediction algorithm and applies it to various practical problems, such as online combinatorial optimization, extensive - form games, and uncertainty quantification. These methods are not only of great theoretical significance but also show strong performance in practical applications.

High-Dimensional Prediction for Sequential Decision Making