Abstract:Reliability of sequential hypothesis testing can be greatly improved when decision maker is given the freedom to adaptively take an action that determines the distribution of the current collected sample. Such advantage of sampling adaptivity has been realized since Chernoff's seminal paper in 1959 [1]. While a large body of works have explored and investigated the gain of adaptivity, in the general multiple-hypothesis setting, the fundamental limits of individual error probabilities have not been fully understood. In particular, in the asymptotic regime as the expected stopping time tends to infinity, the error exponents are only characterized in specific cases, such as that of the total error probability. In this paper, we consider a general setup of active sequential multiple-hypothesis testing where at each time slot, a temporally varying subset of data sources (out of a known set) emerges from which the decision maker can select to collect samples, subject to a family of expected selection budget constraints. The selection of sources, understood as the "action" at each time slot, is constrained in a predefined action space. At the end of each time slot, the decision maker either decides to make the inference on the $M$ hypotheses, or continues to observe the data sources for the next time slot. The optimal tradeoffs among $M(M-1)$ types of error exponents are characterized. A companion asymptotically optimal test that strikes the balance between exploration and exploitation is proposed to achieve any target error exponents within the region. To the best of our knowledge, this is the first time in the literature to identify such tradeoffs among error exponents, and it uncovers the tension among different action taking policies even in the basic setting of Chernoff [1].

The Framing And Evaluation Of Multiple Hypotheses

The Study of Cognitive Theory for Framing Effect in Entrepreneurial Risky Decision-making

The Study of Cognitive Mechanism of Framing Effect:A Priority Heuristic Perspective

The Professional Judgment of Audit:a Research Based on Lens Model

A model of multiple hypothesis testing

An Experimental Study on the Influence of the Expenditure of Cognitive Resource on Framing Effect

A Study on the Celebral Hemisphere Positioning of Processing for the Framing Effect

Doublethink: simultaneous Bayesian-frequentist model-averaged hypothesis testing

A formal model of fuzzy-trace theory: Variations on framing effects and the Allais Paradox.

A confidence framing effect: Flexible use of evidence in metacognitive monitoring

Confidences in Hypotheses

Heuristic assumptions

The relevance of Kahneman and Tversky's concept of framing to organization behavior

Multiple testing with the structure adaptive Benjamini-Hochberg algorithm

An Empirical Examination of the Evaluative AI Framework

Comparisons of methods for multiple hypothesis testing in neuropsychological research.

Hypothesis Testing in Econometrics

Tradeoffs among Action Taking Policies Matter in Active Sequential Multi-Hypothesis Testing: the Optimal Error Exponent Region

Deep thoughts and shallow frames: on the susceptibility to framing effects

On LAO Testing of Multiple Hypotheses for Many Independent Objects

Using Bayes factors to limit forensic testimony to forensics: composite hypotheses