Abstract:This paper re-visits the problem of deciding between two simple hypotheses, the setting considered by Neyman and Pearson in developing their fundamental lemma. It studies the decision process induced by the most powerful test and the receiver operating characteristic function associated with this decision process. It addresses the question of how to report the decision arising from the decision function. It also examines the P-functional (the P-value statistic) and its role in the decision-making process. The impetus of this work is the continuing criticisms of statistical decision-making procedures that use the P-functional and a level of significance (LoS) of 0.05. A point made is that if one is going to use the value of the P-functional, then it should be used in an equivalent manner as the most powerful decision function, but if one wants to obtain from its value the degree of support for either hypotheses, then the value of its density under the alternative is the proper quantity to use. Replicability of results are discussed. Knowledge updating through Bayes theorem when given the decision or the value of the P-functional is also discussed, and it is argued that sequential learning is a coherent way of finding the truth. But the impact of publication bias is also demonstrated to be quite serious in the search for truth. It is argued that decision-makers are free to choose their own LoS, since the additional summary measures will automatically take their LoS choices into consideration. Three approaches for choosing an optimal LoS are discussed and a procedure for sample size determination is described. Ideas are illustrated by concrete problems and by the lady tea-tasting experiment of Fisher which ushered null hypothesis significance testing. It is hoped that by considering this fundamental setting of simple hypotheses, a better understanding of more complex settings will ensue.

The Optimal Decision by Knowledge Distribution

From Data to Decisions: Distributionally Robust Optimization Is Optimal

The Role of Time in the Creation of Knowledge

Optimal Scoring Rule Design under Partial Knowledge

A Pareto Dominance Principle for Data-Driven Optimization

Selecting Optimal Decisions Via Distributionally Robust Nearest-Neighbor Regression.

Distributionally Robust Optimization Under Distorted Expectations

Incomplete Information System And Its Optimal Selections

Distributionally Robust Decision Making Leveraging Conditional Distributions

Optimal Observables for Minimum-Error State Discrimination in General Probabilistic Theories

Optimal Decision Rules Based on Inclusion Degree Theory

Learning and Decision-Making with Data: Optimal Formulations and Phase Transitions

Decision-making psychology and method under zero-knowledge context

Distributionally Robust Optimization with Decision-Dependent Information Discovery

Knowledge-based system for three-way decision-making under uncertainty

Behavioral Utility-Based Distributed Detection With Conditionally Independent Observations

Probabilistic vs. non-probabilistic approaches to the neurobiology of perceptual decision-making

Sequential Processing of Observations in Human Decision-Making Systems

Distributionally Favorable Optimization: A Framework for Data-Driven Decision-Making with Endogenous Outliers

The Search for Truth through Data: NP Decision Processes, ROC Functions, $P$-Functionals, Knowledge Updating and Sequential Learning

Distributionally Robust Multi-Stage Decision Problem and Satisficing