Abstract:This paper re-visits the problem of deciding between two simple hypotheses, the setting considered by Neyman and Pearson in developing their fundamental lemma. It studies the decision process induced by the most powerful test and the receiver operating characteristic function associated with this decision process. It addresses the question of how to report the decision arising from the decision function. It also examines the P-functional (the P-value statistic) and its role in the decision-making process. The impetus of this work is the continuing criticisms of statistical decision-making procedures that use the P-functional and a level of significance (LoS) of 0.05. A point made is that if one is going to use the value of the P-functional, then it should be used in an equivalent manner as the most powerful decision function, but if one wants to obtain from its value the degree of support for either hypotheses, then the value of its density under the alternative is the proper quantity to use. Replicability of results are discussed. Knowledge updating through Bayes theorem when given the decision or the value of the P-functional is also discussed, and it is argued that sequential learning is a coherent way of finding the truth. But the impact of publication bias is also demonstrated to be quite serious in the search for truth. It is argued that decision-makers are free to choose their own LoS, since the additional summary measures will automatically take their LoS choices into consideration. Three approaches for choosing an optimal LoS are discussed and a procedure for sample size determination is described. Ideas are illustrated by concrete problems and by the lady tea-tasting experiment of Fisher which ushered null hypothesis significance testing. It is hoped that by considering this fundamental setting of simple hypotheses, a better understanding of more complex settings will ensue.

Social teaching: Being informative vs. being right in sequential decision making

Beliefs in Decision-Making Cascades

Sequential Bayesian learning in linear networks with random decision making

Sequential Processing of Observations in Human Decision-Making Systems

Bayesian social learning with decision making in multiple rounds

Social Influence Undermines the Wisdom of the Crowd in Sequential Decision Making

Social Learning with Heterogeneous Agents and Sequential Decision Making

Experimental Researches of Sequential Decision-Making under Imperfect Information——Experimental Tests of the Bayesian Model for Herd Behavior

Decision Making in Star Networks With Incorrect Beliefs

Bayesian decision making in human collectives with binary choices

Distributed Hypothesis Testing With Social Learning and Symmetric Fusion

Bayesian Social Learning in Linear Networks of Agents with Random Behavior

Bayesian Heuristics for Group Decisions

The Search for Truth through Data: NP Decision Processes, ROC Functions, $P$-Functionals, Knowledge Updating and Sequential Learning

On Non-Bayesian Social Learning

Social Learning with Bayesian Agents and Random Decision Making

Sequential Bayesian Learning with A Self-Interested Coordinator

Social Opinion Formation and Decision Making Under Communication Trends

Opinion Dynamics in Multi-Agent Systems with Binary Decision Exchanges

Revealed Information