PHYSTAT Informal Review: Marginalizing versus Profiling of Nuisance Parameters

Robert D. Cousins,Larry Wasserman
2024-04-26
Abstract:This is a writeup, with some elaboration, of the talks by the two authors (a physicist and a statistician) at the first PHYSTAT Informal review on January 24, 2024. We discuss Bayesian and frequentist approaches to dealing with nuisance parameters, in particular, integrated versus profiled likelihood methods. In regular models, with finitely many parameters and large sample sizes, the two approaches are asymptotically equivalent. But, outside this setting, the two methods can lead to different tests and confidence intervals. Assessing which approach is better generally requires comparing the power of the tests or the length of the confidence intervals. This analysis has to be conducted on a case-by-case basis. In the extreme case where the number of nuisance parameters is very large, possibly infinite, neither approach may be useful. Part I provides an informal history of usage in high energy particle physics, including a simple illustrative example. Part II includes an overview of some more recently developed methods in the statistics literature, including methods applicable when the use of the likelihood function is problematic.
Data Analysis, Statistics and Probability,Applications
What problem does this paper attempt to address?
This paper discusses two different statistical methods for handling nuisance parameters in physical experiments: marginalization and profiled likelihood methods. These two methods are asymptotically equivalent in the context of regular models and large sample sizes, but may lead to different tests and confidence intervals in other situations. Evaluating which method is better often requires comparing the power of tests or the length of confidence intervals, which needs to be analyzed on a case-by-case basis. The paper is divided into two parts. The first part provides a historical overview of how physicists have dealt with nuisance parameters in high-energy particle physics, including a simple example that explains why the profiled likelihood method is more commonly used in practice. The second part introduces some newer methods from the statistical literature, with a particular focus on how to handle problems with the likelihood function. The authors discuss how to construct confidence intervals under the likelihood function, regularity, and large sample conditions, and compare parameterized bootstrap and marginalization methods. They also explore five different methods for calculating the significance Z in the Poisson mean ratio problem. Furthermore, the paper points out that in the case of a large number of nuisance parameters (potentially infinite), neither method may be suitable. The paper emphasizes that in practical applications, the choice between the two methods depends on the specific situation, and coverage studies of the statistical methods are needed to ensure the accuracy of parameter estimation. The paper also mentions historical developments and statistical tools, such as the MINUIT program, as well as the widely used profiled likelihood ratio as a test statistic in high-energy physics experiments. In summary, this paper aims to facilitate dialogue between physicists and statisticians, reviewing and comparing different statistical strategies for handling nuisance parameters, as well as their applications and limitations in physical experiments.