Abstract:Measures of discordance between datasets have become an essential part of cosmological analyses. It is important to accurately evaluate the significance of such discordances when present. We propose here a Bayesian interpretation of inconsistency measures that can extract information about physical inconsistencies in the presence of data scatter. The new framework is based on the conditional probability distribution of the level of physical inconsistency given the obtained value of the measure. We use the index of inconsistency as a case study to illustrate the new interpretation framework, but this can be generalized to other metrics. Importantly, there are two aspects in the quantification of inconsistency that behave differently as the number of model parameters increases. The first is the probability for the level of physical inconsistency to reach a threshold which drops with the increase of the number of parameters under consideration. Then the second is the actual level of physical inconsistency which remains rather insensitive to such an increase in parameters. This often-overlooked ambiguity leads to an underestimation of the physical inconsistency by the commonly adopted significance level when used, but is resolved by the Bayesian interpretation because the conditional probability distribution includes all the statistical information of the level of physical inconsistency. Finally, we apply the Bayesian interpretation to examine the (in)consistency between Planck versus the Cepheid-based local measurement, the Dark Energy Survey (DES), the Atacama Cosmology Telescope (ACT) and WMAP. We confirm and revisit the degrees of previous physical inconsistencies and show the stability of the new interpretation to the number of cosmological parameters compared to the commonly used n-$\sigma$ interpretation when applied to cosmological tensions in multi-parameter spaces.

Local inconsistency detection using the Kullback–Leibler divergence measure

Detecting abnormal situations using the Kullback-Leibler divergence.

On the Exploration of Local Significant Differences for Two-Sample Test.

Inconsistency identification in network meta-analysis via stochastic search variable selection

Consistency of invariance-based randomization tests

Distributional Overlap: Simple, Multivariate, Parametric, and Nonparametric Tests for Alienation, Convergence, and General Distributional Difference Issues

New inconsistency indicators for incomplete pairwise comparisons matrices

Comparison of approaches for local testing with functional test statistics

A Bayesian interpretation of inconsistency measures in cosmology

Assessing compatibility of two laboratories: formulations as a statistical hypothesis testing problem

Identification-robust methods for comparing inequality with an application to regional disparities

Reducing multivariate independence testing to two bivariate means comparisons

BET on Independence

The key properties of inconsistency indicators for a triad in pairwise comparison matrices

On consistency and inconsistency of nonparametric tests

A Ball Divergence Based Measure For Conditional Independence Testing

On the choice of the splitting ratio for the split likelihood ratio test

Measuring the Significance of Inconsistency in the Viewpoints Framework

More accurate tests for the statistical significance of result differences

On the prevalence of information inconsistency in normal linear models

A low variance consistent test of relative dependency