Abstract:Recent likelihood theory produces $p$-values that have remarkable accuracy and wide applicability. The calculations use familiar tools such as maximum likelihood values (MLEs), observed information and parameter rescaling. The usual evaluation of such $p$-values is by simulations, and such simulations do verify that the global distribution of the $p$-values is uniform(0, 1), to high accuracy in repeated sampling. The derivation of the $p$-values, however, asserts a stronger statement, that they have a uniform(0, 1) distribution conditionally, given identified precision information provided by the data. We take a simple regression example that involves exact precision information and use large sample techniques to extract highly accurate information as to the statistical position of the data point with respect to the parameter: specifically, we examine various $p$-values and Bayesian posterior survivor $s$-values for validity. With observed data we numerically evaluate the various $p$-values and $s$-values, and we also record the related general formulas. We then assess the numerical values for accuracy using Markov chain Monte Carlo (McMC) methods. We also propose some third-order likelihood-based procedures for obtaining means and variances of Bayesian posterior distributions, again followed by McMC assessment. Finally we propose some adaptive McMC methods to improve the simulation acceptance rates. All these methods are based on asymptotic analysis that derives from the effect of additional data. And the methods use simple calculations based on familiar maximizing values and related informations. The example illustrates the general formulas and the ease of calculations, while the McMC assessments demonstrate the numerical validity of the $p$-values as percentage position of a data point. The example, however, is very simple and transparent, and thus gives little indication that in a wide generality of models the formulas do accurately separate information for almost any parameter of interest, and then do give accurate $p$-value determinations from that information. As illustration an enigmatic problem in the literature is discussed and simulations are recorded; various examples in the literature are cited.

Extending the Scope of Inference About Predictive Ability to Machine Learning Methods

Adaptive Lasso in high-dimensional settings

A NEW PERSPECTIVE ON ROBUST M-ESTIMATION: FINITE SAMPLE THEORY AND APPLICATIONS TO DEPENDENCE-ADJUSTED MULTIPLE TESTING

More powerful post-selection inference, with application to the Lasso

Simultaneous Inference for High-Dimensional Linear Models

Inference for High-Dimensional Linear Expectile Regression with De-Biasing Method

Electronic structure study by means of X-ray spectroscopy and theoretical calculations of the "ferric star" single molecule magnet

Efficient Inference on High-Dimensional Linear Models with Missing Outcomes

Machine-Learning Tests for Effects on Multiple Outcomes

The out-of-sample prediction error of the square-root-LASSO and related estimators

Assessing Sensitivity of Machine Learning Predictions.A Novel Toolbox with an Application to Financial Literacy

Revisiting inference after prediction

On Lasso and adaptive Lasso for non-random sample in credit scoring

High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking

The Risk of Machine Learning

Maximum-Subsampling Test of Equal Predictive Ability

Highly adaptive LASSO: Machine learning that provides valid nonparametric inference in realistic models

Inject Machine Learning into Significance Test for Misspecified Linear Models

Higher Accuracy for Bayesian and Frequentist Inference: Large Sample Theory for Small Sample Likelihood

Predictive ability tests with possibly overlapping models

Should Humans Lie to Machines? The Incentive Compatibility of Lasso and GLM Structured Sparsity Estimators