Abstract:In statistical inference, uncertainty is unknown and all models are wrong. That is to say, a person who makes a statistical model and a prior distribution is simultaneously aware that both are fictional candidates. To study such cases, statistical measures have been constructed, such as cross validation, information criteria and marginal likelihood; however, their mathematical properties have not yet been completely clarified when statistical models are under- or over-parametrized. We introduce a place of mathematical theory of Bayesian statistics for unknown uncertainty, which clarifies general properties of cross validation, information criteria and marginal likelihood, even if an unknown data-generating process is unrealizable by a model or even if the posterior distribution cannot be approximated by any normal distribution. Hence it gives a helpful standpoint for a person who cannot believe in any specific model and prior. This paper consists of three parts. The first is a new result, whereas the second and third are well-known previous results with new experiments. We show there exists a more precise estimator of the generalization loss than leave-one-out cross validation, there exists a more accurate approximation of marginal likelihood than Bayesian information criterion, and the optimal hyperparameters for generalization loss and marginal likelihood are different. This article is part of the theme issue ‘Bayesian inference: challenges, perspectives, and prospects’.

The Mathematical Model of Multivariant Dynamic Observation Process

Multi-mode process monitoring method based on PCA mixture model

Mathematic Methods of Biological Process

Dynamic Processes Modeling and Monitoring Based on a Novel Dynamic Latent Variable Model.

Multimode Dynamic Process Monitoring Based on Mixture Canonical Variate Analysis Model

Diagnosing Multivariate Continuous-Time Models with Application to Affine Term Structure Models

The Influence of Observation Interference on System Evolution Prediction

Theory and Inference for a Class of Observation-driven Models with Application to Time Series of Counts

Prediction of Moving Target Behavior Optimization Based on Dirichlet Model

Multimode Progress Monitoring Based on Variational Bayesian Algorithm

Inference for Multiple and Conditional Observers

Predicting the tendency of topic discussion on the online social networks using a dynamic probability model

Detecting Stochastic Governing Laws with Observation on Stationary Distributions

Non-Bayesian Social Learning with Multiview Observations

Probability of constructing prediction model for observable of a dynamical process via time series

Model discovery for nonautonomous translation-invariant problems

A Multi-Model Modeling Approach to MIMO Nonlinear Systems

Mathematical theory of Bayesian statistics for unknown information source

ANALYSIS OF THE MULTIVARIATE TIME SERIES-IN AN OBSERVATION WINDOW

Multivariate Matérn Models -- A Spectral Approach

Mixture ratio modeling of dynamic systems