A Bayesian Measure of Model Accuracy

Gabriel Hideki Vatanabe Brunello,Eduardo Yoshio Nakano

DOI: https://doi.org/10.3390/e26060510

IF: 2.738

2024-06-13

Entropy

Abstract:Ensuring that the proposed probabilistic model accurately represents the problem is a critical step in statistical modeling, as choosing a poorly fitting model can have significant repercussions on the decision-making process. The primary objective of statistical modeling often revolves around predicting new observations, highlighting the importance of assessing the model's accuracy. However, current methods for evaluating predictive ability typically involve model comparison, which may not guarantee a good model selection. This work presents an accuracy measure designed for evaluating a model's predictive capability. This measure, which is straightforward and easy to understand, includes a decision criterion for model rejection. The development of this proposal adopts a Bayesian perspective of inference, elucidating the underlying concepts and outlining the necessary procedures for application. To illustrate its utility, the proposed methodology was applied to real-world data, facilitating an assessment of its practicality in real-world scenarios.

physics, multidisciplinary

What problem does this paper attempt to address?

The problem this paper attempts to address is the proposal of a new Bayesian method to evaluate the quality of statistical models, particularly based on their predictive ability. Specifically:

1. **Model Accuracy Evaluation**: Ensuring that the proposed probabilistic model accurately represents the problem is crucial, as choosing an inappropriate model can significantly impact the decision-making process. Current methods for evaluating predictive ability often involve model comparison, which may not guarantee good model selection.

2. **Simplified Evaluation Criteria**: The paper proposes a simple and easy-to-understand accuracy measurement method, including a decision criterion for model rejection. This method adopts a Bayesian inference perspective, elucidates the fundamental concepts, and outlines the procedures required for application.

3. **Practicality and Applicability Verification**: The method's practicality is verified by applying it to real data, demonstrating its usefulness in real-world scenarios. Additionally, the method is simulated within the exponential distribution of generalized linear models to test its performance.

Overall, this paper aims to introduce a Bayesian method based on predictive ability to evaluate the appropriateness of models and provide an objective standard for assessing model quality.