Spectral smooth tests for goodness-of-fit
Victor Candido Reis,Rafael Izbicki
DOI: https://doi.org/10.48550/arXiv.2308.06601
2023-08-12
Abstract:Goodness-of-fit tests are crucial tools for assessing the validity of statistical models. In this paper, we introduce a novel approach, the Spectral Smooth Test (SST), that generalizes Neyman's smooth test to high-dimensional data settings. While conventional goodness-of-fit tests for univariate data are well-established, extending them to high dimensions, such as images, trajectories, and SNPs, poses significant challenges. Our proposed SST leverages spectral bases, which adapt naturally to the geometry of feature spaces, to model multivariate distributions. Unlike traditional orthogonal bases, these spectral bases are tailored to the data distribution, enabling more effective function modeling. The SST framework offers a principled way to estimate the underlying model, thereby providing actionable insights even when the null hypothesis is rejected. We present experimental results demonstrating the robustness of SST across various tuning parameter choices and compare its performance against other goodness-of-fit tests. Furthermore, we apply SST to the MNIST dataset as a real-world example, showcasing its effectiveness in high-dimensional scenarios.
Methodology