Spatial Bayesian Neural Networks

Andrew Zammit-Mangion,Michael D. Kaminski,Ba-Hien Tran,Maurizio Filippone,Noel Cressie
2024-04-05
Abstract:interpretable, and well understood models that are routinely employed even though, as is revealed through prior and posterior predictive checks, these can poorly characterise the spatial heterogeneity in the underlying process of interest. Here, we propose a new, flexible class of spatial-process models, which we refer to as spatial Bayesian neural networks (SBNNs). An SBNN leverages the representational capacity of a Bayesian neural network; it is tailored to a spatial setting by incorporating a spatial ``embedding layer'' into the network and, possibly, spatially-varying network parameters. An SBNN is calibrated by matching its finite-dimensional distribution at locations on a fine gridding of space to that of a target process of interest. That process could be easy to simulate from or we may have many realisations from it. We propose several variants of SBNNs, most of which are able to match the finite-dimensional distribution of the target process at the selected grid better than conventional BNNs of similar complexity. We also show that an SBNN can be used to represent a variety of spatial processes often used in practice, such as Gaussian processes, lognormal processes, and max-stable processes. We briefly discuss the tools that could be used to make inference with SBNNs, and we conclude with a discussion of their advantages and limitations.
Machine Learning
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is: **How to construct a flexible and highly adaptable model to better capture the heterogeneity and non - stationarity in spatial data, thereby improving the limitations of traditional spatial statistical models?** Specifically, the paper points out that currently, models commonly used in spatial statistical analysis (such as the Gaussian process model) are simple and easy to use, but perform poorly when describing complex spatial heterogeneity and non - stationary behaviors. To solve this problem, the author proposes a new model framework - **Spatial Bayesian Neural Networks (SBNNs)**. By combining the powerful representation ability of Bayesian neural networks with specific spatial embedding layers and spatially - varying parameters, SBNNs can more accurately match the finite - dimensional distribution of the target spatial process. The following is a summary of the specific problems that the paper attempts to solve: 1. **Limitations of traditional models**: - Currently, commonly - used spatial statistical models (such as the Gaussian process, log - normal process, etc.) are insufficient in describing complex non - stationarity and spatial dependence. - These models usually require a large number of modeling decisions and diagnostic checks, increasing the complexity of the analysis. 2. **Design goals of SBNNs**: - Propose a general spatial process model that can be flexibly adapted regardless of the data or application scenarios. - By matching the finite - dimensional distribution of SBNNs on a fine spatial grid with the distribution of the target spatial process, reduce the need for specific model selection. 3. **Specific technical challenges**: - How to design SBNNs to effectively capture spatial covariance and non - stationary behaviors. - How to calibrate the hyper - parameters of SBNNs so that they can match the distribution characteristics of the target spatial process. - During the calibration process, how to handle the exploration of high - dimensional distribution functions. 4. **Core ideas of the solution**: - Introduce a spatial embedding layer, using radial basis functions (RBFs) to encode the input spatial location information. - Design spatially - varying parameters so that the network weights and biases change with the spatial position, thereby enhancing the ability to model non - stationarity. - Use the Wasserstein distance as a calibration index, and optimize the hyper - parameters by minimizing the distribution differences between SBNNs and the target spatial process. Through these designs, SBNNs can provide greater flexibility and adaptability in complex spatial data analysis tasks while reducing the dependence on traditional model selection and diagnosis.