Abstract:Analysis of geospatial data has traditionally been model-based, with a mean model, customarily specified as a linear regression on the covariates, and a Gaussian process covariance model, encoding the spatial dependence. While nonlinear machine learning algorithms like neural networks are increasingly being used for spatial analysis, current approaches depart from the model-based setup and cannot explicitly incorporate spatial covariance. We propose NN-GLS , embedding neural networks directly within the traditional Gaussian process (GP) geostatistical model to accommodate nonlinear mean functions while retaining all other advantages of GP, like explicit modeling of the spatial covariance and predicting at new locations via kriging. In NN-GLS, estimation of the neural network parameters for the nonlinear mean of the Gaussian Process explicitly accounts for the spatial covariance through use of the generalized least squares (GLS) loss, thus, extending the linear case. We show that NN-GLS admits a representation as a special type of graph neural network (GNN). This connection facilitates the use of standard neural network computational techniques for irregular geospatial data, enabling novel and scalable mini-batching, backpropagation, and kriging schemes. We provide methodology to obtain uncertainty bounds for estimation and predictions from NN-GLS. Theoretically, we show that NN-GLS will be consistent for irregularly observed spatially correlated data processes. We also provide a finite sample concentration rate, which quantifies the need to accurately model the spatial covariance in neural networks for dependent data. To our knowledge, these are the first large-sample results for any neural network algorithm for irregular spatial data. We demonstrate the methodology through numerous simulations and an application to air pollution modeling. We develop a software implementation of NN-GLS in the Python package geospaNN. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets

High-Dimensional Bayesian Geostatistics

Highly Scalable Bayesian Geostatistical Modeling via Meshed Gaussian Processes on Partitioned Domains

spNNGP R package for Nearest Neighbor Gaussian Process models

Fast Bayesian inference of block Nearest Neighbor Gaussian models for large data

Bayesian Geostatistical Modeling for Discrete-Valued Processes

A Scalable Partitioned Approach to Model Massive Nonstationary Non-Gaussian Spatial Datasets

Nearest-Neighbor Mixture Models for Non-Gaussian Spatial Processes

Multivariate nearest‐neighbors Gaussian processes with random covariance matrices

High-dimensional Multivariate Geostatistics: A Bayesian Matrix-Normal Approach

smashGP: Large-Scale Spatial Modeling via Matrix-Free Gaussian Processes

Bayesian inference for high-dimensional nonstationary Gaussian processes

Neural Networks for Geospatial Data

Bayesian Nonstationary Spatial Modeling for Very Large Datasets

Hierarchical Mixture-of-Experts Model for Large-Scale Gaussian Process Regression

Spatial Factor Models for High-Dimensional and Large Spatial Data: An Application in Forest Variable Mapping

A Variational Approach for Modeling High-dimensional Spatial Generalized Linear Mixed Models

Spatial Factor Modeling: A Bayesian Matrix-Normal Approach for Misaligned Data

Grid-Parametrize-Split (GriPS) for Improved Scalable Inference in Spatial Big Data Analysis

Scalable Gaussian Process Using Inexact Admm For Big Data

Radial Neighbors for Provably Accurate Scalable Approximations of Gaussian Processes