Abstract:Spatial nonstationarity, the location variance of features’ statistical distributions, is ubiquitous in many natural settings. For example, in geological reservoirs rock matrix porosity varies vertically due to geomechanical compaction trends, in mineral deposits grades vary due to sedimentation and concentration processes, in hydrology rainfall varies due to the atmosphere and topography interactions, and in metallurgy crystalline structures vary due to differential cooling. Conventional geostatistical modeling workflows rely on the assumption of stationarity to be able to model spatial features for geostatistical inference. Nevertheless, this is often not a realistic assumption when dealing with nonstationary spatial data and this has motivated a variety of nonstationary spatial modeling workflows such as trend and residual decomposition, cosimulation with secondary features, and spatial segmentation and independent modeling over stationary subdomains. The advent of deep learning technologies has enabled new workflows for modeling spatial relationships. However, there is a paucity of demonstrated best practice and general guidance on mitigation of spatial nonstationarity with deep learning in the geospatial context. We demonstrate the impact of two common types of geostatistical spatial nonstationarity on deep learning model prediction performance and propose the mitigation of such impacts using self-attention (vision transformer) models. We demonstrate the utility of vision transformers for the mitigation of nonstationarity with relative errors as low as 10%, exceeding the performance of alternative deep learning methods such as convolutional neural networks. We establish best practice by demonstrating the ability of self-attention networks for modeling large-scale spatial relationships in the presence of commonly observed geospatial nonstationarity.

Probabilistic Spatial Transformer Networks

Understanding when spatial transformer networks do not support invariance, and what to do about it

A Reinforcement Learning Approach for Sequential Spatial Transformer Networks

Spatial Transformer Introspective Neural Network.

STN-enhanced Message Passing Guided by Adversarial Learning for Human Pose Estimation

DeSTNet: Densely Fused Spatial Transformer Networks

Entropy Transformer Networks: A Learning Approach via Tangent Bundle Data Manifold

A Spatio-Temporal Transformer Network for Human Motion Prediction in Human-Robot Collaboration

Hierarchical Spatial Transformer Network

Polar Transformer Networks

Attention-Guided Spatial Transformer Networks for Fine-Grained Visual Recognition

Learning Joint Spatial-Temporal Transformations for Video Inpainting

Spatially Constrained Transformer with Efficient Global Relation Modelling for Spatio-Temporal Prediction

Spatial Transformer Networks for Curriculum Learning

Hyper-STTN: Social Group-aware Spatial-Temporal Transformer Network for Human Trajectory Prediction with Hypergraph Reasoning

Spatio-Temporal Graph Scattering Transform

Spatial-Temporal Convolutional Transformer Network for Multivariate Time Series Forecasting

Mitigation of spatial nonstationarity with vision transformers

Spatiotemporal Transformer Neural Network for Time-Series Forecasting

Spatial Transformer for 3D Point Clouds

3D Morphable Models as Spatial Transformer Networks