Abstract:High throughput spatial transcriptomics (HST) is a rapidly emerging class of experimental technologies that allow for profiling gene expression in tissue samples at or near single‐cell resolution while retaining the spatial location of each sequencing unit within the tissue sample. Through analyzing HST data, we seek to identify sub‐populations of cells within a tissue sample that may inform biological phenomena. Existing computational methods either ignore the spatial heterogeneity in gene expression profiles, fail to account for important statistical features such as skewness, or are heuristic‐based network clustering methods that lack the inferential benefits of statistical modeling. To address this gap, we develop SPRUCE: a Bayesian spatial multivariate finite mixture model based on multivariate skew‐normal distributions, which is capable of identifying distinct cellular sub‐populations in HST data. We further implement a novel combination of Pólya–Gamma data augmentation and spatial random effects to infer spatially correlated mixture component membership probabilities without relying on approximate inference techniques. Via a simulation study, we demonstrate the detrimental inferential effects of ignoring skewness or spatial correlation in HST data. Using publicly available human brain HST data, SPRUCE outperforms existing methods in recovering expertly annotated brain layers. Finally, our application of SPRUCE to human breast cancer HST data indicates that SPRUCE can distinguish distinct cell populations within the tumor microenvironment. An R package spruce for fitting the proposed models is available through The Comprehensive R Archive Network (CRAN). This article is protected by copyright. All rights reserved

Mixtures of Spatial Spline Regressions

Spatial Heterogeneous Additive Partial Linear Model: A Joint Approach of Bivariate Spline and Forest Lasso

Flexible Basis Representations for Modeling Large Non-Gaussian Spatial Data

A bayesian multivariate mixture model for high throughput spatial transcriptomics

Algorithms for Space Mapping Method on Spline Spaces over Modified Hierarchical T-Meshes

Combining heterogeneous spatial datasets with process-based spatial fusion models: A unifying framework

TSSS: A Novel Triangulated Spherical Spline Smoothing for Surface-based Data

A Semiparametric Bayesian Approach to Heterogeneous Spatial Autoregressive Models

Multivariate Splines and Their Applications

Bayesian Function-on-Function Regression for Spatial Functional Data

Multidimensional Adaptive Penalised Splines with Application to Neurons' Activity Studies

Efficient Bayesian Multivariate Surface Regression

Identification of Linear Directions in Multivariate Adaptive Spline Models

A flexible spatial autoregressive modelling framework for mixed covariates of multiple data types

Spatial Homogeneity Pursuit of Regression Coefficients for Large Datasets

Gaussian modeling with B-splines for spatial functional data on irregular domains

Spatial Functional Data analysis: Irregular spacing and Bernstein polynomials

Bayesian Modeling with Spatial Curvature Processes

Nonstationary Spatial Gaussian Markov Random Fields

Mixture of Gaussian Processes and its Applications