Abstract:Characterising intractable high-dimensional random variables is one of the fundamental challenges in stochastic computation. The recent surge of transport maps offers a mathematical foundation and new insights for tackling this challenge by coupling intractable random variables with tractable reference random variables. This paper generalises the functional tensor-train approximation of the inverse Rosenblatt transport recently developed by Dolgov et al. (Stat Comput 30:603--625, 2020) to a wide class of high-dimensional non-negative functions, such as unnormalised probability density functions. First, we extend the inverse Rosenblatt transform to enable the transport to general reference measures other than the uniform measure. We develop an efficient procedure to compute this transport from a squared tensor-train decomposition which preserves the monotonicity. More crucially, we integrate the proposed order-preserving functional tensor-train transport into a nested variable transformation framework inspired by the layered structure of deep neural networks. The resulting deep inverse Rosenblatt transport significantly expands the capability of tensor approximations and transport maps to random variables with complicated nonlinear interactions and concentrated density functions. We demonstrate the efficiency of the proposed approach on a range of applications in statistical learning and uncertainty quantification, including parameter estimation for dynamical systems and inverse problems constrained by partial differential equations.

Arithmetic circuit tensor networks, multivariable function representation, and high-dimensional integration

What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?

Tensor Neural Network and Its Numerical Integration

Compressing multivariate functions with tree tensor networks

Quantics Tensor Cross Interpolation for High-Resolution, Parsimonious Representations of Multivariate Functions in Physics and Beyond

Quantics Tensor Cross Interpolation for High-Resolution Parsimonious Representations of Multivariate Functions

On Faster Marginalization with Squared Circuits via Orthonormalization

Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods

Learning tensor networks with tensor cross interpolation: new algorithms and libraries

Lower and Upper Bounds on the VC-Dimension of Tensor Network Models

Tensor-network-assisted variational quantum algorithm

Tensor Neural Network Interpolation and Its Applications

Fully tensorial approach to hypercomplex neural networks

Tensor Network Computations That Capture Strict Variationality, Volume Law Behavior, and the Efficient Representation of Neural Network States

Deep composition of tensor-trains using squared inverse Rosenblatt transports

Convolutions Through the Lens of Tensor Networks

Tensor Methods for Generating Compact Uncertainty Quantification and Deep Learning Models

Solving High-dimensional Parametric Elliptic Equation Using Tensor Neural Network

Tensor product approximations of high dimensional potentials

PINNIES: An Efficient Physics-Informed Neural Network Framework to Integral Operator Problems

Functional Tensor Decompositions for Physics-Informed Neural Networks