Abstract:Learning approximations to smooth target functions of many variables from finite sets of pointwise samples is an important task in scientific computing and its many applications in computational science and engineering. Despite well over half a century of research on high-dimensional approximation, this remains a challenging problem. Yet, significant advances have been made in the last decade towards efficient methods for doing this, commencing with so-called sparse polynomial approximation methods and continuing most recently with methods based on Deep Neural Networks (DNNs). In tandem, there have been substantial advances in the relevant approximation theory and analysis of these techniques. In this work, we survey this recent progress. We describe the contemporary motivations for this problem, which stem from parametric models and computational uncertainty quantification; the relevant function classes, namely, classes of infinite-dimensional, Banach-valued, holomorphic functions; fundamental limits of learnability from finite data for these classes; and finally, sparse polynomial and DNN methods for efficiently learning such functions from finite data. For the latter, there is currently a significant gap between the approximation theory of DNNs and the practical performance of deep learning. Aiming to narrow this gap, we develop the topic of practical existence theory, which asserts the existence of dimension-independent DNN architectures and training strategies that achieve provably near-optimal generalization errors in terms of the amount of training data.

Regression and Classification With Spline-Based Separable Expansions

Polynomial spline regression: Theory and Application

Multivariate Lévy Adaptive B-Spline Regression

Estimating Functions and Derivatives Via Adaptive Penalized Splines.

ADDITIVE COEFFICIENT MODELING VIA POLYNOMIAL SPLINE

Semi-Supervised Classification Via Local Spline Regression

B-spline surface fitting by iterative geometric interpolation/approximation algorithms

Multivariate Splines and Their Applications

More efficient approximation of smoothing splines via space-filling basis selection

Sparse and Efficient Estimation for Partial Spline Models with Increasing Dimension

Robust Supervised Spline Embedding

Learning smooth functions in high dimensions: from sparse polynomials to deep neural networks

Spline regression with automatic knot selection

Spline estimator for ultra-high dimensional partially linear varying coefficient models

Approximation of splines in Wasserstein spaces

Extended Object Tracking and Classification based on Linear Splines

Mapping-to-Parameter Nonlinear Functional Regression with Novel B-spline Free Knot Placement Algorithm

Hessian Based Smoothing Splines for Manifold Learning

Gaussian Process Regression under Computational and Epistemic Misspecification

Scalable Higher-Order Tensor Product Spline Models

Approximation of curve-based sleeve functions in high dimensions