Abstract:The past decade has seen increasing interest in applying Deep Learning (DL) to Computational Science and Engineering (CSE). Driven by impressive results in applications such as computer vision, Uncertainty Quantification (UQ), genetics, simulations and image processing, DL is increasingly supplanting classical algorithms, and seems poised to revolutionize scientific computing. However, DL is not yet well-understood from the standpoint of numerical analysis. Little is known about the efficiency and reliability of DL from the perspectives of stability, robustness, accuracy, and sample complexity. In particular, approximating solutions to parametric PDEs is an objective of UQ for CSE. Training data for such problems is often scarce and corrupted by errors. Moreover, the target function is a possibly infinite-dimensional smooth function taking values in the PDE solution space, generally an infinite-dimensional Banach space. This paper provides arguments for Deep Neural Network (DNN) approximation of such functions, with both known and unknown parametric dependence, that overcome the curse of dimensionality. We establish practical existence theorems that describe classes of DNNs with dimension-independent architecture size and training procedures based on minimizing the (regularized) $\ell^2$-loss which achieve near-optimal algebraic rates of convergence. These results involve key extensions of compressed sensing for Banach-valued recovery and polynomial emulation with DNNs. When approximating solutions of parametric PDEs, our results account for all sources of error, i.e., sampling, optimization, approximation and physical discretization, and allow for training high-fidelity DNN approximations from coarse-grained sample data. Our theoretical results fall into the category of non-intrusive methods, providing a theoretical alternative to classical methods for high-dimensional approximation.

Deep Component Analysis via Alternating Direction Neural Networks

DDPG-Driven Deep-Unfolding with Adaptive Depth for Channel Estimation with Sparse Bayesian Learning

Component Decomposition Analysis for Hyperspectral Anomaly Detection

Deep Convolutional Compressed Sensing for LiDAR Depth Completion

Deep Anchored Convolutional Neural Networks

Deep Vanishing Component Analysis Network for Pattern Classification

Toward Deep Adaptive Hinging Hyperplanes

Component-based Sketching for Deep ReLU Nets

Modern Neighborhood Components Analysis: A Deep Tabular Baseline Two Decades Later

Towards Better Analysis of Deep Convolutional Neural Networks

Scalable and Effective Deep CCA Via Soft Decorrelation

> ? ∗ > 0 B ? ∗ > 0 C ? ∗ > 0 DEC Conv = Full-image Encoder Conv Conv Conv Conv Conv Conv Convs ASPP # Dense Feature Extractor Scene Understanding Modular Ordinal Regression Input Output

Deep Deterministic Independent Component Analysis for Hyperspectral Unmixing

Deciphering the Feature Representation of Deep Neural Networks for High-Performance AI

Make Deep Networks Shallow Again

Reframing Neural Networks: Deep Structure in Overcomplete Representations

Deep Contrastive Principal Component Analysis Adaptive to Nonlinear Data

A Compromise Principle in Deep Monocular Depth Estimation

Understand the Effectiveness of Shortcuts through the Lens of DCA

Dissecting Deep Neural Networks

Near-optimal learning of Banach-valued, high-dimensional functions via deep neural networks