Abstract:With the rapid development of deep learning in various fields of science and technology, such as speech recognition, image classification, and natural language processing, recently it is also widely applied in the functional data analysis (FDA) with some empirical success. However, due to the infinite dimensional input, we need a powerful dimension reduction method for functional learning tasks, especially for the nonlinear functional regression. In this paper, based on the idea of smooth kernel integral transformation, we propose a functional deep neural network with an efficient and fully data-dependent dimension reduction method. The architecture of our functional net consists of a kernel embedding step: an integral transformation with a data-dependent smooth kernel; a projection step: a dimension reduction by projection with eigenfunction basis based on the embedding kernel; and finally an expressive deep ReLU neural network for the prediction. The utilization of smooth kernel embedding enables our functional net to be discretization invariant, efficient, and robust to noisy observations, capable of utilizing information in both input functions and responses data, and have a low requirement on the number of discrete points for an unimpaired generalization performance. We conduct theoretical analysis including approximation error and generalization error analysis, and numerical simulations to verify these advantages of our functional net.

Regularized Least Square Regression for Functional Data

Online Regularized Learning Algorithm for Functional Data

Kernel Matrix Learning with a General Regularized Risk Functional Criterion

Support Vector Regression for Functional Data

Functional Linear Regression with Mixed Predictors

Analysis of Regularized Learning for Linear-functional Data in Banach Spaces

Structured functional additive regression in reproducing kernel Hilbert spaces.

Kernel partial least squares regression for functional nonlinear models

Nonlinear functional regression by functional deep neural network with kernel embedding

Least Square Regularized Regression in Sum Space.

Optimal Rates for Functional Linear Regression with General Regularization

Nonparametric Operator-Regularized Covariance Function Estimation for Functional Data

Robust Functional Regression with Discretely Sampled Predictors

Functional Linear Regression: Dependence and Error Contamination

Statistical Optimality of Divide and Conquer Kernel-based Functional Linear Regression

Analysis of Least Squares Regularized Regression in Reproducing Kernel Krein Spaces

Functional Regression with Mode-Sparsity Constraint

Functional Adaptive Huber Linear Regression

Statistical inference for function-on-function linear regression

Spectral algorithms for functional linear regression

Partially Functional Linear Regression in High Dimensions