An Efficient Approach to Regression Problems with Tensor Neural Networks

Yongxin Li,Yifan Wang,Zhongshuo Lin,Hehu Xie

2024-09-13

Abstract:This paper introduces a tensor neural network (TNN) to address nonparametric regression problems, leveraging its distinct sub-network structure to effectively facilitate variable separation and enhance the approximation of complex, high-dimensional functions. The TNN demonstrates superior performance compared to conventional Feed-Forward Networks (FFN) and Radial Basis Function Networks (RBN) in terms of both approximation accuracy and generalization capacity, even with a comparable number of parameters. A significant innovation in our approach is the integration of statistical regression and numerical integration within the TNN framework. This allows for efficient computation of high-dimensional integrals associated with the regression function and provides detailed insights into the underlying data structure. Furthermore, we employ gradient and Laplacian analysis on the regression outputs to identify key dimensions influencing the predictions, thereby guiding the design of subsequent experiments. These advancements make TNN a powerful tool for applications requiring precise high-dimensional data analysis and predictive modeling.

Machine Learning

What problem does this paper attempt to address?

The problem this paper attempts to address is the high-dimensional nonparametric regression problem. Specifically, the authors introduce a Tensor Neural Network (TNN) to tackle this issue, leveraging its unique sub-network structure to effectively achieve variable separation and enhance the approximation capability for complex high-dimensional functions. Compared to traditional Feedforward Neural Networks (FFN) and Radial Basis Function Networks (RBN), TNN demonstrates superior approximation accuracy and generalization ability, even with a comparable number of parameters. The main innovations of the paper include: 1. **Combination of Statistical Regression and Numerical Integration**: The integration of statistical regression and numerical integration techniques within the TNN framework allows for efficient computation of high-dimensional integrals related to the regression function and provides detailed insights into the data structure. 2. **Gradient and Laplacian Analysis**: By analyzing the gradient and Laplacian operator of the regression output, the key dimensions affecting the predictions are identified, thereby guiding subsequent experimental design. These innovations make TNN a powerful tool for applications requiring precise high-dimensional data analysis and predictive modeling.

An Efficient Approach to Regression Problems with Tensor Neural Networks

Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression

Stable tensor neural networks for efficient deep learning

Tensor Neural Network Interpolation and Its Applications

Stable Tensor Neural Networks for Rapid Deep Learning

Variational Tensor Neural Networks for Deep Learning

Tensor Neural Network and Its Numerical Integration

Factor Augmented Tensor-on-Tensor Neural Networks

Quantized Tensor Neural Network

An Augmented Regression Model for Tensors With Missing Values

Computational and Statistical Guarantees for Tensor-on-Tensor Regression with Tensor Train Decomposition

Performance Analysis for Tensor-Train Decomposition to Deep Neural Network Based Vector-to-Vector Regression

High-dimensional Tensor Response Regression using the t-Distribution

Kernel learning with tensor networks

Robust Supervised Learning Based on Tensor Network Method

Residual Tensor Train: a Flexible and Efficient Approach for Learning Multiple Multilinear Correlations

Efficient Decision Trees for Tensor Regressions

Twin Neural Network Regression is a Semi-Supervised Regression Algorithm

TT-SNN: Tensor Train Decomposition for Efficient Spiking Neural Network Training

Solving High-dimensional Parametric Elliptic Equation Using Tensor Neural Network