Abstract:This paper studies a tensor-structured linear regression model with a scalar response variable and tensor-structured predictors, such that the regression parameters form a tensor of order $d$ (i.e., a $d$-fold multiway array) in $\mathbb{R}^{n_1 \times n_2 \times \cdots \times n_d}$. It focuses on the task of estimating the regression tensor from $m$ realizations of the response variable and the predictors where $m\ll n = \prod \nolimits_{i} n_i$. Despite the seeming ill-posedness of this problem, it can still be solved if the parameter tensor belongs to the space of sparse, low Tucker-rank tensors. Accordingly, the estimation procedure is posed as a non-convex optimization program over the space of sparse, low Tucker-rank tensors, and a tensor variant of projected gradient descent is proposed to solve the resulting non-convex problem. In addition, mathematical guarantees are provided that establish the proposed method linearly converges to an appropriate solution under a certain set of conditions. Further, an upper bound on sample complexity of tensor parameter estimation for the model under consideration is characterized for the special case when the individual (scalar) predictors independently draw values from a sub-Gaussian distribution. The sample complexity bound is shown to have a polylogarithmic dependence on $\bar{n} = \max \big\{n_i: i\in \{1,2,\ldots,d \} \big\}$ and, orderwise, it matches the bound one can obtain from a heuristic parameter counting argument. Finally, numerical experiments demonstrate the efficacy of the proposed tensor model and estimation method on a synthetic dataset and a collection of neuroimaging datasets pertaining to attention deficit hyperactivity disorder. Specifically, the proposed method exhibits better sample complexities on both synthetic and real datasets, demonstrating the usefulness of the model and the method in settings where $n \gg m$.

Partially Observed Dynamic Tensor Response Regression

An Augmented Regression Model for Tensors With Missing Values

STORE: Sparse Tensor Response Regression and Neuroimaging Analysis

Reduced-Rank Tensor-on-Tensor Regression and Tensor-variate Analysis of Variance

Decomposable Sparse Tensor on Tensor Regression

Tensor Regression Using Low-rank and Sparse Tucker Decompositions

High-Dimensional Tensor Discriminant Analysis with Incomplete Tensors

Dynamic Tensor Decomposition via Neural Diffusion-Reaction Processes

Beyond Low Rank: A Data-Adaptive Tensor Completion Method

Smooth Compact Tensor Ring Regression

Sparse regularized low-rank tensor regression with applications in genomic data analysis

Low-Rank Tensor Train Coefficient Array Estimation for Tensor-on-Tensor Regression

Incremental Bayesian tensor learning for structural monitoring data imputation and response forecasting

Dynamic Matrix Recovery

Doubly Decomposing Nonparametric Tensor Regression

Covariate-assisted Sparse Tensor Completion

Tensor Neural Controlled Differential Equations

Low-Rank Autoregressive Tensor Completion for Multivariate Time Series Forecasting

An algorithm for online tensor prediction

Nonlinear dynamic transfer partial least squares for domain adaptive regression

Exponential Family Tensor Completion with Auxiliary Information