Abstract:Recent technological advances have led to contemporary applications that demand real-time processing and analysis of sequentially arriving tensor data. Traditional offline learning, involving the storage and utilization of all data in each computational iteration, becomes impractical for high-dimensional tensor data due to its voluminous size. Furthermore, existing low-rank tensor methods lack the capability for statistical inference in an online fashion, which is essential for real-time predictions and informed decision-making. This paper addresses these challenges by introducing a novel online inference framework for low-rank tensor learning. Our approach employs Stochastic Gradient Descent (SGD) to enable efficient real-time data processing without extensive memory requirements, thereby significantly reducing computational demands. We establish a non-asymptotic convergence result for the online low-rank SGD estimator, nearly matches the minimax optimal rate of estimation error in offline models that store all historical data. Building upon this foundation, we propose a simple yet powerful online debiasing approach for sequential statistical inference in low-rank tensor learning. The entire online procedure, covering both estimation and inference, eliminates the need for data splitting or storing historical data, making it suitable for on-the-fly hypothesis testing. Given the sequential nature of our data collection, traditional analyses relying on offline methods and sample splitting are inadequate. In our analysis, we control the sum of constructed super-martingales to ensure estimates along the entire solution path remain within the benign region. Additionally, a novel spectral representation tool is employed to address statistical dependencies among iterative estimates, establishing the desired asymptotic normality.

A fast parallel tensor decomposition with optimal stochastic gradient descent: an application in structural damage identification

Online Tensor-Based Learning for Multi-Way Data

Fast Learnings of Coupled Nonnegative Tensor Decomposition Using Optimal Gradient and Low-rank Approximation

Accelerating regularized tensor decomposition using the alternating direction method of multipliers with multiple Nesterov's extrapolations

CTD: Fast, accurate, and interpretable method for static and dynamic tensor decompositions

Distributed-Memory Randomized Algorithms for Sparse Tensor CP Decomposition

Randomized Online CP Decomposition

Fast and Accurate Dual-Way Streaming PARAFAC2 for Irregular Tensors -- Algorithm and Application

A block-randomized stochastic method with importance sampling for CP tensor decomposition

An algorithm for online tensor prediction

SliceNStitch: Continuous CP Decomposition of Sparse Tensor Streams

An inexactly accelerated algorithm for nonnegative tensor CP decomposition with the column unit constraints

Escaping from Saddle Points --- Online Stochastic Gradient for Tensor Decomposition

CANDECOMP/PARAFAC Decomposition of High-Order Tensors Through Tensor Reshaping

Accelerating Training using Tensor Decomposition

Generalized Higher-Order Tensor Decomposition via Parallel ADMM

Scalable CP Decomposition for Tensor Learning using GPU Tensor Cores

Contextual Tensor Decomposition by Projected Alternating Least Squares

Inertial Accelerated Stochastic Mirror Descent for Large-Scale Generalized Tensor CP Decomposition

Provable Online CP/PARAFAC Decomposition of a Structured Tensor via Dictionary Learning

Online Tensor Inference