Abstract:The state-of-the-art tensor network Kalman filter lifts the curse of dimensionality for high-dimensional recursive estimation problems. However, the required rounding operation can cause filter divergence due to the loss of positive definiteness of covariance matrices. We solve this issue by developing, for the first time, a tensor network square root Kalman filter, and apply it to high-dimensional online Gaussian process regression. In our experiments, we demonstrate that our method is equivalent to the conventional Kalman filter when choosing a full-rank tensor network. Furthermore, we apply our method to a real-life system identification problem where we estimate $4^{14}$ parameters on a standard laptop. The estimated model outperforms the state-of-the-art tensor network Kalman filter in terms of prediction accuracy and uncertainty quantification.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the problem of filter divergence in the existing Tensor Network Kalman Filter (TNKF) due to the loss of positive definiteness during the covariance matrix update process in high - dimensional recursive estimation problems. Specifically: 1. **The curse of dimensionality in high - dimensional recursive estimation**: Traditional Kalman filters encounter the "curse of dimensionality" when dealing with high - dimensional data, that is, as the input dimension increases, the computational complexity grows exponentially. The existing Tensor Network Kalman Filter (TNKF) has partially solved this problem by using the Tensor Network (TN) method, but there is still a significant defect. 2. **The positive definiteness problem of covariance matrix update**: TNKF needs to perform specific Tensor Network (TN) rounding operations when updating the covariance matrix, which may lead to the loss of positive definiteness of the covariance matrix and thus cause the filter to diverge. Positive definiteness is one of the key conditions for ensuring the stability of the Kalman filter. To solve the above problems, the paper proposes a new method - Tensor Network Square - Root Kalman Filter (TNSRKF). TNSRKF avoids the problem of losing positive definiteness during the covariance matrix update process by calculating the square - root factor of the covariance matrix in the Tensor Train format (TT format). Specific improvements include: - **Square - root form of Kalman filter**: TNSRKF adopts the square - root form of Kalman filter (SRKF) to recursively update the posterior distribution of parameters at each time step. - **Tensor network representation**: Use tensor networks (especially Tensor Train matrices, TTm) to represent the mean of the weight vector and the square - root factor of the covariance matrix, thereby achieving linear computational complexity. - **SVD algorithm for QR decomposition**: In order to keep the rank of the tensor network unchanged, TNSRKF uses the SVD - based QR decomposition algorithm to restore the square - root factor of the covariance matrix to its original size after each update. Through these improvements, TNSRKF not only solves the positive definiteness problem in the covariance matrix update process but also can provide more accurate predictions and uncertainty quantification in high - dimensional online Gaussian process regression tasks. Experimental results show that TNSRKF is equivalent to the traditional Kalman filter in the full - rank setting and performs better than the existing TNKF method in practical system identification problems.

Tensor network square root Kalman filter for online Gaussian process regression

Tensor Network Kalman Filtering for Large-Scale LS-SVMs

The Square-Root Unscented and the Square-Root Cubature Kalman Filters on Manifolds

Square-root filtering via covariance SVD factors in the accurate continuous-discrete extended-cubature Kalman filter

Adaptive square-root unscented Kalman filter algorithm

Matrix output extension of the tensor network Kalman filter with an application in MIMO Volterra system identification

A Square-Root Kalman Filter Using Only QR Decompositions

Square root unscented Kalman filter incorporating Gaussian process regression

Ensemble Kalman Filtering for Online Gaussian Process Regression and Learning

Maximum Correntropy Square-Root Cubature Kalman Filter with Application to SINS/GPS Integrated Systems.

An algorithm for online tensor prediction

Solving high-dimensional nonlinear filtering problems using a tensor train decomposition method

Nonlinear Kalman Filtering with Reparametrization Gradients

Constructing numerically stable Kalman filter-based algorithms for gradient-based adaptive filtering

The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions

Solving Nonlinear Filtering Problems Using a Tensor Train Decomposition Method

Square Root Cubature Kalman Filter Algorithm and Application

MATLAB-based general approach for square-root extended-unscented and fifth-degree cubature Kalman filtering methods

Improved Square Root Unscented Kalman Filter and Its Application in Rendezvous and Docking

Traffic matrix estimation based on a square root Kalman filtering algorithm

Computation-Aware Kalman Filtering and Smoothing