Abstract:Inference and simulation in the context of high-dimensional dynamical systems remain computationally challenging problems. Some form of dimensionality reduction is required to make the problem tractable in general. In this paper, we propose a novel approximate Gaussian filtering and smoothing method which propagates low-rank approximations of the covariance matrices. This is accomplished by projecting the Lyapunov equations associated with the prediction step to a manifold of low-rank matrices, which are then solved by a recently developed, numerically stable, dynamical low-rank integrator. Meanwhile, the update steps are made tractable by noting that the covariance update only transforms the column space of the covariance matrix, which is low-rank by construction. The algorithm differentiates itself from existing ensemble-based approaches in that the low-rank approximations of the covariance matrices are deterministic, rather than stochastic. Crucially, this enables the method to reproduce the exact Kalman filter as the low-rank dimension approaches the true dimensionality of the problem. Our method reduces computational complexity from cubic (for the Kalman filter) to \emph{quadratic} in the state-space size in the worst-case, and can achieve \emph{linear} complexity if the state-space model satisfies certain criteria. Through a set of experiments in classical data-assimilation and spatio-temporal regression, we show that the proposed method consistently outperforms the ensemble-based methods in terms of error in the mean and covariance with respect to the exact Kalman filter. This comes at no additional cost in terms of asymptotic computational complexity.

Effective Filtering on a Random Slow Manifold

Approximation of Random Slow Manifolds and Settling of Inertial Particles Under Uncertainty

Data assimilation and parameter estimation for a multiscale stochastic system with alpha-stable Levy noise

Distilling Slow Process Probability Density from Fast Random Data

Strong approximation of nonlinear filtering for multiscale McKean-Vlasov stochastic systems

Weak approximation of nonlinear filtering for multiscale McKean-Vlasov stochastic systems

Infinite dimensional Slow Manifolds for a Linear Fast-Reaction System

The Rank-Reduced Kalman Filter: Approximate Dynamical-Low-Rank Filtering In High Dimensions

A Dynamic Balanced Flow for Filtering Point-Sampled Geometry

Gradient Flows in Filtering and Fisher-Rao Geometry

Fitting a manifold to data in the presence of large noise

Distributed Learning via Filtered Hyperinterpolation on Manifolds

Filtering for Stochastic Systems Driven by Poisson Processes

Coupled Stochastic-Statistical Equations for Filtering Multiscale Turbulent Systems

Dissipativity-Based Filtering of Nonlinear Periodic Markovian Jump Systems: the Discrete-Time Case

Filter Based Methods for Statistical Linear Inverse Problems

Fast Convergence of Langevin Dynamics on Manifold: Geodesics meet Log-Sobolev

Manifold Fitting under Unbounded Noise

Dimension reduction for systems with slow relaxation

Deep Filtering With Adaptive Learning Rates

Direct Method for Yau Filtering System with Nonlinear Observations