A note on the formulation of the Ensemble Adjustment Kalman Filter

Ian Grooms

DOI: https://doi.org/10.48550/arXiv.2006.02941

2020-06-04

Methodology

Abstract:The ensemble adjustment Kalman filter (EAKF; Anderson, 2001) is one of the earliest ensemble square root filters. This note clarifies the correct formulation of the EAKF, which depends on a careful treatment of an eigen-decomposition of one of the matrices involved in the formulation.

What problem does this paper attempt to address?

Commentary: on the Efficiency of Covariance Localisation of the Ensemble Kalman Filter Using Augmented Ensembles

Craig H. Bishop,Jeffrey S. Whitaker,Lili Lei

DOI: https://doi.org/10.3389/fams.2020.00002

2020-01-01

Frontiers in Applied Mathematics and Statistics

Abstract:(i)The predecessor of the GETKF eigenvalue form of the modified gain matrix equation appeared in Bishop (2012, 2014)The spectral shift theorem reduces the differences in the numerical cancellation errors referred to by Farchi and Bouquet. (iii) The eigenvalue form enables Wang et al's (2004) corrections for ensemble rank deficiency. (iv) A proof of the equivalence of the eigenvalue form and Bouquet's form.To be specific about the forms of the modified gain matrix, let (  − − −   = −  − − −   = − x x x x x x Z R x x x x x x Z (1)where K is the total number of ensemble members in the ensemble forecast and where the nvector f ix is the i th member of the prior ensemble forecast and where the p-vector( ) f iH x is the i th member of the prior ensemble forecast of the p-vector y of p observations. When pp. Eq (9) is just an eigenvalue form of the modified gain matrix of Whitaker and Hamill's (2002) Ensemble Square Root Filter.3) Equivalence of (9) and (8) Bouquet's (2016) equation (25) can be derived from (9) with the following steps:(i)Drop the dimension subscripts and manipulate ( ) 1/2 1 − −  −  +   II as follows () ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 1(ii) Use (10) in (9)(iv) Using (12) and (3) in (11) gives ( )( ) ( ) ( ) ( ) ( ) ( ) ( )Equation (13) is equivalent to (8) and Bouquet's (2016) equation (25).Numerical cancellation errors increase when the condition number of the matrix increases. Let
The Ensemble Kalman Filter: theoretical formulation and practical implementation

Geir Evensen

DOI: https://doi.org/10.1007/s10236-003-0036-9

2003-11-01

Ocean Dynamics

Abstract:The purpose of this paper is to provide a comprehensive presentation and interpretation of the Ensemble Kalman Filter (EnKF) and its numerical implementation. The EnKF has a large user group, and numerous publications have discussed applications and theoretical aspects of it. This paper reviews the important results from these studies and also presents new ideas and alternative interpretations which further explain the success of the EnKF. In addition to providing the theoretical framework needed for using the EnKF, there is also a focus on the algorithmic formulation and optimal numerical implementation. A program listing is given for some of the key subroutines. The paper also touches upon specific issues such as the use of nonlinear measurements, in situ profiles of temperature and salinity, and data which are available with high frequency in time. An ensemble based optimal interpolation (EnOI) scheme is presented as a cost-effective approach which may serve as an alternative to the EnKF in some applications. A fairly extensive discussion is devoted to the use of time correlated model errors and the estimation of model bias.

oceanography
An Iterative Ensemble Kalman Filter for Data Assimilation

Gaoming Li,Albert C. Reynolds

DOI: https://doi.org/10.2118/109808-pa

2007-01-01

Abstract:Summary The ensemble Kalman filter (EnKF) is a subject of intensive investigation for use as a reservoir management tool. For strongly nonlinear problems, however, the EnKF can fail to achieve an acceptable data match at certain times in the data assimilation process. Here, we provide two iterative EnKF procedures to remedy this deficiency and explore the validity of these iterative methods compared to the standard EnKF by considering two examples. In both examples, we are able to obtain better data matches using iterative methods than with the standard EnKF. The simplest derivation of the EnKF analysis equation "linearizes" the objective function by adding the vector of predicted data to the original combined state vector of model parameters and dynamical variables. We show that there is no assurance that this trick for turning a nonlinear problem into a linear problem results in a correct sampling of the probability density function (pdf) one wishes to sample. However, we show that augmenting the state vector with the data results in a correct procedure for sampling the pdf if, at every data assimilation step, the predicted data vector is a linear function of the combined (unaugmented) state vector. Without this assumption, we know of no way to show EnKF samples correctly.
Accuracy of the Ensemble Kalman Filter in the Near-Linear Setting

Edoardo Calvello,Pierre Monmarché,Andrew M. Stuart,Urbain Vaes

2024-09-16

Abstract:The filtering distribution captures the statistics of the state of a dynamical system from partial and noisy observations. Classical particle filters provably approximate this distribution in quite general settings; however they behave poorly for high dimensional problems, suffering weight collapse. This issue is circumvented by the ensemble Kalman filter which is an equal-weight interacting particle system. However, this finite particle system is only proven to approximate the true filter in the linear Gaussian case. In practice, however, it is applied in much broader settings; as a result, establishing its approximation properties more generally is important. There has been recent progress in the theoretical analysis of the algorithm, establishing stability and error estimates in non-Gaussian settings, but the assumptions on the dynamics and observation models rule out the unbounded vector fields that arise in practice and the analysis applies only to the mean field limit of the ensemble Kalman filter. The present work establishes error bounds between the filtering distribution and the finite particle ensemble Kalman filter when the model exhibits linear growth.

Statistics Theory,Dynamical Systems,Numerical Analysis,Optimization and Control
Convergence of the Square Root Ensemble Kalman Filter in the Large Ensemble Limit

Evan Kwiatkowski,Jan Mandel

DOI: https://doi.org/10.1137/140965363

2014-10-10

Abstract:Ensemble filters implement sequential Bayesian estimation by representing the probability distribution by an ensemble mean and covariance. Unbiased square root ensemble filters use deterministic algorithms to produce an analysis (posterior) ensemble with prescribed mean and covariance, consistent with the Kalman update. This includes several filters used in practice, such as the Ensemble Transform Kalman Filter (ETKF), the Ensemble Adjustment Kalman Filter (EAKF), and a filter by Whitaker and Hamill. We show that at every time index, as the number of ensemble members increases to infinity, the mean and covariance of an unbiased ensemble square root filter converge to those of the Kalman filter, in the case a linear model and an initial distribution of which all moments exist. The convergence is in $L^{p}$ and the convergence rate does not depend on the model dimension. The result holds in the infinitely dimensional Hilbert space as well.

Statistics Theory
Nonlinear stability and ergodicity of ensemble based Kalman filters

Xin T Tong,Andrew J Majda,David Kelly

DOI: https://doi.org/10.1088/0951-7715/29/2/657

2016-01-25

Nonlinearity

Abstract:The ensemble Kalman filter (EnKF) and ensemble square root filter (ESRF) are data assimilation methods used to combine high dimensional, nonlinear dynamical models with observed data. Despite their widespread usage in climate science and oil reservoir simulation, very little is known about the long-time behavior of these methods and why they are effective when applied with modest ensemble sizes in large dimensional turbulent dynamical systems. By following the basic principles of energy dissipation and controllability of filters, this paper establishes a simple, systematic and rigorous framework for the nonlinear analysis of EnKF and ESRF with arbitrary ensemble size, focusing on the dynamical properties of boundedness and geometric ergodicity. The time uniform boundedness guarantees that the filter estimate will not diverge to machine infinity in finite time, which is a potential threat for EnKF and ESQF known as the catastrophic filter divergence. Geometric ergodicity ensures in addition that the filter has a unique invariant measure and that initialization errors will dissipate exponentially in time. We establish these results by introducing a natural notion of observable energy dissipation. The time uniform bound is achieved through a simple Lyapunov function argument, this result applies to systems with complete observations and strong kinetic energy dissipation, but also to concrete examples with incomplete observations. With the Lyapunov function argument established, the geometric ergodicity is obtained by verifying the controllability of the filter processes; in particular, such analysis for ESQF relies on a careful multivariate perturbation analysis of the covariance eigen-structure.

mathematics, applied,physics, mathematical
A Quick Guide for the Iterated Extended Kalman Filter on Manifolds

Jianzhu Huai,Xiang Gao

2023-10-04

Abstract:The extended Kalman filter (EKF) is a common state estimation method for discrete nonlinear systems. It recursively executes the propagation step as time goes by and the update step when a set of measurements arrives. In the update step, the EKF linearizes the measurement function only once. In contrast, the iterated EKF (IEKF) refines the state in the update step by iteratively solving a least squares problem. The IEKF has been extended to work with state variables on manifolds which have differentiable $\boxplus$ and $\boxminus$ operators, including Lie groups. However, existing descriptions are often long, deep, and even with errors. This note provides a quick reference for the IEKF on manifolds, using freshman-level matrix calculus. Besides the bare-bone equations, we highlight the key steps in deriving them.

Systems and Control
Bayesian Recursive Update for Ensemble Kalman Filters

Kristen Michaelson,Andrey A. Popov,Renato Zanetti

2023-10-28

Abstract:Few real-world systems are amenable to truly Bayesian filtering; nonlinearities and non-Gaussian noises can wreak havoc on filters that rely on linearization and Gaussian uncertainty approximations. This article presents the Bayesian Recursive Update Filter (BRUF), a Kalman filter that uses a recursive approach to incorporate information from nonlinear measurements. The BRUF relaxes the measurement linearity assumption of the Extended Kalman Filter (EKF) by dividing the measurement update into a user-defined number of steps. The proposed technique is extended for ensemble filters in the Bayesian Recursive Update Ensemble Kalman Filter (BRUEnKF). The performance of both filters is demonstrated in numerical examples, and new filters are introduced which exploit the theoretical foundation of the BRUF in different ways. A comparison between the BRUEnKF and Gromov flow, a popular particle flow algorithm, is presented in detail. Finally, the BRUEnKF is shown to outperform the EnKF for a very high-dimensional system.

Numerical Analysis,Computational Engineering, Finance, and Science,Optimization and Control
Analysis of the Ensemble Kalman Filter for Inverse Problems

Claudia Schillings,Andrew M. Stuart

DOI: https://doi.org/10.1137/16m105959x

IF: 3.039

2017-01-01

SIAM Journal on Numerical Analysis

Abstract:The ensemble Kalman filter (EnKF) is a widely used methodology for state estimation in partially, noisily observed dynamical systems and for parameter estimation in inverse problems. Despite its widespread use in the geophysical sciences, and its gradual adoption in many other areas of application, analysis of the method is in its infancy. Furthermore, much of the existing analysis deals with the large ensemble limit, far from the regime in which the method is typically used. The goal of this paper is to analyze the method when applied to inverse problems with fixed ensemble size. A continuous time limit is derived and the long-time behavior of the resulting dynamical system is studied. Most of the rigorous analysis is confined to the linear forward problem, where we demonstrate that the continuous time limit of the EnKF corresponds to a set of gradient flows for the data misfit in each ensemble member, coupled through a common preconditioner which is the empirical covariance matrix of the ensemble. Numerical results demonstrate that the conclusions of the analysis extend beyond the linear inverse problem setting. Numerical experiments are also given which demonstrate the benefits of various extensions of the basic methodology.

mathematics, applied
Notes on Kalman Filter (KF, EKF, ESKF, IEKF, IESKF)

Gyubeom Im

2024-06-28

Abstract:The Kalman Filter (KF) is a powerful mathematical tool widely used for state estimation in various domains, including Simultaneous Localization and Mapping (SLAM). This paper presents an in-depth introduction to the Kalman Filter and explores its several extensions: the Extended Kalman Filter (EKF), the Error-State Kalman Filter (ESKF), the Iterated Extended Kalman Filter (IEKF), and the Iterated Error-State Kalman Filter (IESKF). Each variant is meticulously examined, with detailed derivations of their mathematical formulations and discussions on their respective advantages and limitations. By providing a comprehensive overview of these techniques, this paper aims to offer valuable insights into their applications in SLAM and enhance the understanding of state estimation methodologies in complex environments.

Robotics
Uniform error bounds of the ensemble transform Kalman filter for infinite-dimensional dynamics with multiplicative covariance inflation

Kota Takeda,Takashi Sakajo

2024-02-06

Abstract:Data assimilation is a method of uncertainty quantification to estimate the hidden true state by updating the prediction owing to model dynamics with observation data. As a prediction model, we consider a class of nonlinear dynamical systems on Hilbert spaces including the two-dimensional Navier-Stokes equations and the Lorenz '63 and '96 equations. For nonlinear model dynamics, the ensemble Kalman filter (EnKF) is often used to approximate the mean and covariance of the probability distribution with a set of particles called an ensemble. In this paper, we consider a deterministic version of the EnKF known as the ensemble transform Kalman filter (ETKF), performing well even with limited ensemble sizes in comparison to other stochastic implementations of the EnKF. When the ETKF is applied to large-scale systems, an ad-hoc numerical technique called a covariance inflation is often employed to reduce approximation errors. Despite the practical effectiveness of the ETKF, little is theoretically known. The present study aims to establish the theoretical analysis of the ETKF. We obtain that the estimation error of the ETKF with and without the covariance inflation is bounded for any finite time. In particular, the uniform-in-time error bound is obtained when an inflation parameter is chosen appropriately, justifying the effectiveness of the covariance inflation in the ETKF.

Statistics Theory,Numerical Analysis
Gain Form of the Ensemble Transform Kalman Filter and Its Relevance to Satellite Data Assimilation with Model Space Ensemble Covariance Localization

Craig H. Bishop,Jeffrey S. Whitaker,Lili Lei

DOI: https://doi.org/10.1175/mwr-d-17-0102.1

2017-01-01

Monthly Weather Review

Abstract:To ameliorate suboptimality in ensemble data assimilation, methods have been introduced that involve expanding the ensemble size. Such expansions can incorporate model space covariance localization and/or estimates of climatological or model error covariances. Model space covariance localization in the vertical overcomes problematic aspects of ensemble-based satellite data assimilation. In the case of the ensemble transform Kalman filter (ETKF), the expanded ensemble size associated with vertical covariance localization would also enable the simultaneous update of entire vertical columns of model variables from hyperspectral and multispectral satellite sounders. However, if the original formulation of the ETKF were applied to an expanded ensemble, it would produce an analysis ensemble that was the same size as the expanded forecast ensemble. This article describes a variation on the ETKF called the gain ETKF (GETKF) that takes advantage of covariances from the expanded ensemble, while producing an analysis ensemble that has the required size of the unexpanded forecast ensemble. The approach also yields an inflation factor that depends on the localization length scale that causes the GETKF to perform differently to an ensemble square root filter (EnSRF) using the same expanded ensemble. Experimentation described herein shows that the GETKF outperforms a range of alternative ETKF-based solutions to the aforementioned problems. In cycling data assimilation experiments with a newly developed storm-track version of the Lorenz-96 model, the GETKF analysis root-mean-square error (RMSE) matches the EnSRF RMSE at shorter than optimal localization length scales but is superior in that it yields smaller RMSEs for longer localization length scales.
A Note on the Extended Kalman Filter on a Manifold

Yixiao Ge,Pieter van Goor,Robert Mahony

2023-09-12

Abstract:The kinematics of many control systems, especially in the robotics field, naturally live on smooth manifolds. Most classical state-estimation algorithms, including the extended Kalman filter, are posed on Euclidean space. Although any filter algorithm can be adapted to a manifold setting by implementing it in local coordinates and ignoring the geometric structure, it has always been clear that there would be advantages in taking the geometric structure into consideration in developing the algorithm. In this paper, we argue that the minimum geometric structure required to adapt the extended Kalman filter to a manifold is that of an affine connection. With this structure, we show that a naive coordinate implementation of the EKF fails to account for geometry of the manifold in the update step and in the reset step. We provide geometric modifications to the classical EKF based on parallel transport of the measurement covariance (for the update) and a-posteriori state covariance (for the reset) that address these limitations. Preliminary results for attitude estimation with two directional measurements demonstrate that the proposed modifications significantly improve the transient behavior of the filter.

Systems and Control
A comparison of nonlinear extensions to the ensemble Kalman filter: Gaussian Anamorphosis and Two-Step Ensemble Filters

Ian Grooms

DOI: https://doi.org/10.1007/s10596-022-10141-x

2022-01-29

Abstract:Ensemble Kalman filters are based on a Gaussian assumption, which can limit their performance in some non-Gaussian settings. This paper reviews two nonlinear, non-Gaussian extensions of the Ensemble Kalman Filter: Gaussian anamorphosis (GA) methods and two-step updates, of which the rank histogram filter (RHF) is a prototypical example. GA-EnKF methods apply univariate transforms to the state and observation variables to make their distribution more Gaussian before applying an EnKF. The two-step methods use a scalar Bayesian update for the first step, followed by linear regression for the second step. The connection of the two-step framework to the full Bayesian problem is made, which opens the door to more advanced two-step methods in the full Bayesian setting. A new method for the first part of the two-step framework is proposed, with a similar form to the RHF but a different motivation, called the `improved RHF' (iRHF). A suite of experiments with the Lorenz-`96 model demonstrate situations where the GA-EnKF methods are similar to EnKF, and where they outperform EnKF. The experiments also strongly support the accuracy of the RHF and iRHF filters for nonlinear and non-Gaussian observations; these methods uniformly beat the EnKF and GA-EnKF methods in the experiments reported here. The new iRHF method is only more accurate than RHF at small ensemble sizes in the experiments reported here.

Computation,Methodology
Uniform Error Bounds of the Ensemble Transform Kalman Filter for Chaotic Dynamics with Multiplicative Covariance Inflation

Kota Takeda,Takashi Sakajo

DOI: https://doi.org/10.1137/24m1637192

IF: 2.089

2024-11-13

SIAM/ASA Journal on Uncertainty Quantification

Abstract:SIAM/ASA Journal on Uncertainty Quantification, Volume 12, Issue 4, Page 1315-1335, December 2024. Data assimilation is a method of uncertainty quantification to estimate the hidden true state by updating the prediction owing to model dynamics with observation data. As a prediction model, we consider a class of nonlinear dynamical systems on Hilbert spaces including the two-dimensional Navier–Stokes equations and the Lorenz 63 and 96 equations. For nonlinear model dynamics, the ensemble Kalman filter (EnKF) is often used to approximate the mean and covariance of the probability distribution with a set of particles called an ensemble. In this paper, we consider a deterministic version of the EnKF known as the ensemble transform Kalman filter (ETKF), performing well even with limited ensemble sizes compared to other stochastic implementations of the EnKF. When the ETKF is applied to large-scale systems, an ad-hoc numerical technique called a covariance inflation is often employed to reduce approximation errors. Despite the practical effectiveness of the ETKF, little is theoretically known. The present study aims to establish the theoretical analysis of the ETKF. We obtain that the state estimation error of the ETKF with and without the covariance inflation is bounded for any finite time. Furthermore, when the system is finite-dimensional, we show that the uniform-in-time error bound is obtained when an inflation parameter is chosen appropriately, justifying the effectiveness of the covariance inflation in the ETKF.

mathematics, interdisciplinary applications,physics, mathematical
A comparison of error subspace Kalman filters

LARS NERGER,WOLFGANG HILLER,JENS SCHRÖTER,JENS SCHROTER

DOI: https://doi.org/10.1111/j.1600-0870.2005.00141.x

2005-10-01

Tellus A

Abstract:Three advanced filter algorithms based on the Kalman filter are reviewed and presented in a unified notation. They are the well‐known ensemble Kalman filter (EnKF), the singular evolutive extended Kalman (SEEK) filter, and the less common singular evolutive interpolated Kalman (SEIK) filter. For comparison, the mathematical formulations of the filters are reviewed in relation to the extended Kalman filter as error subspace Kalman filters. The algorithms are presented in their original form and possible variations are discussed. A comparison of the algorithms shows their theoretical capabilities for efficient data assimilation with large‐scale non‐linear systems. In particular, problems of the analysis equations are apparent in the original EnKF algorithm due to the Monte Carlo sampling of ensembles. Theoretically, the SEIK filter appears to be a numerically very efficient algorithm with high potential for use with non‐linear models. The superiority of the SEIK filter is demonstrated on the basis of identical twin experiments using a shallow‐water model with non‐linear evolution. Identical initial conditions for all three filters allow for a consistent comparison of the data assimilation results. These show how choices of particular state ensembles and assimilation schemes lead to significant variations of the filter performance. This is related to different qualities of the predicted error subspaces, as is demonstrated in an examination of the predicted state covariance matrices.
A Reduced Basis Ensemble Kalman Method

Francesco A. B. Silva,Cecilia Pagliantini,Martin Grepl,Karen Veroy

DOI: https://doi.org/10.48550/arXiv.2210.02279

2022-10-05

Abstract:In the process of reproducing the state dynamics of parameter dependent distributed systems, data from physical measurements can be incorporated into the mathematical model to reduce the parameter uncertainty and, consequently, improve the state prediction. Such a Data Assimilation process must deal with the data and model misfit arising from experimental noise as well as model inaccuracies and uncertainties. In this work, we focus on the ensemble Kalman method (EnKM), a particle-based iterative regularization method designed for \textit{a posteriori} analysis of time series. The method is gradient free and, like the ensemble Kalman filter (EnKF), relies on a sample of parameters or particle ensemble to identify the state that better reproduces the physical observations, while preserving the physics of the system as described by the best knowledge model. We consider systems described by parameterized parabolic partial differential equations and employ model order reduction (MOR) techniques to generate surrogate models of different accuracy with uncertain parameters. Their use in combination with the EnKM involves the introduction of the model bias which constitutes a new source of systematic error. To mitigate its impact, an algorithm adjustment is proposed accounting for a prior estimation of the bias in the data. The resulting RB-EnKM is tested in different conditions, including different ensemble sizes and increasing levels of experimental noise. The results are compared to those obtained with the standard EnKF and with the unadjusted algorithm.

Numerical Analysis
The Fundamental Subspaces of Ensemble Kalman Inversion

Elizabeth Qian,Christopher Beattie

2024-09-13

Abstract:Ensemble Kalman Inversion (EKI) methods are a family of iterative methods for solving weighted least-squares problems, especially those arising in scientific and engineering inverse problems in which unknown parameters or states are estimated from observed data by minimizing the weighted square norm of the data misfit. Implementation of EKI requires only evaluation of the forward model mapping the unknown to the data, and does not require derivatives or adjoints of the forward model. The methods therefore offer an attractive alternative to gradient-based optimization approaches in large-scale inverse problems where evaluating derivatives or adjoints of the forward model is computationally intractable. This work presents a new analysis of the behavior of both deterministic and stochastic versions of basic EKI for linear observation operators, resulting in a natural interpretation of EKI's convergence properties in terms of ``fundamental subspaces'' analogous to Strang's fundamental subspaces of linear algebra. Our analysis directly examines the discrete EKI iterations instead of their continuous-time limits considered in previous analyses, and provides spectral decompositions that define six fundamental subspaces of EKI spanning both observation and state spaces. This approach verifies convergence rates previously derived for continuous-time limits, and yields new results describing both deterministic and stochastic EKI convergence behavior with respect to the standard minimum-norm weighted least squares solution in terms of the fundamental subspaces. Numerical experiments illustrate our theoretical results.

Numerical Analysis
On the Mathematical Theory of Ensemble (Linear-Gaussian) Kalman-Bucy Filtering

Adrian N. Bishop,Pierre Del Moral

DOI: https://doi.org/10.1007/s00498-023-00357-2

2023-06-15

Abstract:The purpose of this review is to present a comprehensive overview of the theory of ensemble Kalman-Bucy filtering for continuous-time, linear-Gaussian signal and observation models. We present a system of equations that describe the flow of individual particles and the flow of the sample covariance and the sample mean in continuous-time ensemble filtering. We consider these equations and their characteristics in a number of popular ensemble Kalman filtering variants. Given these equations, we study their asymptotic convergence to the optimal Bayesian filter. We also study in detail some non-asymptotic time-uniform fluctuation, stability, and contraction results on the sample covariance and sample mean (or sample error track). We focus on testable signal/observation model conditions, and we accommodate fully unstable (latent) signal models. We discuss the relevance and importance of these results in characterising the filter's behaviour, e.g. it's signal tracking performance, and we contrast these results with those in classical studies of stability in Kalman-Bucy <a class="link-external link-http" href="http://filtering.We" rel="external noopener nofollow">this http URL</a> also provide a novel (and negative) result proving that the bootstrap particle filter cannot track even the most basic unstable latent signal, in contrast with the ensemble Kalman filter (and the optimal filter). We provide intuition for how the main results extend to nonlinear signal models and comment on their consequence on some typical filter behaviours seen in practice, e.g. catastrophic divergence.

Statistics Theory,Optimization and Control,Probability
Redesigning the ensemble Kalman filter with a dedicated model of epistemic uncertainty

Chatchuea Kimchaiwong,Jeremie Houssineau,Adam M. Johansen

2024-11-28

Abstract:The problem of incorporating information from observations received serially in time is widespread in the field of uncertainty quantification. Within a probabilistic framework, such problems can be addressed using standard filtering techniques. However, in many real-world problems, some (or all) of the uncertainty is epistemic, arising from a lack of knowledge, and is difficult to model probabilistically. This paper introduces a possibilistic ensemble Kalman filter designed for this setting and characterizes some of its properties. Using possibility theory to describe epistemic uncertainty is appealing from a philosophical perspective, and it is easy to justify certain heuristics often employed in standard ensemble Kalman filters as principled approaches to capturing uncertainty within it. The possibilistic approach motivates a robust mechanism for characterizing uncertainty which shows good performance with small sample sizes, and can outperform standard ensemble Kalman filters at given sample size, even when dealing with genuinely aleatoric uncertainty.

Methodology,Artificial Intelligence

A note on the formulation of the Ensemble Adjustment Kalman Filter

Commentary: on the Efficiency of Covariance Localisation of the Ensemble Kalman Filter Using Augmented Ensembles

The Ensemble Kalman Filter: theoretical formulation and practical implementation

An Iterative Ensemble Kalman Filter for Data Assimilation

Accuracy of the Ensemble Kalman Filter in the Near-Linear Setting

Convergence of the Square Root Ensemble Kalman Filter in the Large Ensemble Limit

Nonlinear stability and ergodicity of ensemble based Kalman filters

A Quick Guide for the Iterated Extended Kalman Filter on Manifolds

Bayesian Recursive Update for Ensemble Kalman Filters

Analysis of the Ensemble Kalman Filter for Inverse Problems

Notes on Kalman Filter (KF, EKF, ESKF, IEKF, IESKF)

Uniform error bounds of the ensemble transform Kalman filter for infinite-dimensional dynamics with multiplicative covariance inflation

Gain Form of the Ensemble Transform Kalman Filter and Its Relevance to Satellite Data Assimilation with Model Space Ensemble Covariance Localization

A Note on the Extended Kalman Filter on a Manifold

A comparison of nonlinear extensions to the ensemble Kalman filter: Gaussian Anamorphosis and Two-Step Ensemble Filters

Uniform Error Bounds of the Ensemble Transform Kalman Filter for Chaotic Dynamics with Multiplicative Covariance Inflation

A comparison of error subspace Kalman filters

A Reduced Basis Ensemble Kalman Method

The Fundamental Subspaces of Ensemble Kalman Inversion

On the Mathematical Theory of Ensemble (Linear-Gaussian) Kalman-Bucy Filtering

Redesigning the ensemble Kalman filter with a dedicated model of epistemic uncertainty