Abstract:Practical Bayes filters often assume the state distribution of each time step to be Gaussian for computational tractability, resulting in the so-called Gaussian filters. When facing nonlinear systems, Gaussian filters such as extended Kalman filter (EKF) or unscented Kalman filter (UKF) typically rely on certain linearization techniques, which can introduce large estimation errors. To address this issue, this paper reconstructs the prediction and update steps of Gaussian filtering as solutions to two distinct optimization problems, whose optimal conditions are found to have analytical forms from Stein's lemma. It is observed that the stationary point for the prediction step requires calculating the first two moments of the prior distribution, which is equivalent to that step in existing moment-matching filters. In the update step, instead of linearizing the model to approximate the stationary points, we propose an iterative approach to directly minimize the update step's objective to avoid linearization errors. For the purpose of performing the steepest descent on the Gaussian manifold, we derive its natural gradient that leverages Fisher information matrix to adjust the gradient direction, accounting for the curvature of the parameter space. Combining this update step with moment matching in the prediction step, we introduce a new iterative filter for nonlinear systems called \textit{N}atural Gr\textit{a}dient Gaussia\textit{n} Appr\textit{o}ximation filter, or NANO filter for short. We prove that NANO filter locally converges to the optimal Gaussian approximation at each time step. Furthermore, the estimation error is proven exponentially bounded for nearly linear measurement equation and low noise levels through constructing a supermartingale-like property across consecutive time steps.

A novel nonlinear optimization method for fitting a noisy Gaussian activation function

Nonlinear semi-parametric modeling method based on GA-ANN

Stochastic Behavior of the Nonnegative Least Mean Fourth Algorithm for Stationary Gaussian Inputs and Slow Learning

Heuristic Search for Activation Functions of Neural Networks Based on Gaussian Processes.

New Noise-Tolerant Neural Algorithms for Future Dynamic Nonlinear Optimization with Estimation on Hessian Matrix Inversion

A gradient-type noise-tolerant finite-time neural network for convex optimization

Identification of Nonlinear System Based on Fuzzy Model with Enhanced Gradient Search

A Gaussian approximation filter for nonlinear systems with compound noises

Neural network with optimal neuron activation functions based on additive Gaussian process regression

Nonlinear Bayesian Filtering with Natural Gradient Gaussian Approximation

Self-adaptive Robust Nonlinear Regression for Unknown Noise Via Mixture of Gaussians

Feasibility study on the least square method for fitting non-Gaussian noise data

Neural Dynamics and Newton–Raphson Iteration for Nonlinear Optimization

Random Weighting-Based Nonlinear Gaussian Filtering

A New Chaotic Neural Network Algorithm for “0-1” Optimization Problems Based Non-monotonous Activation Function

Adaptive Nonlinear Neuro-Controller with an Integrated Evaluation Algorithm for Nonlinear Active Noise Systems

GSdyn: Learning training dynamics via online Gaussian optimization with gradient states

The limitation of neural nets for approximation and optimization

An Adaptive Stochastic Gradient Method with Non-negative Gauss-Newton Stepsizes

A Novel Maximum Likelihood-Based Stochastic Gradient Algorithm for Hammerstein Nonlinear Systems with Coloured Noise.