New Truncated Error Estimate for Fast Gauss Transform

JIANG Hong,ZHU Wen-qiu,HU Yong-xiang

DOI: https://doi.org/10.3778/j.issn.1002-8331.2008.25.010

2008-01-01

Computer Engineering and Applications Journal

Abstract:A new truncated error estimate for fast gauss transform is proved using stirling formulas.Analysis show the new error is smaller and more accurate than the error X.Wan and G.E.Karniadakis presented.

What problem does this paper attempt to address?

Fast Gaussian particle filtering algorithm

Peng Chen,Hui Qian,Miaoliang Zhu

2008-01-01

Abstract:A fast Gaussian particle filtering (GPF) algorithm without sampling step is presented to improve the real-time performance of GPF algorithm. The new estimator yields better performance by replacing Gaussian sampling with linear transformation. The distribution function represented by the particles of these two algorithms is identical and the computing time of linear transformation is less than that of Gaussian sampling. Particle filtering (PF) algorithm, GPF algorithm, and improved GPF algorithm are applied to two numerical examples respectively, one is a one-dimensional discrete-time nonlinear time-varying model, the other is a two-dimensional bearing only tracking (BOT) model. Simulation results are presented to demonstrate the lower complexity of the new GPF algorithm over conventional GPF algorithm and PF algorithm. Improved GPF algorithm can generate 1000 particles in 22 ms, 6 ms less than that of GPF algorithm with the same estimation accuracy.
A new version of the adaptive fast Gauss transform for discrete and continuous sources

Leslie Greengard,Shidong Jiang,Manas Rachh,Jun Wang

2023-05-12

Abstract:We present a new version of the fast Gauss transform (FGT) for discrete and continuous sources. Classical Hermite expansions are avoided entirely, making use only of the plane-wave representation of the Gaussian kernel and a new hierarchical merging scheme. For continuous source distributions sampled on adaptive tensor-product grids, we exploit the separable structure of the Gaussian kernel to accelerate the computation. For discrete sources, the scheme relies on the nonuniform fast Fourier transform (NUFFT) to construct near field plane wave representations. The scheme has been implemented for either free-space or periodic boundary conditions. In many regimes, the speed is comparable to or better than that of the conventional FFT in work per gridpoint, despite being fully adaptive.

Numerical Analysis
Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

Chong Li,Nuchun Hu,Jinhua Wang

DOI: https://doi.org/10.1016/j.jco.2010.02.001

IF: 1.333

2010-01-01

Journal of Complexity

Abstract:The notions of Lipschitz conditions with L average are introduced to the study of convergence analysis of Gauss–Newton’s method for singular systems of equations. Unified convergence criteria ensuring the convergence of Gauss–Newton’s method for one kind of singular systems of equations with constant rank derivatives are established and unified estimates of radii of convergence balls are also obtained. Applications to some special cases such as the Kantorovich type conditions, γ-conditions and the Smale point estimate theory are provided and some important known results are extended and/or improved.
Approximate Gauss–Newton methods for solving underdetermined nonlinear least squares problems

Ji-Feng Bao,Chong Li,Wei-Ping Shen,Jen-Chih Yao,Sy-Ming Guu

DOI: https://doi.org/10.1016/j.apnum.2016.08.007

IF: 2.994

2017-01-01

Applied Numerical Mathematics

Abstract:We propose several approximate Gauss–Newton methods, i.e., the truncated, perturbed, and truncated-perturbed GN methods, for solving underdetermined nonlinear least squares problems. Under the assumption that the Fréchet derivatives are Lipschitz continuous and of full row rank, Kantorovich-type convergence criteria of the truncated GN method are established and local convergence theorems are presented with the radii of convergence balls obtained. As consequences of the convergence results for the truncated GN method, convergence theorems of the perturbed and truncated-perturbed GN methods are also presented. Finally, numerical experiments are presented where the comparisons with the standard inexact Gauss–Newton method and the inexact trust-region method for bound-constrained least squares problems [23] are made.
Aggressively Truncated Taylor Series Method For Accurate Computation Of Exponentials Of Essentially Nonnegative Matrices

Meiyue Shao,Weiguo Gao,Jungong Xue

DOI: https://doi.org/10.1137/120894294

IF: 1.908

2014-01-01

SIAM Journal on Matrix Analysis and Applications

Abstract:Small relative perturbations to the entries of an essentially nonnegative matrix introduce small relative errors to entries of its exponential. It is thus desirable to compute the exponential with high componentwise relative accuracy. Taylor series approximation coupled with scaling and squaring is used to compute the exponential of an essentially nonnegative matrix. An a priori componentwise relative error bound of truncation is established, from which one can choose the degree of Taylor series expansion and the scale factor so that the exponential is computed with desired componentwise relative accuracy. To reduce the computational cost, the degree of the Taylor series expansion is chosen small, while the scale factor is chosen sufficiently large to achieve the desired accuracy. The rounding errors in the squaring stage are not serious as squaring is forward stable for nonnegative matrices. We also establish a posteriori componentwise error bounds and derive a novel interval algorithm for the matrix exponential of an essentially nonnegative matrix. Rounding error analysis and numerical experiments demonstrate the efficiency and accuracy of the proposed methods.
Smale's α-theory for inexact Newton methods under the γ-condition

Weiping Shen,Chong Li

DOI: https://doi.org/10.1016/j.jmaa.2010.01.056

IF: 1.417

2010-01-01

Journal of Mathematical Analysis and Applications

Abstract:The present paper is concerned with the convergence problem of inexact Newton methods. Assuming that the nonlinear operator satisfies the γ-condition, a convergence criterion for inexact Newton methods is established which includes Smale's type convergence criterion. The concept of an approximate zero for inexact Newton methods is proposed in this paper and the criterion for judging an initial point being an approximate zero is established. Consequently, Smale's α-theory is generalized to inexact Newton methods. Furthermore, a numerical example is presented to illustrate the applicability of our main results.
Improved Fast Fractional-Fourier-transform Algorithm.

XP Yang,QF Tan,XF Wei,Y Xiang,YB Yan,GF Jin

DOI: https://doi.org/10.1364/josaa.21.001677

2004-01-01

Abstract:Through the optimization of the main interval of the fractional order, an improved fast algorithm for numerical calculation of the fractional Fourier transforms is proposed. With this improved algorithm, the fractional Fourier transforms of a rectangular function and a Gaussian function are calculated. Its calculation errors are compared with those calculated with the previously published algorithm, and the results show that the calculation accuracy of the improved algorithm is much higher.
A New Recursive Bounded GM Estimator for Regression

Li-hua CHENG,Jun-min ZHANG

DOI: https://doi.org/10.11830/ISSN.1000-5013.2015.03.0359

2015-01-01

Abstract:A new recursive bounded GM estimator for regression is proposed.Unlike other GM estimators,the new esti-mator is based on one more general framework and uses a cost function with bounded M-estimate function.The new esti-mator,in effect,is a recursive one-step iteration solution of the"normal equations"corresponding to the cost function.In the new estimator,a weight function is designed to reject or to reduce the influence of the outliers.Furthermore,by in-troducing an augment variable,the proposed estimator is modified to a very adaptive version for the estimation of autore-gressive parameters.The simulation results show that both the proposed estimator and its modification are more effective than other related estimators in suppressing the adverse influence of outliers;the proposed estimator,with the same set-tings,can keep a high accuracy and stable convergence performance in a variety of non-stationary environments.
Generalized Neumann Expansion and Its Application in Stochastic Finite Element Methods

Xiangyu Wang,Song Cen,Chenfeng Li

DOI: https://doi.org/10.1155/2013/325025

IF: 1.43

2013-01-01

Mathematical Problems in Engineering

Abstract:An acceleration technique, termed generalized Neumann expansion (GNE), is presented for evaluating the responses of uncertain systems. The GNE method, which solves stochastic linear algebraic equations arising in stochastic finite element analysis, is easy to implement and is of high efficiency. The convergence condition of the new method is studied, and a rigorous error estimator is proposed to evaluate the upper bound of the relative error of a given GNE solution. It is found that the third-order GNE solution is sufficient to achieve a good accuracy even when the variation of the source stochastic field is relatively high. The relationship between the GNE method, the perturbation method, and the standard Neumann expansion method is also discussed. Based on the links between these three methods, quantitative error estimations for the perturbation method and the standard Neumann method are obtained for the first time in the probability context.
ON CONVERGENCE OF A TRUNCATED GAUSS-NEWTON METHOD FOR SOLVING UNDERDETERMINED NONLINEAR LEAST SQUARES PROBLEMS

Ji-Feng Bao,Sy-Ming Guu,Jinhua Wang,Yaohua Hu,Chong Li

2018-01-01

Journal of nonlinear and convex analysis

Abstract:We consider a truncated Gauss-Newton method for solving nonlinear least squares problems (NLSP) for the underdetermined case. Under some mild conditions, the method converges to a solution at rate of nu when the involved parameter nu in the truncated method satisfies nu is an element of(1, 2], and superlinearly when nu = 1 and theta(k) -> 0. It should be remarked that our techniques for convergence analysis are quite different from that used in.
Faster stochastic trace estimation with a Chebyshev product identity

Eric Hallman

DOI: https://doi.org/10.1016/j.aml.2021.107246

IF: 4.294

2021-10-01

Applied Mathematics Letters

Abstract:Methods for stochastic trace estimation often require the repeated evaluation of expressions of the form <math>zTpn(A)z</math>, where <math>A</math> is a symmetric matrix and <math>pn</math> is a degree <math>n</math> polynomial written in the standard or Chebyshev basis. We show how to evaluate these expressions using only <math>⌈n∕2⌉</math> matrix–vector products, thus substantially reducing the cost of existing trace estimation algorithms that use Chebyshev interpolation or Taylor series.

mathematics, applied
A Dynamic Low-Rank Fast Gaussian Transform

Baihe Huang,Zhao Song,Omri Weinstein,Junze Yin,Hengjie Zhang,Ruizhe Zhang

2024-02-06

Abstract:The \emph{Fast Gaussian Transform} (FGT) enables subquadratic-time multiplication of an $n\times n$ Gaussian kernel matrix $\mathsf{K}_{i,j}= \exp ( - \| x_i - x_j \|_2^2 ) $ with an arbitrary vector $h \in \mathbb{R}^n$, where $x_1,\dots, x_n \in \mathbb{R}^d$ are a set of \emph{fixed} source points. This kernel plays a central role in machine learning and random feature maps. Nevertheless, in most modern data analysis applications, datasets are dynamically changing (yet often have low rank), and recomputing the FGT from scratch in (kernel-based) algorithms incurs a major computational overhead ($\gtrsim n$ time for a single source update $\in \mathbb{R}^d$). These applications motivate a \emph{dynamic FGT} algorithm, which maintains a dynamic set of sources under \emph{kernel-density estimation} (KDE) queries in \emph{sublinear time} while retaining Mat-Vec multiplication accuracy and speed. Assuming the dynamic data-points $x_i$ lie in a (possibly changing) $k$-dimensional subspace ($k\leq d$), our main result is an efficient dynamic FGT algorithm, supporting the following operations in $\log^{O(k)}(n/\varepsilon)$ time: (1) Adding or deleting a source point, and (2) Estimating the ``kernel-density'' of a query point with respect to sources with $\varepsilon$ additive accuracy. The core of the algorithm is a dynamic data structure for maintaining the \emph{projected} ``interaction rank'' between source and target boxes, decoupled into finite truncation of Taylor and Hermite expansions.

Data Structures and Algorithms
Fast simulation of truncated Gaussian distributions

Nicolas Chopin

DOI: https://doi.org/10.1007/s11222-009-9168-1

2012-01-30

Abstract:We consider the problem of simulating a Gaussian vector X, conditional on the fact that each component of X belongs to a finite interval [a_i,b_i], or a semi-finite interval [a_i,+infty). In the one-dimensional case, we design a table-based algorithm that is computationally faster than alternative algorithms. In the two-dimensional case, we design an accept-reject algorithm. According to our calculations and our numerical studies, the acceptance rate of this algorithm is bounded from below by 0.5 for semi-finite truncation intervals, and by 0.47 for finite intervals. Extension to 3 or more dimensions is discussed.

Computation
An adaptive fast Gauss transform in two dimensions

Jun Wang,Leslie Greengard

DOI: https://doi.org/10.48550/arXiv.1712.00380

2017-12-01

Abstract:A variety of problems in computational physics and engineering require the convolution of the heat kernel (a Gaussian) with either discrete sources, densities supported on boundaries, or continuous volume distributions. We present a unified fast Gauss transform for this purpose in two dimensions, making use of an adaptive quad-tree discretization on a unit square which is assumed to contain all sources. Our implementation permits either free-space or periodic boundary conditions to be imposed, and is efficient for any choice of variance in the Gaussian.

Numerical Analysis
An Inexact Shift‐and‐invert Arnoldi Algorithm for Toeplitz Matrix Exponential

Gang Wu,Ting-ting Feng,Yimin Wei

DOI: https://doi.org/10.1002/nla.1992

2015-01-01

Numerical Linear Algebra with Applications

Abstract:We revisit the shift‐and‐invert Arnoldi method proposed in [S. Lee, H. Pang, and H. Sun. Shift‐invert Arnoldi approximation to the Toeplitz matrix exponential, SIAM J. Sci. Comput., 32: 774–792, 2010] for numerical approximation to the product of Toeplitz matrix exponential with a vector. In this approach, one has to solve two large‐scale Toeplitz linear systems in advance. However, if the desired accuracy is high, the cost will be prohibitive. Therefore, it is interesting to investigate how to solve the Toeplitz systems inexactly in this method. The contribution of this paper is in three regards. First, we give a new stability analysis on the Gohberg–Semencul formula (GSF) and define the GSF condition number of a Toeplitz matrix. It is shown that when the size of the Toeplitz matrix is large, our result is sharper than the one given in [M. Gutknecht and M. Hochbruck. The stability of inversion formulas for Toeplitz matrices, Linear Algebra Appl., 223/224: 307–324, 1995]. Second, we establish a relation between the error of Toeplitz systems and the residual of Toeplitz matrix exponential. We show that if the GSF condition number of the Toeplitz matrix is medium‐sized, then the Toeplitz systems can be solved in a low accuracy. Third, based on this relationship, we present a practical stopping criterion for relaxing the accuracy of the Toeplitz systems and propose an inexact shift‐and‐invert Arnoldi algorithm for the Toeplitz matrix exponential problem. Numerical experiments illustrate the numerical behavior of the new algorithm and show the effectiveness of our theoretical results. Copyright © 2015 John Wiley & Sons, Ltd.
An Improved Error Bound for Gaussian Interpolation

Lin-Tian Luh

DOI: https://doi.org/10.48550/arXiv.0712.0863

2007-12-06

Abstract:An error bound for Gaussian Interpolation which is better than the current exponential-type error bound is presented.

Numerical Analysis
The a Priori and a Posteriori Error Estimates of DG Method for the Steklov Eigenvalue Problem in Inverse Scattering

Li Yanjun,Bi Hai,Yang Yidu

DOI: https://doi.org/10.1007/s10915-022-01787-x

2022-01-01

Journal of Scientific Computing

Abstract:In this paper, we study the discontinuous Galerkin finite element method for the Steklov eigenvalue problem arising in inverse scattering. We present a complete error estimates including the a refined priori error estimate and the a posteriori error estimate, and prove the reliability and efficiency of the a posteriori error estimators for eigenfunctions up to higher order terms, and we also analyze the reliability of estimators for eigenvalues. Moreover, we carry out the numerical experiments in adaptive fashion which together with theoretical analysis show that our method reach the optimal convergence order.
Error estimates of finite element methods for nonlinear fractional stochastic differential equations

Yanpeng Zhang,Xiaoyuan Yang,Xiaocui Li

DOI: https://doi.org/10.1186/s13662-018-1665-0

2018-06-20

Advances in Difference Equations

Abstract:In this paper, we consider the Galerkin finite element approximations of the initial value problem for the nonlinear fractional stochastic partial differential equations with multiplicative noise. We study a spatial semidiscrete scheme with the standard Galerkin finite element method and a fully discrete scheme based on the Goreno–Mainardi–Moretti–Paradisi (GMMP) scheme. We establish strong convergence error estimates for both semidiscrete and fully discrete schemes.

mathematics, applied
Pass-efficient truncated UTV for low-rank approximations

Ying Ji,Yuehua Feng,Yongxin Dong

DOI: https://doi.org/10.1007/s40314-023-02584-4

2024-01-25

Computational and Applied Mathematics

Abstract:We propose the pass-efficient truncated UTV algorithm, a faster variant of the TUXV algorithm for low-rank approximations. Compared with the TUXV algorithm, data transfer and the complexity of the proposed algorithm are reduced. Therefore, our algorithm is suitable for large matrices stored out of memory or generated by streaming data. We also develop residual error upper bounds and singular value approximation error bounds for the pass-efficient truncated UTV algorithm. Numerical experiments are reported to demonstrate the efficiency and effectiveness of our algorithm.

mathematics, applied
Stochastic Collocation Method for Statistical Static Timing Analysis with Input Truncation Technique

Yi Wang,Xuan Zeng,Jun Tao,Hengliang Zhu,Xu Luo,Changhao Yan,Wei Cai

2008-01-01

Abstract:A novel stochastic collocation method with sparse grid and input truncation technique is proposed to perform statistical static timing analysis considering process variations. The proposed method first transforms the key operator MAX with statistical inputs into a set of deterministic MAX problems on a set of collocation points generated with sparse grid, and then solves the unknown coefficients with weighted least square technique. A novel input truncation technique is proposed to further reduce the computational time. Experimental results show that the algorithm achieved obvious improvements on accuracy compared with an existing moment matching based method and a stochastic Galerkin method with dimension reduction technique while kept the same order of efficiency. In comparison with 10000 Monte Carlo simulation results, the proposed method achieved relative errors of mean and variance mostly below 5%, with nearly 100X speeds up.

New Truncated Error Estimate for Fast Gauss Transform

Fast Gaussian particle filtering algorithm

A new version of the adaptive fast Gauss transform for discrete and continuous sources

Convergence Behavior of Gauss-Newton's Method and Extensions of the Smale Point Estimate Theory.

Approximate Gauss–Newton methods for solving underdetermined nonlinear least squares problems

Aggressively Truncated Taylor Series Method For Accurate Computation Of Exponentials Of Essentially Nonnegative Matrices

Smale's α-theory for inexact Newton methods under the γ-condition

Improved Fast Fractional-Fourier-transform Algorithm.

A New Recursive Bounded GM Estimator for Regression

Generalized Neumann Expansion and Its Application in Stochastic Finite Element Methods

ON CONVERGENCE OF A TRUNCATED GAUSS-NEWTON METHOD FOR SOLVING UNDERDETERMINED NONLINEAR LEAST SQUARES PROBLEMS

Faster stochastic trace estimation with a Chebyshev product identity

A Dynamic Low-Rank Fast Gaussian Transform

Fast simulation of truncated Gaussian distributions

An adaptive fast Gauss transform in two dimensions

An Inexact Shift‐and‐invert Arnoldi Algorithm for Toeplitz Matrix Exponential

An Improved Error Bound for Gaussian Interpolation

The a Priori and a Posteriori Error Estimates of DG Method for the Steklov Eigenvalue Problem in Inverse Scattering

Error estimates of finite element methods for nonlinear fractional stochastic differential equations

Pass-efficient truncated UTV for low-rank approximations

Stochastic Collocation Method for Statistical Static Timing Analysis with Input Truncation Technique