Optimal Algorithms for Numerical Integration: Recent Results and Open Problems

Erich Novak

2023-07-13

Abstract:We present recent results on optimal algorithms for numerical integration and several open problems. The paper has six parts: 1. Introduction 2. Lower Bounds 3. Universality 4. General Domains 5. iid Information 6. Concluding Remarks

Numerical Analysis,Computational Complexity

What problem does this paper attempt to address?

Optimization of Approximate Solution of Integral Equations of Several Variables

Wan Ma,Xing Hua Wang

DOI: https://doi.org/10.1007/s10114-002-0175-9

2002-01-01

Acta Mathematica Sinica English Series

Abstract:In this paper, we consider the problem of optimization of adaptive direct methods of operator equations. Adaptivity of a direct method is understood in the sense that the subspace on the basis of which it is constructed is chosen depending on the operator of the concrete equation (otherwise, nonadaptive direct method is then concerned), which would essentially let us increase the precision. For some classes of the second kind of Fredhlom integral equations with anisotropic smooth kernels we determine the exact order of the error of adaptive direct methods, and we also give an optimal algorithm.
Optimal randomized changing dimension algorithms for infinite-dimensional integration on function spaces with ANOVA-type decomposition

Josef Dick,Michael Gnewuch

DOI: https://doi.org/10.1016/j.jat.2014.04.014

2013-06-12

Abstract:We study the numerical integration problem for functions with infinitely many variables. The function spaces of integrands we consider are weighted reproducing kernel Hilbert spaces with norms related to the ANOVA decomposition of the integrands. The weights model the relative importance of different groups of variables. We investigate randomized quadrature algorithms and measure their quality by estimating the randomized worst-case integration error. In particular, we provide lower error bounds for a very general class of randomized algorithms that includes non-linear and adaptive algorithms. Furthermore, we propose new randomized changing dimension algorithms and present favorable upper error bounds. For product weights and finite-intersection weights our lower and upper error bounds match and show that our changing dimension algorithms are optimal in the sense that they achieve convergence rates arbitrarily close to the best possible convergence rate. As more specific examples, we discuss unanchored Sobolev spaces of different degrees of smoothness and randomized changing dimension algorithms that use as building blocks scrambled polynomial lattice rules. Our analysis extends the analysis given in [J. Baldeaux, M. Gnewuch. Optimal randomized multilevel algorithms for infinite-dimensional integration on function spaces with ANOVA-type decomposition. <a class="link-https" data-arxiv-id="1209.0882v1" href="https://arxiv.org/abs/1209.0882v1">arXiv:1209.0882v1</a> [math.NA], Preprint 2012]. In contrast to the previous article we now investigate a different cost model for algorithms. With respect to that cost model, randomized multilevel algorithms cannot, in general, achieve optimal convergence rates, but, as we prove for important classes of weights, changing dimension algorithms actually can.

Numerical Analysis
Numerical Methods of Optimal Accuracy for Weakly Singular Volterra Integral Equations

I.V. Boykov,A.N. Tynda

DOI: https://doi.org/10.48550/arXiv.1306.2904

2013-06-13

Abstract:Weakly singular Volterra integral equations of the different types are considered. The construction of accuracy-optimal numerical methods for one-dimensional and multidimensional equations is discussed. Since this question is closely related with the optimal approximation problem, the orders of the Babenko and Kolmogorov $n-$widths of compact sets from some classes of functions have been evaluated. In conclusion we adduce some numerical illustrations for 2-D Volterra equations.

Numerical Analysis
Numerical aspects of integration in physics-informed neural networks modelling

Josef Daněk,Jan Pospíšil

2024-08-10

Abstract:In this paper we numerically analyse problems that arise in numerical integration used in various problems that are nowadays being solved by physics-informed neural networks. In particular we show how inaccurately evaluated integrands can lead to severe precision problems. The tricky and serious part of the problem is especially in its detection, when for example a simple macroscopic visualization of the integrand does not indicate any potential problems and obtained results are often wrongly treated as correct. To solve the problem, it is crucial to decide if it is sufficient to evaluate the integrand in the standard double arithmetic or if a higher precision arithmetic has to be used. We compare and recommend numerical quadratures for selected examples and different parameter values, especially for problematic cases.

Numerical Analysis
On Optimal Recovery and Information Complexity in Numerical Differentiation and Summation

Y.V. Semenova,S.G. Solodky

2024-11-11

Abstract:In this paper, we study optimization problems of numerical differentiation and summation methods on classes of univariate functions. Sharp estimates (in order) of the optimal recovery error and information complexity are calculated for these classes. Algorithms are constructed based on the truncation method and Chebyshev polynomials to implement these estimates. Moreover, we establish under what conditions the summation problem is well-posed.

Numerical Analysis
Numerical Algorithm for Optimal Control of Continuity Equations

Nikolay Pogodaev

DOI: https://doi.org/10.48550/arXiv.1708.05516

2017-08-18

Abstract:An optimal control problem for the continuity equation is considered. The aim of a controller is to maximize the total mass within a target set at a given type moment. An iterative numerical algorithm for solving this problem is presented.

Optimization and Control
Optimal numerical integration and approximation of functions on $\mathbb{R}^d$ equipped with Gaussian measure

Dinh Dũng,Van Kien Nguyen

2023-06-17

Abstract:We investigate the numerical approximation of integrals over $\mathbb{R}^d$ equipped with the standard Gaussian measure $\gamma$ for integrands belonging to the Gaussian-weighted Sobolev spaces $W^\alpha_p(\mathbb{R}^d, \gamma)$ of mixed smoothness $\alpha \in \mathbb{N}$ for $1 < p < \infty$. We prove the asymptotic order of the convergence of optimal quadratures based on $n$ integration nodes and propose a novel method for constructing asymptotically optimal quadratures. As for related problems, we establish by a similar technique the asymptotic order of the linear, Kolmogorov and sampling $n$-widths in the Gaussian-weighted space $L_q(\mathbb{R}^d, \gamma)$ of the unit ball of $W^\alpha_p(\mathbb{R}^d, \gamma)$ for $1 \leq q < p < \infty$ and $q=p=2$.

Numerical Analysis
A Random Integration Algorithm for High-dimensional Function Spaces

Liang Chen,Minqiang Xu,Haizhang Zhang

2024-06-24

Abstract:We introduce a novel random integration algorithm that boasts both high convergence order and polynomial tractability for functions characterized by sparse frequencies or rapidly decaying Fourier coefficients. Specifically, for integration in periodic isotropic Sobolev space and the isotropic Sobolev space with compact support, our approach attains a nearly optimal root mean square error (RMSE) bound. In contrast to previous nearly optimal algorithms, our method exhibits polynomial tractability, ensuring that the number of samples does not scale exponentially with increasing dimensions. Our integration algorithm also enjoys nearly optimal bound for weighted Korobov space. Furthermore, the algorithm can be applied without the need for prior knowledge of weights, distinguishing it from the component-by-component algorithm. For integration in the Wiener algebra, the sample complexity of our algorithm is independent of the decay rate of Fourier coefficients. The effectiveness of the integration is confirmed through numerical experiments.

Numerical Analysis
Remarks on numerical integration, discrepancy, and diaphony

V.N. Temlyakov

DOI: https://doi.org/10.48550/arXiv.1711.07017

2017-11-19

Abstract:The goal of this paper is twofold. First, we present a unified way of formulating numerical integration problems from both approximation theory and discrepancy theory. Second, we show how techniques, developed in approximation theory, work in proving lower bounds for recently developed new type of discrepancy -- the smooth discrepancy.

Numerical Analysis
Numerical integration without smoothness assumption

Vladimir Temlyakov

DOI: https://doi.org/10.48550/arXiv.2003.14331

2020-03-30

Numerical Analysis

Abstract:We consider numerical integration in classes, for which we do not impose any smoothness assumptions. We illustrate how nonlinear approximation, in particular greedy approximation, allows us to guarantee some rate of decay of errors of numerical integration even in such a general setting with no smoothness assumptions.
New Time Domain Decomposition Methods for Parabolic Optimal Control Problems I: Dirichlet–Neumann and Neumann–Dirichlet Algorithms

Martin J. Gander,Liu-Di Lu

DOI: https://doi.org/10.1137/23m1584502

IF: 3.039

2024-08-25

SIAM Journal on Numerical Analysis

Abstract:SIAM Journal on Numerical Analysis, Volume 62, Issue 4, Page 2048-2070, August 2024. We present new Dirichlet–Neumann and Neumann–Dirichlet algorithms with a time domain decomposition applied to unconstrained parabolic optimal control problems. After a spatial semidiscretization, we use the Lagrange multiplier approach to derive a coupled forward-backward optimality system, which can then be solved using a time domain decomposition. Due to the forward-backward structure of the optimality system, three variants can be found for the Dirichlet–Neumann and Neumann–Dirichlet algorithms. We analyze their convergence behavior and determine the optimal relaxation parameter for each algorithm. Our analysis reveals that the most natural algorithms are actually only good smoothers, and there are better choices which lead to efficient solvers. We illustrate our analysis with numerical experiments.

mathematics, applied
A simple universal algorithm for high-dimensional integration

Takashi Goda,David Krieg

2024-11-28

Abstract:We present a simple universal algorithm for high-dimensional integration which has the optimal error rate (independent of the dimension) in all weighted Korobov classes both in the randomized and the deterministic setting. Our theoretical findings are complemented by numerical tests.

Numerical Analysis,Computational Complexity
Integration in reproducing kernel Hilbert spaces of Gaussian kernels

Toni Karvonen,Chris J. Oates,Mark Girolami,Chris Oates

DOI: https://doi.org/10.1090/mcom/3659

IF: 2.118

2021-06-18

Mathematics of Computation

Abstract:The Gaussian kernel plays a central role in machine learning, uncertainty quantification and scattered data approximation, but has received relatively little attention from a numerical analysis standpoint. The basic problem of finding an algorithm for efficient numerical integration of functions reproduced by Gaussian kernels has not been fully solved. In this article we construct two classes of algorithms that use N N evaluations to integrate d d -variate functions reproduced by Gaussian kernels and prove the exponential or super-algebraic decay of their worst-case errors. In contrast to earlier work, no constraints are placed on the length-scale parameter of the Gaussian kernel. The first class of algorithms is obtained via an appropriate scaling of the classical Gauss–Hermite rules. For these algorithms we derive lower and upper bounds on the worst-case error of the forms exp ⁡ ( − c 1 N 1 / d ) N 1 / ( 4 d ) \exp (-c_1 N^{1/d}) N^{1/(4d)} and exp ⁡ ( − c 2 N 1 / d ) N − 1 / ( 4 d ) \exp (-c_2 N^{1/d}) N^{-1/(4d)} , respectively, for positive constants c 1 > c 2 c_1 > c_2 . The second class of algorithms we construct is more flexible and uses worst-case optimal weights for points that may be taken as a nested sequence. For these algorithms we derive upper bounds of the form exp ⁡ ( − c 3 N 1 / ( 2 d ) ) \exp (-c_3 N^{1/(2d)}) for a positive constant c 3 c_3 .

mathematics, applied
On the power of random information

Aicke Hinrichs,David Krieg,Erich Novak,Joscha Prochno,Mario Ullrich

DOI: https://doi.org/10.48550/arXiv.1903.00681

2019-03-02

Abstract:We study approximation and integration problems and compare the quality of optimal information with the quality of random information. For some problems random information is almost optimal and for some other problems random information is much worse than optimal information. We prove new results and give a short survey of known results.

Numerical Analysis,Probability
Probabilistic ODE Solvers for Integration Error-Aware Numerical Optimal Control

Amon Lahr,Filip Tronarp,Nathanael Bosch,Jonathan Schmidt,Philipp Hennig,Melanie N. Zeilinger

2024-06-13

Abstract:Appropriate time discretization is crucial for real-time applications of numerical optimal control, such as nonlinear model predictive control. However, if the discretization error strongly depends on the applied control input, meeting accuracy and sampling time requirements simultaneously can be challenging using classical discretization methods. In particular, neither fixed-grid nor adaptive-grid discretizations may be suitable, when they suffer from large integration error or exceed the prescribed sampling time, respectively. In this work, we take a first step at closing this gap by utilizing probabilistic numerical integrators to approximate the solution of the initial value problem, as well as the computational uncertainty associated with it, inside the optimal control problem (OCP). By taking the viewpoint of probabilistic numerics and propagating the numerical uncertainty in the cost, the OCP is reformulated such that the optimal input reduces the computational uncertainty insofar as it is beneficial for the control objective. The proposed approach is illustrated using a numerical example, and potential benefits and limitations are discussed.

Optimization and Control
Numerical integration of stochastic differential equations

Riccardo Mannella

DOI: https://doi.org/10.48550/arXiv.cond-mat/9709326

1997-09-30

Abstract:Numerical algorithms for the integration of stochastic differential equations in the presence of white noise are introduced and compared. Algorithms for the integration of stochastic correlated forces are also briefly reviewed. Finally, a specialised algorithm for two dimensional systems is derived, having in mind the integration of particles in the liquid state.

Statistical Mechanics,Disordered Systems and Neural Networks
A general quantum algorithm for numerical integration

Guoqiang Shu,Zheng Shan,Jinchen Xu,Jie Zhao,Shuya Wang

DOI: https://doi.org/10.1038/s41598-024-61010-9

IF: 4.6

2024-05-09

Scientific Reports

Abstract:Quantum algorithms have shown their superiority in many application fields. However, a general quantum algorithm for numerical integration, an indispensable tool for processing sophisticated science and engineering issues, is still missing. Here, we first proposed a quantum integration algorithm suitable for any continuous functions that can be approximated by polynomials. More impressively, the algorithm achieves quantum encoding of any integrable functions through polynomial approximation, then constructs a quantum oracle to mark the number of points in the integration area and finally converts the statistical results into the phase angle in the amplitude of the superposition state. The quantum algorithm introduced in this work exhibits quadratic acceleration over the classical integration algorithms by reducing computational complexity from O ( N ) to O (√ N ). Our work addresses the crucial impediments for improving the generality of quantum integration algorithm, which provides a meaningful guidance for expanding the superiority of quantum computing.

multidisciplinary sciences
High order variational integrators in the optimal control of mechanical systems

Cédric M. Campos,Sina Ober-Blöbaum,Emmanuel Trélat

DOI: https://doi.org/10.48550/arXiv.1502.00325

2015-02-01

Optimization and Control

Abstract:In recent years, much effort in designing numerical methods for the simulation and optimization of mechanical systems has been put into schemes which are structure preserving. One particular class are variational integrators which are momentum preserving and symplectic. In this article, we develop two high order variational integrators which distinguish themselves in the dimension of the underling space of approximation and we investigate their application to finite-dimensional optimal control problems posed with mechanical systems. The convergence of state and control variables of the approximated problem is shown. Furthermore, by analyzing the adjoint systems of the optimal control problem and its discretized counterpart, we prove that, for these particular integrators, dualization and discretization commute.
Two Numerical Schemes Via an Auxiliary Function for Nonlocal Problems With Integrable Kernels

Yutong Li,Xiaobo Yin

DOI: https://doi.org/10.1002/num.23153

2024-10-11

Numerical Methods for Partial Differential Equations

Abstract:In this study, we explore volume constrained nonlocal problem with integrable kernels. A central challenge addressed is the discontinuity of true solutions across the boundary, posing difficulties for designing efficient numerical methods. To address this issue, we introduce an auxiliary function which serves as the solution of an auxiliary problem. This approach effectively transforms the original problem to a new nonlocal problem with a true solution which is continuous across the boundary. We propose two numerical schemes to solve the new nonlocal problem: one is exclusively applicable to one‐dimensional case, while the other is suitable for general dimensional cases. Experimental results show that both schemes exhibit optimal convergence order under mild condition.

mathematics, applied
Numerical Integration over Implicitly Defined Domains with Topological Guarantee

Tianhui Yang,Ammar Qarariyah,Hongmei Kang,Jiansong Deng

DOI: https://doi.org/10.1007/s40304-019-00178-3

2019-01-01

Communications in Mathematics and Statistics

Abstract:Numerical integration over the implicitly defined domains is challenging due to topological variances of implicit functions. In this paper, we use interval arithmetic to identify the boundary of the integration domain exactly, thus getting the correct topology of the domain. Furthermore, a geometry-based local error estimate is explored to guide the hierarchical subdivision and save the computation cost. Numerical experiments are presented to demonstrate the accuracy and the potential of the proposed method.

Optimal Algorithms for Numerical Integration: Recent Results and Open Problems

Optimization of Approximate Solution of Integral Equations of Several Variables

Optimal randomized changing dimension algorithms for infinite-dimensional integration on function spaces with ANOVA-type decomposition

Numerical Methods of Optimal Accuracy for Weakly Singular Volterra Integral Equations

Numerical aspects of integration in physics-informed neural networks modelling

On Optimal Recovery and Information Complexity in Numerical Differentiation and Summation

Numerical Algorithm for Optimal Control of Continuity Equations

Optimal numerical integration and approximation of functions on $\mathbb{R}^d$ equipped with Gaussian measure

A Random Integration Algorithm for High-dimensional Function Spaces

Remarks on numerical integration, discrepancy, and diaphony

Numerical integration without smoothness assumption

New Time Domain Decomposition Methods for Parabolic Optimal Control Problems I: Dirichlet–Neumann and Neumann–Dirichlet Algorithms

A simple universal algorithm for high-dimensional integration

Integration in reproducing kernel Hilbert spaces of Gaussian kernels

On the power of random information

Probabilistic ODE Solvers for Integration Error-Aware Numerical Optimal Control

Numerical integration of stochastic differential equations

A general quantum algorithm for numerical integration

High order variational integrators in the optimal control of mechanical systems

Two Numerical Schemes Via an Auxiliary Function for Nonlocal Problems With Integrable Kernels

Numerical Integration over Implicitly Defined Domains with Topological Guarantee