Optimal recovery of convolutions of $n$ functions according to linear information

V. F. Babenko,M. S. Gunko,V. F.Babenko,M. S.Gunko

DOI: https://doi.org/10.48550/arXiv.1505.02345

2015-05-10

Functional Analysis

Abstract:We found the optimal linear information and the optimal method of its use to recovery of the convolution of n functions on some classes of $2\pi$ - periodic functions.

What problem does this paper attempt to address?

Optimal recovery of linear operators from information of random functions

K. Yu. Osipenko

2024-05-19

Abstract:The paper concerns problems of the recovery of linear operators defined on sets of functions from information of these functions given with stochastic errors. The constructed optimal recovery methods, in general, do not use all the available information. As a consequence, optimal methods are obtained for recovering derivatives of functions from Sobolev classes by the information of their Fourier transforms given with stochastic errors. A similar problem is considered for solutions of the heat equation.

Numerical Analysis
Convergence Analysis of General Neural Networks under Almost Periodic Stimuli

Zhenkun Huang,S. Mohamad,Xinghua Wang,Chunhua Feng

DOI: https://doi.org/10.1002/cta.490

IF: 2.378

2008-01-01

International Journal of Circuit Theory and Applications

Abstract:In this paper, we investigate the convergence dynamics of 2(N) almost periodic encoded patterns of general neural networks subjected to external almost periodic stimuli, including almost periodic delays. Invariant regions are established for the existence of 2(N) almost periodic encoded patterns under two classes of activation functions. By employing the property of M-cone and inequality technique, attracting basins are estimated and some criteria are derived for the networks to converge exponentially toward 2(N) almost periodic encoded patterns. The results obtained are new; they extend and generalize the corresponding results existing in the previous literature. Copyright (C) 2008 John Wiley & Sons, Ltd.
On the Optimal Recovery of One Family of Operators on a Class of Functions from Approximate Information about Its Spectrum

E. V. Abramova,E. O. Sivkova

DOI: https://doi.org/10.1134/s0037446624020010

2024-03-27

Siberian Mathematical Journal

Abstract:We find explicit expressions for optimal recovery methods in the problem of recovering the values of continuous linear operators on a Sobolev function class from the following information: The Fourier transform of functions is known approximately on some measurable subset of the finite-dimensional space on which the functions are defined. As corollaries, we obtain optimal methods for recovering the solution to the heat equation and solving the Dirichlet problem for a half-space.

mathematics
Near-Optimal Recovery of Linear and N-Convex Functions on Unions of Convex Sets

Anatoli Juditsky,Arkadi Nemirovski

2019-03-30

Abstract:In this paper we build provably near-optimal, in the minimax sense, estimates of linear forms and, more generally, "$N$-convex functionals" (the simplest example being the maximum of several fractional-linear functions) of unknown "signal" known to belong to the union of finitely many convex compact sets from indirect noisy observations of the signal. Our main assumption is that the observation scheme in question is good in the sense of A. Goldenshluger, A. Juditsky, A. Nemirovski, Electr. J. Stat. 9(2) (2015),

Mathematics
Optimal Recovery of a Family of Operators from Inaccurate Measurements on a Compact Set

E. O. Sivkova

DOI: https://doi.org/10.1134/s0037446624020228

2024-03-27

Siberian Mathematical Journal

Abstract:Given a one-parameter family of continuous linear operators , with , we consider the optimal recovery of the values of on the whole space by approximate information on the values of , where runs over a compact set and . We find a family of optimal methods for recovering the values of . Each of these methods uses approximate measurements at no more than two points in and depends linearly on these measurements. As a corollary, we provide some families of optimal methods for recovering the solution of the heat equation at a given moment of time from inaccurate measurements on other time intervals and for solving the Dirichlet problem for a half-space on a hyperplane by inaccurate measurements on other hyperplanes. The optimal recovery of the values of from the indicated information reduces to finding the value of an extremal problem for the maximum with continuum many inequality-type constraints, i.e., to finding the exact upper bound of the maximized functional under these constraints. This rather complicated task reduces to the infinite-dimensional problem of linear programming on the vector space of all finite real measures on the -algebra of Lebesgue measurable sets in . This problem can be solved by some generalization of the Karush–Kuhn–Tucker theorem, and its significance coincides with the significance of the original problem.

mathematics
On the construction of families of optimal recovery methods for linear operators

Konstantin Yur'evich Osipenko

DOI: https://doi.org/10.4213/im9384e

2024-04-05

Izvestiya Mathematics

Abstract:The paper proposes an approach for construction of families of optimal methods for the recovery of linear operators from inaccurately given information. The proposed method of construction is then applied to recover derivatives from inaccurately specified other derivatives in the multidimensional case and to recover solutions of the heat equation from inaccurately specified temperature distributions at some time instants.

mathematics
On Optimal Recovery and Information Complexity in Numerical Differentiation and Summation

Y.V. Semenova,S.G. Solodky

2024-11-11

Abstract:In this paper, we study optimization problems of numerical differentiation and summation methods on classes of univariate functions. Sharp estimates (in order) of the optimal recovery error and information complexity are calculated for these classes. Algorithms are constructed based on the truncation method and Chebyshev polynomials to implement these estimates. Moreover, we establish under what conditions the summation problem is well-posed.

Numerical Analysis
On optimal recovering high order partial derivatives of bivariate functions

Y.V. Semenova,S.G. Solodky

2023-09-11

Abstract:The problem of recovering partial derivatives of high orders of bivariate functions with finite smoothness is studied. Based on the truncation method, a numerical differentiation algorithm was constructed, which is optimal by the order, both in the sense of accuracy and in the sense of the amount of Galerkin information involved. Numerical demonstrations are provided to illustrate that the proposed method can be implemented successfully.

Numerical Analysis
Optimal Rates for Functional Linear Regression with General Regularization

Naveen Gupta,S. Sivananthan,Bharath K. Sriperumbudur

2024-06-14

Abstract:Functional linear regression is one of the fundamental and well-studied methods in functional data analysis. In this work, we investigate the functional linear regression model within the context of reproducing kernel Hilbert space by employing general spectral regularization to approximate the slope function with certain smoothness assumptions. We establish optimal convergence rates for estimation and prediction errors associated with the proposed method under a Hölder type source condition, which generalizes and sharpens all the known results in the literature.

Statistics Theory
Towards optimal algorithms for the recovery of low-dimensional models with linear rates

Yann Traonmilin,Jean François Aujol,Antoine Guennec

2024-10-09

Abstract:We consider the problem of recovering elements of a low-dimensional model from linear measurements. From signal and image processing to inverse problems in data science, this question has been at the center of many applications. Lately, with the success of models and methods relying on deep neural networks leading to non-convex formulations, traditional convex variational approaches have shown their limits. Furthermore, the multiplication of algorithms and recovery results makes identifying the best methods a complex task. In this article, we study recovery with a class of widely used algorithms without considering any underlying functional. This result leads to a class of projected gradient descent algorithms that recover a given low-dimensional with linear rates. The obtained rates decouple the impact of the quality of the measurements with respect to the model from its intrinsic complexity. As a consequence, we can directly measure the performance of this class of projected gradient descents through a restricted Lipschitz constant of the projection. By optimizing this constant, we define optimal algorithms. Our general approach provides an optimality result in the case of sparse recovery. Moreover, we uncover underlying linear rates of convergence for some ''plug and play'' imaging methods relying on deep priors by interpreting our results in this context, thus linking low-dimensional recovery and recovery with deep priors under a unified theory, validated by experiments on synthetic and real data.

Signal Processing
About optimization of methods for mixed derivatives of bivariate functions

Y.V. Semenova,S.G. Solodky

2023-09-18

Abstract:The problem of optimal recovering high-order mixed derivatives of bivariate functions with finite smoothness is studied. On the basis of the truncation method, an algorithm for numerical differentiation is constructed, which is order-optimal both in the sense of accuracy and in terms of the amount of involved Galerkin information.

Numerical Analysis
An Optimal Method for High-Order Mixed Derivatives of Bivariate Functions

Evgeniya V. Semenova,Sergiy G. Solodky

DOI: https://doi.org/10.1515/cmam-2023-0137

2024-01-02

Computational Methods in Applied Mathematics

Abstract:The problem of optimal recovering high-order mixed derivatives of bivariate functions with finite smoothness is studied. Based on the truncation method, an algorithm for numerical differentiation is constructed, which is order-optimal both in the sense of accuracy and in terms of the amount of involved Galerkin information. Numerical examples are provided to illustrate the fact that our approach can be implemented successfully.

mathematics, applied
Radius of Information for Two Intersected Centered Hyperellipsoids and Implications in Optimal Recovery from Inaccurate Data

Simon Foucart,Chunyang Liao

2024-01-20

Abstract:For objects belonging to a known model set and observed through a prescribed linear process, we aim at determining methods to recover linear quantities of these objects that are optimal from a worst-case perspective. Working in a Hilbert setting, we show that, if the model set is the intersection of two hyperellipsoids centered at the origin, then there is an optimal recovery method which is linear. It is specifically given by a constrained regularization procedure whose parameters, short of being explicit, can be precomputed by solving a semidefinite program. This general framework can be swiftly applied to several scenarios: the two-space problem, the problem of recovery from $\ell_2$-inaccurate data, and the problem of recovery from a mixture of accurate and $\ell_2$-inaccurate data. With more effort, it can also be applied to the problem of recovery from $\ell_1$-inaccurate data. For the latter, we reach the conclusion of existence of an optimal recovery method which is linear, again given by constrained regularization, under a computationally verifiable sufficient condition. Experimentally, this condition seems to hold whenever the level of $\ell_1$-inaccuracy is small enough. We also point out that, independently of the inaccuracy level, the minimal worst-case error of a linear recovery method can be found by semidefinite programming.

Optimization and Control,Functional Analysis
Near-Optimal Estimation of Linear Functionals with Log-Concave Observation Errors

Simon Foucart,Grigoris Paouris

2023-07-29

Abstract:This note addresses the question of optimally estimating a linear functional of an object acquired through linear observations corrupted by random noise, where optimality pertains to a worst-case setting tied to a symmetric, convex, and closed model set containing the object. It complements the article "Statistical Estimation and Optimal Recovery" published in the Annals of Statistics in 1994. There, Donoho showed (among other things) that, for Gaussian noise, linear maps provide near-optimal estimation schemes relatively to a performance measure relevant in Statistical Estimation. Here, we advocate for a different performance measure arguably more relevant in Optimal Recovery. We show that, relatively to this new measure, linear maps still provide near-optimal estimation schemes even if the noise is merely log-concave. Our arguments, which make a connection to the deterministic noise situation and bypass properties specific to the Gaussian case, offer an alternative to parts of Donoho's proof.

Statistics Theory,Functional Analysis
Sparse Convolution for Approximate Sparse Instance

Xiaoxiao Li,Zhao Song,Guangyi Zhang

2023-06-04

Abstract:Computing the convolution $A \star B$ of two vectors of dimension $n$ is one of the most important computational primitives in many fields. For the non-negative convolution scenario, the classical solution is to leverage the Fast Fourier Transform whose time complexity is $O(n \log n)$. However, the vectors $A$ and $B$ could be very sparse and we can exploit such property to accelerate the computation to obtain the result. In this paper, we show that when $\|A \star B\|_{\geq c_1} = k$ and $\|A \star B\|_{\leq c_2} = n-k$ holds, we can approximately recover the all index in $\mathrm{supp}_{\geq c_1}(A \star B)$ with point-wise error of $o(1)$ in $O(k \log (n) \log(k)\log(k/\delta))$ time. We further show that we can iteratively correct the error and recover all index in $\mathrm{supp}_{\geq c_1}(A \star B)$ correctly in $O(k \log(n) \log^2(k) (\log(1/\delta) + \log\log(k)))$ time.

Data Structures and Algorithms
On the Geometry and Optimization of Polynomial Convolutional Networks

Vahid Shahverdi,Giovanni Luca Marchetti,Kathlén Kohn

2024-10-01

Abstract:We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from algebraic geometry, we explore the geometric properties of the image in function space of this map -- typically referred to as neuromanifold. In particular, we compute the dimension and the degree of the neuromanifold, which measure the expressivity of the model, and describe its singularities. Moreover, for a generic large dataset, we derive an explicit formula that quantifies the number of critical points arising in the optimization of a regression loss.

Machine Learning,Algebraic Geometry
Volume estimates and their applications in the problem of optimal recovery

Alexander Kushpel

DOI: https://doi.org/10.48550/arXiv.2212.03834

2022-12-08

Abstract:We study volumes of sections of convex origin-symmetric bodies in $\mathbb{R}% ^{n}$ induced by orthonormal systems on probability spaces. The approach is based on volume estimates of \ John-Löwner ellipsoids and expectations of norms induced by the respective systems. The estimates obtained allows us to establish lower bounds for the radii of sections which gives lower bounds for Gelfand widths (or linear cowidths). As an application we offer a new method of evaluation of Gelfand and Kolmogorov widths of multiplier operators. In particular, we establish sharp orders of widths of standard Sobolev classes $W_{p}^{\gamma }$ in $L_{q}$ in the difficult case, i.e. $% 1<q\leq p\leq \infty $.

Functional Analysis
Efficient recovery of non-periodic multivariate functions from few scattered samples

Felix Bartel,Kai Lüttgen,Nicolas Nagel,Tino Ullrich

2023-07-21

Abstract:It has been observed by several authors that well-known periodization strategies like tent or Chebychev transforms lead to remarkable results for the recovery of multivariate functions from few samples. So far, theoretical guarantees are missing. The goal of this paper is twofold. On the one hand, we give such guarantees and briefly describe the difficulties of the involved proof. On the other hand, we combine these periodization strategies with recent novel constructive methods for the efficient subsampling of finite frames in $\mathbb{C}$. As a result we are able to reconstruct non-periodic multivariate functions from very few samples. The used sampling nodes are the result of a two-step procedure. Firstly, a random draw with respect to the Chebychev measure provides an initial node set. A further sparsification technique selects a significantly smaller subset of these nodes with equal approximation properties. This set of sampling nodes scales linearly in the dimension of the subspace on which we project and works universally for the whole class of functions. The method is based on principles developed by Batson, Spielman, and Srivastava and can be numerically implemented. Samples on these nodes are then used in a (plain) least-squares sampling recovery step on a suitable hyperbolic cross subspace of functions resulting in a near-optimal behavior of the sampling error. Numerical experiments indicate the applicability of our results.

Numerical Analysis,Functional Analysis
An optimal method for high order mixed derivatives of bivariate functions

Y.V. Semenova,S.G. Solodky

2024-05-05

Abstract:The problem of optimal recovering high-order mixed derivatives of bivariate functions with finite smoothness is studied. Based on the truncation method, an algorithm for numerical differentiation is constructed, which is order-optimal both in the sense of accuracy and in terms of the amount of involved Galerkin information. Numerical examples are provided to illustrate the fact that our approach can be implemented successfully.

Numerical Analysis
Full Recovery from Point Values: an Optimal Algorithm for Chebyshev Approximability Prior

Simon Foucart

2023-07-29

Abstract:Given pointwise samples of an unknown function belonging to a certain model set, one seeks in Optimal Recovery to recover this function in a way that minimizes the worst-case error of the recovery procedure. While it is often known that such an optimal recovery procedure can be chosen to be linear, e.g. when the model set is based on approximability by a subspace of continuous functions, a construction of the procedure is rarely available. This note uncovers a practical algorithm to construct a linear optimal recovery map when the approximation space is a Chevyshev space of dimension at least three and containing the constant functions.

Numerical Analysis

Optimal recovery of convolutions of $n$ functions according to linear information

Optimal recovery of linear operators from information of random functions

Convergence Analysis of General Neural Networks under Almost Periodic Stimuli

On the Optimal Recovery of One Family of Operators on a Class of Functions from Approximate Information about Its Spectrum

Near-Optimal Recovery of Linear and N-Convex Functions on Unions of Convex Sets

Optimal Recovery of a Family of Operators from Inaccurate Measurements on a Compact Set

On the construction of families of optimal recovery methods for linear operators

On Optimal Recovery and Information Complexity in Numerical Differentiation and Summation

On optimal recovering high order partial derivatives of bivariate functions

Optimal Rates for Functional Linear Regression with General Regularization

Towards optimal algorithms for the recovery of low-dimensional models with linear rates

About optimization of methods for mixed derivatives of bivariate functions

An Optimal Method for High-Order Mixed Derivatives of Bivariate Functions

Radius of Information for Two Intersected Centered Hyperellipsoids and Implications in Optimal Recovery from Inaccurate Data

Near-Optimal Estimation of Linear Functionals with Log-Concave Observation Errors

Sparse Convolution for Approximate Sparse Instance

On the Geometry and Optimization of Polynomial Convolutional Networks

Volume estimates and their applications in the problem of optimal recovery

Efficient recovery of non-periodic multivariate functions from few scattered samples

An optimal method for high order mixed derivatives of bivariate functions

Full Recovery from Point Values: an Optimal Algorithm for Chebyshev Approximability Prior