Regularization independent of the noise level: an analysis of quasi-optimality

Frank Bauer,Markus Reiss

DOI: https://doi.org/10.1088/0266-5611/24/5/055009

2007-10-05

Abstract:The quasi-optimality criterion chooses the regularization parameter in inverse problems without taking into account the noise level. This rule works remarkably well in practice, although Bakushinskii has shown that there are always counterexamples with very poor performance. We propose an average case analysis of quasi-optimality for spectral cut-off estimators and we prove that the quasi-optimality criterion determines estimators which are rate-optimal {\em on average}. Its practical performance is illustrated with a calibration problem from mathematical finance.

Numerical Analysis,Statistics Theory

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is about how to select the regularization parameter without relying on the noise level when solving inverse problems. Specifically, the paper explores the effectiveness and theoretical basis of the quasi - optimality criterion.

### Overview of the Main Problem

When dealing with linear inverse problems, the observed data \(y\) can be expressed as:
\[y = Ax+\xi\]
where \(A\) is a compact operator, \(x\) is the true solution to be recovered, and \(\xi\) is the noise variable. In order to stably approximate the true solution \(x\), regularization methods are usually used. A typical form of Tikhonov regularization is:
\[x_{\alpha}=(A^{*}A + \alpha I)^{-1}A^{*}y,\quad\alpha>0\]

Selecting an appropriate regularization parameter \(\alpha\) is crucial for the results. Traditional selection methods usually rely on the noise level \(\delta\), but the quasi - optimality criterion provides a method that does not depend on the noise level, that is, to select \(\alpha\) by minimizing the following expression:
\[\left\|\frac{dx_{\alpha}}{d\alpha}\right\|\to\min_{\alpha}\]

### Bakushinskii's Counterexample

Bakushinskii (1984) pointed out that any method for selecting regularization parameters that does not depend on the noise level may have counterexamples with extremely poor performance. However, the core view of this paper is that although Bakushinskii's result is correct for worst - case analysis, in the average case, the quasi - optimality criterion can perform very well.

### Average - Case Analysis

The author conducts an average - case analysis by introducing the Bayesian prior distribution and the spectral truncation estimator. Assume that the singular value decomposition coefficients of the solution \(x\) follow a normal distribution:
\[x_{k}\sim N(0,\gamma(k)^{2}),\quad k\geq1\]

Based on this assumption, the author proves that the regularization parameter selected using the quasi - optimality criterion can make the estimator achieve the optimal convergence rate in the average case.

### Numerical Experiments

The paper also demonstrates the practical performance of the quasi - optimality criterion through a calibration problem in mathematical finance. Specifically, the author considers the option pricing problem of an exponential Lévy model and compares the performance of different parameter selection methods, including the quasi - optimality criterion, the Lepski method, and the Hardened Balancing Principle. The experimental results show that the quasi - optimality criterion and the Hardened Balancing Principle perform better and are more stable in most cases.

### Conclusion

The main contribution of this paper is to prove that the quasi - optimality criterion is effective in the average case and provide theoretical support. In addition, numerical experiments also verify the superior performance of this method in practical applications.