Abstract:We discuss the upper and lower estimates for the rate of convergence of Pure and Orthogonal Greedy Algorithms for dictionary with bounded cumulative coherence.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the convergence rate estimation problem of Greedy Algorithms with bounded cumulative coherence in dictionaries. Specifically, the paper focuses on the convergence speeds of the Pure Greedy Algorithm (PGA) and the Orthogonal Greedy Algorithm (OGA) on different function classes under specific conditions.
### Main research contents:
1. **Definition and background**:
- The research object is the dictionary \(D\) in the Hilbert space \(H\), where each element \(g\in D\) satisfies \(\|g\| = 1\), and \(\text{span}D = H\).
- It discusses how to approximate a given element \(f\in H\) through a finite - term combination \(f\approx\sum_{k = 1}^m c_k(f)g_k(f)\).
- Two main greedy algorithms are introduced: the Pure Greedy Algorithm (PGA) and the Orthogonal Greedy Algorithm (OGA).
2. **Convergence rate analysis**:
- For the dictionary \(D\) with bounded cumulative coherence, the convergence rates of PGA and OGA are studied.
- Special attention is paid to the cumulative coherence \(\mu_1(D)\) of the dictionary \(D\), which is defined as:
\[
\mu_1(D):=\sup_{g\in D}\sum_{e_g\in D,e_g\neq g}|\langle e_g,g\rangle|
\]
- The convergence rates on different function classes \(A_p(D)\), especially when \(1\leq p < 2\), are analyzed.
3. **Main conclusions**:
- For the dictionary with \(\mu_1(D)<1/3\), the convergence rate of PGA on \(A_1(D)\) is \(|f|_1m^{- 1/2}\).
- For \(\mu_1(D)<1/3\) and \(1\leq p < 2\), the convergence rate of PGA on \(A_p(D)\) is \(C_1C_2|f|_pm^{-1/p + 1/2}\).
- For the dictionary with large cumulative coherence, PGA may not be able to provide an exponential convergence rate, and may even be worse than \(Cm^{-1/2}\).
### Solved problems:
- **Convergence rate estimation**: It provides upper and lower bound estimates of the convergence rates of PGA and OGA on different function classes.
- **Influence of cumulative coherence**: It explores the specific influence of the cumulative coherence \(\mu_1(D)\) of the dictionary on the convergence rate of the greedy algorithm.
- **Optimality analysis**: It discusses that under certain conditions, OGA provides the optimal convergence rate, while PGA is not necessarily optimal.
Through these studies, the paper provides a theoretical basis for understanding the performance of greedy algorithms in sparse approximation and points out the key role of cumulative coherence in it.