Abstract:Two adaptive relaxation strategies are proposed for Anderson acceleration. They are specifically designed for applications in which mappings converge to a fixed point. Their superiority over alternative Anderson acceleration is demonstrated for linear contraction mappings. Both strategies perform well in three nonlinear fixed-point applications that include partial differential equations and the EM algorithm. One strategy surpasses all other Anderson acceleration implementations tested in terms of computation time across various specifications, including composite Anderson acceleration.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: **How to improve the convergence speed and stability of Anderson Acceleration (AA) in fixed - point iteration through adaptive relaxation strategies**. Specifically, the authors propose two new adaptive relaxation strategies, specifically for the application scenarios where mappings converge to fixed points, aiming to improve the computational efficiency and robustness of AA.
### Background and Problem Description of the Paper
Anderson Acceleration (AA) is a method for accelerating fixed - point iteration and is widely used in numerical solution problems in physics, mathematics, and other disciplines. The core idea of AA is to accelerate convergence by linearly combining historical iteration points. However, in practical applications, the performance of AA depends on a key parameter - the relaxation parameter, usually denoted as \(\beta_k\). Traditional AA methods usually use a fixed value of \(\beta_k\), such as 1, but in some cases, this may lead to unstable convergence or slow speed.
### New Methods Proposed in the Paper
To improve this situation, the authors propose two adaptive relaxation strategies:
1. **AAopt1 T**:
- This method optimizes the update step in each iteration by dynamically adjusting \(\beta_k\).
- Specifically, AAopt1 uses non - relaxed small - scale AA iterations and recomputes the optimal \(\beta_k\) every \(T\) iterations.
- It is theoretically proven that for contractive linear mappings, AAopt1 converges faster than the traditional AAopt0.
2. **AAmd**:
- This method selects the optimal \(\beta_k\) by minimizing the distance between the current iteration point and its mapping.
- Only two inner products need to be calculated additionally, without additional mapping calculations, thus reducing the computational cost.
- Experiments show that AAmd significantly improves the convergence speed in nonlinear application scenarios.
### Application and Experimental Verification
To verify the effectiveness of these two new methods, the authors tested them in three fixed - point applications:
1. **Poisson Partial Differential Equation (PDE)**:
- This problem involves solving a nonlinear Poisson equation with challenging boundary conditions.
2. **Expectation - Maximization (EM) Algorithm**:
- It includes two specific EM applications: the proportional hazards model and the mixed - population analysis.
- The EM algorithm is often used to handle statistical models with missing data or latent variables, but its convergence speed is slow, so it is an ideal test object for acceleration methods.
### Conclusion
Through these experiments, the authors demonstrated the superiority of the new adaptive relaxation strategies in different application scenarios. In particular, AAmd shows better convergence speed in all nonlinear applications, and AAopt1 T can also significantly reduce the computation time in some cases. These results provide a more efficient and stable solution for fixed - point iteration problems.
### Summary
The main contribution of this paper lies in proposing two new adaptive relaxation strategies for linear and nonlinear fixed - point iteration problems respectively, which significantly improve the performance of Anderson Acceleration. These methods are not only theoretically guaranteed but also show good performance in practical applications.