Abstract:Anderson Acceleration (AA) has been widely used to solve nonlinear fixed-point problems due to its rapid convergence. This work focuses on a variant of AA in which multiple Picard iterations are performed between each AA step, referred to as the Alternating Anderson-Picard (AAP) method. Despite introducing more 'slow' Picard iterations, this method has been shown to be efficient and even more robust in both linear and nonlinear cases. However, there is a lack of theoretical analysis for AAP in the nonlinear case, which this paper aims to address. We show the equivalence between AAP and a multisecant-GMRES method that uses GMRES to solve a multisecant linear system at each iteration. More interestingly, the incorporation of Picard iterations and AA establishes a deep connection between AAP and the Newton-GMRES method. This connection is evident in terms of the multisecant matrix, the approximate Jacobian inverse, search direction, and optimization gain -- an essential factor in the convergence analysis of AA. We show that these terms converge to their corresponding terms in the Newton-GMRES method as the residual approaches zero. Consequently, we build the convergence analysis of AAP. To validate our theoretical findings, numerical examples are provided.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is about the convergence analysis of the Alternating Anderson - Picard (AAP) method in nonlinear fixed - point problems. Specifically, the paper focuses on the fact that in the nonlinear case, the theoretical convergence analysis of the AAP method has not been fully studied, while this method has been proven to be efficient and robust in practical applications. Therefore, the main objective of this paper is to fill this theoretical gap, and derive the convergence analysis of AAP by establishing the equivalence relationship between AAP and the multi - secant - GMRES method, as well as the deep connection between AAP and the Newton - GMRES method. ### Main Contributions 1. **Establishment of Equivalence Relationship**: The paper shows the equivalence between the AAP method and the multi - secant - GMRES method, that is, in each global iteration, AAP implicitly solves the multi - secant linear system and performs an additional linearized Picard iteration. 2. **Connection with Newton - GMRES Method**: By analyzing the multi - secant matrix \(B_t\), matrices \(S_t\) and \(Y_t\), and the approximate Jacobian inverse \(H_t\), the paper establishes a deep connection between AAP and the Newton - GMRES method. In particular, when the residual approaches zero, these matrices and directions converge to the corresponding terms in the Newton - GMRES method. 3. **Convergence Analysis**: Based on the above - mentioned equivalence relationship and connection, the paper provides the convergence analysis of the AAP method, including one - step residual bounds and local convergence results. ### Key Technical Details - **Multi - Secant Matrix**: Defines the multi - secant matrix \(B_t\) to satisfy the multi - secant equation \(B_tS_t = Y_t\), and discusses its properties and generation methods. - **Krylov Subspace**: Utilizes the properties of Krylov subspaces to show the relationship between the subspaces generated by the history points in the AAP method and the subspaces in the Newton - GMRES method. - **Residual Analysis**: Through lemmas and theorems, analyzes that when the residual \(f_t\) approaches zero, the distance between the multi - secant matrix \(B_t\) and the Jacobian matrix \(J_t\) approaches zero. - **Convergence of Approximate Jacobian Inverse**: Proves that the approximate Jacobian inverse \(H_t\) in the AAP method converges to the update operator in the Newton - GMRES method when the residual approaches zero. ### Conclusion Through rigorous mathematical analysis, the paper proves the convergence of the AAP method in nonlinear fixed - point problems and establishes a deep connection between it and the Newton - GMRES method. These results not only fill the theoretical gap but also provide a solid theoretical basis for the practical application of the AAP method.

Convergence Analysis of the Alternating Anderson-Picard Method for Nonlinear Fixed-point Problems

Anderson Acceleration as a Krylov Method with Application to Asymptotic Convergence Analysis

Asymptotic convergence of restarted Anderson acceleration for certain normal linear systems

On the Convergence of the Accelerated Riccati Iteration Method.

A short report on preconditioned Anderson acceleration method

Anderson Acceleration with Truncated Gram-Schmidt

Anderson Acceleration in Nonsmooth Problems: Local Convergence via Active Manifold Identification

Anderson Acceleration with Truncated Gram–Schmidt

On the Convergence of CROP-Anderson Acceleration Method

Improved Convergence Rates of Windowed Anderson Acceleration for Symmetric Fixed-Point Iterations

Efficient Acceleration Framework for Complex‐Valued Linear Systems: Alternating Anderson Accelerating Preconditioned Richardson Approach

Anderson acceleration of derivative-free projection methods for constrained monotone nonlinear equations

Practical considerations for the adoption of Anderson acceleration in nonlinear diffusion accelerated transport

Anderson acceleration. Convergence analysis and applications to equilibrium chemistry

Anderson acceleration with approximate calculations: applications to scientific computing

Anderson acceleration with adaptive relaxation for convergent fixed-point iterations

Non-stationary Anderson acceleration with optimized damping

Convergence Analysis for Restarted Anderson Mixing and Beyond

A Fast Anderson-Chebyshev Acceleration for Nonlinear Optimization.

A Class of Short-term Recurrence Anderson Mixing Methods and Their Applications

Descent Properties of an Anderson Accelerated Gradient Method With Restarting