Abstract:Anderson acceleration (AA) has a long history of use and a strong recent interest due to its potential ability to dramatically improve the linear convergence of the fixed-point iteration. Most authors are simply using and analyzing the stationary version of Anderson acceleration (sAA) with a constant damping factor or without damping. Little attention has been paid to nonstationary algorithms. However, damping can be useful and is sometimes crucial for simulations in which the underlying fixed-point operator is not globally contractive. The role of this damping factor has not been fully understood. In the present work, we consider the non-stationary Anderson acceleration algorithm with optimized damping (AAoptD) in each iteration to further speed up linear and nonlinear iterations by applying one extra inexpensive optimization. We analyze the convergence rate this procedure and develop an efficient and inexpensive implementation scheme. We show by extensive numerical experiments that the proposed non-stationary Anderson acceleration with optimized damping procedure often converges much faster than stationary AA with constant damping, adaptive damping or without damping, especially in the cases larger window sizes are needed. We also observe that simple strategies like using constant damping factors and adaptive damping factors, sometimes, work very well for some problems while sometimes they are even worse than AA without damping. Our proposed method is usually more robust than AA with constant damping and adaptive damping. Moreover, we also observed from our numerical results that damping can be good, but choosing the wrong damping factors may slow down the convergence rate. Theoretical analysis of the effects of damping factors are needed and important.

Near-optimal Closed-loop Method via Lyapunov Damping for Convex Optimization

A Bounded Optimal Control for Maximizing the Reliability of Randomly Excited Nonlinear Oscillators with Fractional Derivative Damping

Iterative design of suboptimal feedback control for bilinear parabolic PDE systems

POD-based Reduced Order Optimal Control of Parabolic PDE Systems Via Diffusivity-Interior-boundary Actuation

Boundary Stabilization of a Class of Reaction–advection–diffusion Systems Via a Gradient-Based Optimization Approach

A Numerical Method of the Euler-Bernoulli Beam with Optimal Local Kelvin-Voigt Damping

Fast convex optimization via closed-loop time scaling of gradient dynamics

Inertial Methods with Viscous and Hessian driven Damping for Non-Convex Optimization

Optimal damping adapted to a set of initial conditions

Adjoint-Based Design Optimization of Stability Constrained Systems

An SDE Perspective on Stochastic Inertial Gradient Dynamics with Time-Dependent Viscosity and Geometric Damping

Optimal Convergence Rates for Damped Inertial Gradient Dynamics with Flat Geometries

Online convex optimization for robust control of constrained dynamical systems

Semi-active damping optimization of vibrational systems using the reduced basis method

Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization

Enhancing Robotic System Robustness via Lyapunov Exponent-Based Optimization

Friction-adaptive descent: a family of dynamics-based optimization methods

Stability analysis and integral extension of sub‐optimal nonlinear damping control

Non-stationary Anderson acceleration with optimized damping

Optimal convergence rate of inertial gradient system with flat geometries and perturbations