Abstract:We combine momentum from machine learning with evolutionary dynamics, where momentum can be viewed as a simple mechanism of intergenerational memory. Using information divergences as Lyapunov functions, we show that momentum accelerates the convergence of evolutionary dynamics including the replicator equation and Euclidean gradient descent on populations. When evolutionarily stable states are present, these methods prove convergence for small learning rates or small momentum, and yield an analytic determination of the relative decrease in time to converge that agrees well with computations. The main results apply even when the evolutionary dynamic is not a gradient flow. We also show that momentum can alter the convergence properties of these dynamics, for example by breaking the cycling associated to the rock-paper-scissors landscape, leading to either convergence to the ordinarily non-absorbing equilibrium, or divergence, depending on the value and mechanism of momentum.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: how to accelerate the convergence process in evolutionary dynamics by introducing the momentum mechanism, especially in the replicator equation and Euclidean gradient descent. Specifically, the author combines the concept of momentum in machine learning with evolutionary dynamics and explores the role of momentum as an inter - generational memory mechanism.
### Main problems:
1. **Accelerating convergence**: The paper shows how momentum accelerates the convergence of evolutionary dynamics, including the replicator equation and Euclidean gradient descent. Using information divergence as a Lyapunov function, it is proved that when there is an evolutionarily stable state, for a small learning rate or a small momentum value, momentum can accelerate convergence.
2. **Changing convergence properties**: Momentum can also change the convergence characteristics of these dynamic systems. For example, in a "rock - paper - scissors" - like landscape, momentum can break the cyclic behavior, causing the system to either converge to an equilibrium point that is usually not absorbing or diverge, depending on the value and mechanism of momentum.
### Key content:
- **Definition of momentum**: Momentum is a memory mechanism that brings previous gradient information into subsequent iterations, thereby accelerating convergence. In machine learning, momentum can be achieved through Polyak momentum or Nesterov momentum.
- **Lyapunov stability**: The paper proves that for small momentum values, the KL divergence is still a local Lyapunov function of the replicator equation with momentum, and the Euclidean distance is a local Lyapunov function of the projected dynamics with momentum.
- **Convergence time**: Momentum can significantly reduce the time required to reach an evolutionarily stable state. Specifically, for a momentum value of 0 < β < 1, the convergence speed is faster than in the case without momentum, while for β < 0, the convergence speed will be slower.
### Application background:
- **Biological and cultural systems**: The momentum mechanism can explain short - term memory phenomena in biological and cultural systems, such as epigenetics and cultural information transfer. This mechanism may accelerate the process of natural selection and help explain the origin and persistence of certain extra - genetic information.
### Conclusion:
By introducing momentum, evolutionary dynamics can accelerate convergence in the probability simplex, similar to gradient descent in machine learning. The use of the Lyapunov method enables the author to clearly show how momentum reduces the convergence time and maintains the stability of the system under specific conditions. In addition, momentum can also change the convergence characteristics of certain dynamic systems, such as breaking the cyclic behavior in the "rock - paper - scissors" landscape.
### Formula summary:
- **Discrete replicator equation**:
\[
x'_i = x_i \frac{f_i(x)}{\bar{f}(x)}
\]
\[
x^\Delta_i = \frac{x'_i - x_i}{\alpha} = x_i \left( \frac{f_i(x) - \bar{f}(x)}{\bar{f}(x)} \right)
\]
- **Discrete replicator equation with momentum**:
\[
z'_i = \beta z_i + F_i(x)
\]
\[
x^\Delta_i = z'_i
\]
where \( F_i = x_i \left( \frac{f_i(x) - \bar{f}(x)}{\bar{f}(x)} \right) \).
- **Continuous replicator equation**:
\[
\dot{x}_i = x_i (f_i(x) - \bar{f}(x))
\]
- **Continuous replicator equation with momentum**:
\[
\frac{d x_i}{d t} = \frac{1}{1 - \beta} x_i \left( \frac{f_i(x) - \bar{f}(x)}{\bar{f}(x)} \right)
\]
Through these formulas, the paper shows how momentum affects the convergence speed and direction of evolutionary dynamics.