Abstract:We investigate the accuracy of prediction in deterministic learning dynamics of zero-sum games with random initializations, specifically focusing on observer uncertainty and its relationship to the evolution of covariances. Zero-sum games are a prominent field of interest in machine learning due to their various applications. Concurrently, the accuracy of prediction in dynamical systems from mechanics has long been a classic subject of investigation since the discovery of the Heisenberg Uncertainty Principle. This principle employs covariance and standard deviation of particle states to measure prediction accuracy. In this study, we bring these two approaches together to analyze the Follow-the-Regularized-Leader (FTRL) algorithm in two-player zero-sum games. We provide growth rates of covariance information for continuous-time FTRL, as well as its two canonical discretization methods (Euler and Symplectic). A Heisenberg-type inequality is established for FTRL. Our analysis and experiments also show that employing Symplectic discretization enhances the accuracy of prediction in learning dynamics.
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is to explore the prediction accuracy based on deterministic learning dynamics in zero - sum games, especially the relationship between observer uncertainty and covariance evolution under random initialization conditions. Specifically, the research focuses on the following aspects:
1. **Prediction accuracy problem**: How to measure and track the prediction accuracy in learning dynamics, especially under the condition of initial uncertainty.
2. **Performance of the FTRL algorithm**: Study the performance of the Follow - the - Regularized - Leader (FTRL) algorithm in a two - player zero - sum game, especially its covariance growth rate under continuous time and two typical discretization methods (Euler discretization and symplectic discretization).
3. **Analogy of the Heisenberg uncertainty principle**: By introducing the concept similar to the Heisenberg uncertainty principle in quantum mechanics, study the trade - off between the covariance of cumulative strategies and cumulative payoffs.
### Specific problem description
- **Prediction accuracy in zero - sum games**: Zero - sum games are an important research object in the field of machine learning because of their wide applications in various applications. However, predicting the long - term behavior of players in repeated games is a fundamental and challenging problem. The traditional view is that the players' strategies will eventually converge to an equilibrium point, but recent research shows that this belief usually does not hold, especially in FTRL dynamics, which exhibits complex non - convergent behaviors such as periodicity and divergence in zero - sum games.
- **Observer's uncertainty**: In practical applications, the observer may have some uncertainty about the current state of the players. This uncertainty comes from both the perspective of game theory (for example, the unknown influence of external factors on players' preferences) and the perspective of machine learning (for example, sampling from a distribution during system initialization and noise during the training process). Therefore, it is necessary to study how this uncertainty affects the prediction accuracy of future behaviors.
### Solution
The paper studies the covariance evolution of the FTRL algorithm in different situations by introducing the covariance matrix as an uncertainty measure. Specific contributions include:
1. **Proving the limitations of differential entropy**: Prove that under the alternating update rule, the differential entropy remains unchanged, so it cannot effectively capture the uncertainty evolution.
2. **Proposing the covariance matrix as a measurement tool**: Propose the covariance matrix as an uncertainty measure and calculate its growth rate in Euclidean - regularized FTRL.
3. **Establishing the Heisenberg - type inequality**: For generally regularized FTRL, establish the Heisenberg - type inequality between the variances of cumulative strategies and cumulative payoffs, that is, \(\Delta X_{i,\alpha} \Delta y_{i,\alpha} \geq \text{constant}\), which indicates that there is a trade - off in prediction accuracy between the strategy space and the payoff space.
### Conclusion
Through these studies, the paper reveals the advantages of the symplectic discretization method in improving prediction accuracy and provides new perspectives and tools for understanding and improving prediction in learning dynamics.