The Oracle Complexity of Simplex-based Matrix Games: Linear Separability and Nash Equilibria

Guy Kornowski,Ohad Shamir
2024-12-10
Abstract:We study the problem of solving matrix games of the form $\max_{\mathbf{w}\in\mathcal{W}}\min_{\mathbf{p}\in\Delta}\mathbf{p}^{\top}A\mathbf{w}$, where $A$ is some matrix and $\Delta$ is the probability simplex. This problem encapsulates canonical tasks such as finding a linear separator and computing Nash equilibria in zero-sum games. However, perhaps surprisingly, its inherent complexity (as formalized in the standard framework of oracle complexity [Nemirovski and Yudin, 1983]) is not well-understood. In this work, we first identify different oracle models which are implicitly used by prior algorithms, amounting to multiplying the matrix $A$ by a vector from either one or both sides. We then prove complexity lower bounds for algorithms under both access models, which in particular imply a separation between them. Specifically, we start by proving that algorithms for linear separability based on one-sided multiplications must require $\Omega(\gamma_A^{-2})$ iterations, where $\gamma_A$ is the margin, as matched by the Perceptron algorithm. We then prove that accelerated algorithms for this task, which utilize multiplications from both sides, must require $\tilde{\Omega}(\gamma_{A}^{-2/3})$ iterations, establishing the first oracle complexity barrier for such algorithms. Finally, by adapting our lower bound to $\ell_1$ geometry, we prove that computing an $\epsilon$-approximate Nash equilibrium requires $\tilde{\Omega}(\epsilon^{-2/5})$ iterations, which is an exponential improvement over the previously best-known lower bound due to Hadiji et al. [2024].
Computer Science and Game Theory,Machine Learning,Optimization and Control
What problem does this paper attempt to address?
The core problem that this paper attempts to solve is to understand and quantify the complexity of solving matrix game problems, especially the linear separability of the probability simplex and Nash equilibrium computation. Specifically, the author has studied matrix game problems in the following form: \[ \max_{w \in W} \min_{p \in \Delta} p^\top A w \] where \( A \) is a certain matrix and \( \Delta \) is the probability simplex. This problem encompasses classic tasks such as finding linear classifiers and computing Nash equilibria in zero - sum games. However, despite the long history and great importance of these problems, their inherent complexity (formalized through the standard oracle complexity framework) has not been fully understood. ### Main Contributions 1. **One - sided and Two - sided Oracle Models**: - The author first identified and formalized the different oracle models implicitly used by existing algorithms. One oracle model allows querying the rows of matrix \( A \) and "one - sided" multiplication queries \( w \mapsto Aw \), while the stronger oracle model allows "two - sided" multiplication queries \( (p, w) \mapsto (p^\top A, Aw) \). 2. **Lower Bounds on the Complexity of Linear Separability**: - For deterministic algorithms using one - sided oracles, the author proved that in the worst - case, the number of iterations required to find a linear classifier is \( \Omega(\gamma_A^{-2}) \), where \( \gamma_A \) is the margin. This shows the optimality of the perceptron algorithm under this oracle model. - For accelerated algorithms using two - sided oracles, the author proved that in the worst - case, the number of iterations required to find a linear classifier is \( \tilde{\Omega}(\gamma_A^{-2/3}) \). This is the first oracle complexity lower bound applicable to accelerated algorithms. 3. **Lower Bounds on the Complexity of Nash Equilibrium**: - The author also proved that in the two - sided oracle model, the number of oracle calls required for any deterministic algorithm to find an \( \epsilon \)-approximate Nash equilibrium is \( \tilde{\Omega}(\epsilon^{-2/5}) \). This result represents an exponential improvement over the previously best lower bound. ### Formula Summary - Lower bound on the one - sided oracle complexity of the linear separability problem: \[ \Omega(\gamma_A^{-2}) \] - Lower bound on the two - sided oracle complexity of the linear separability problem: \[ \tilde{\Omega}(\gamma_A^{-2/3}) \] - Lower bound on the two - sided oracle complexity of the Nash equilibrium problem: \[ \tilde{\Omega}(\epsilon^{-2/5}) \] These results not only deepen our understanding of these classic problems but also provide theoretical guidance for designing more efficient algorithms.