Abstract:We consider a class of nonsmooth aggregative games over networks in stochastic regimes, where each player is characterized by a composite cost function $f_i+r_i$, $f_i$ is a smooth expectation-valued function dependent on its own strategy and an aggregate function of rival strategies, and $r_i$ is a nonsmooth convex function of its strategy with an efficient prox-evaluation. We design a fully distributed iterative proximal stochastic gradient method overlaid by a Tikhonov regularization, where each player may independently choose its steplengths and regularization parameters while meeting some coordination requirements. Under a monotonicity assumption on the pseudo-gradient mapping, we prove the almost sure convergence to the least-norm Nash equilibrium. In addition, when each $r_i$ is an indicator function of a compact convex set, we establish the convergence rate associated with the expected gap function at the time-averaged sequence. We further establish high probability bounds for the gap function via both Markov's inequality as well as a more refined argument that leverages Azuma's inequality. Furthermore, we consider the extension to the private hierarchical regime where each player is a leader with respect to a collection of private followers competing in a strongly monotone game, parametrized by leader decisions. By leveraging a convolution-smoothing framework, we present amongst the first fully distributed schemes for computing a Nash equilibrium of a game complicated by such a hierarchical structure. Based on this framework, we extend the rate statements to accommodate the computation of a hierarchical stochastic Nash equilibrium by using a Fitzpatrick gap function. Finally, we validate the proposed methods on a networked Nash-Cournot equilibrium problem and a hierarchical generalization, observing that regularization has a beneficial impact on empirical behavior.

A Deep Learning Method for Optimal Investment Under Relative Performance Criteria Among Heterogeneous Agents

Optimal investment in a large population of competitive and heterogeneous agents

Multi-Agent Relative Investment Games in a Jump Diffusion Market with Deep Reinforcement Learning Algorithm

Convergence of deep fictitious play for stochastic differential games

Stochastic Delay Differential Games: Financial Modeling and Machine Learning Algorithms

Dynamic Graph-based Deep Reinforcement Learning with Long and Short-term Relation Modeling for Portfolio Optimization.

Many-player games of optimal consumption and investment under relative performance criteria

Constrained portfolio game with heterogeneous agents

Deep Fictitious Play for Finding Markovian Nash Equilibrium in Multi-Agent Games

Convex Markov Games: A Framework for Fairness, Imitation, and Creativity in Multi-Agent Learning

Finding Mixed Strategy Nash Equilibrium for Continuous Games through Deep Learning

Optimal consumption and investment under relative performance criteria with Epstein-Zin utility

A Unified Perspective on Deep Equilibrium Finding

Graphon Mean Field Games with a Representative Player: Analysis and Learning Algorithm

Learning Graphon Mean Field Games and Approximate Nash Equilibria

Stochastic Graphon Games with Memory

Nash Equilibrium Seeking Algorithm Design for Distributed Nonsmooth Multicluster Games Over Weight-Balanced Digraphs

A hybrid deep learning method for finite-horizon mean-field game problems

Neural Payoff Machines: Predicting Fair and Stable Payoff Allocations Among Team Members

Finite-Agent Stochastic Differential Games on Large Graphs: I. The Linear-Quadratic Case

A Distributed Iterative Tikhonov Method for Networked Monotone Aggregative Hierarchical Stochastic Games