Abstract:In order for agents in multi-agent systems (MAS) to be safe, they need to take into account the risks posed by the actions of other agents. However, the dominant paradigm in game theory (GT) assumes that agents are not affected by risk from other agents and only strive to maximise their expected utility. For example, in hybrid human-AI driving systems, it is necessary to limit large deviations in reward resulting from car crashes. Although there are equilibrium concepts in game theory that take into account risk aversion, they either assume that agents are risk-neutral with respect to the uncertainty caused by the actions of other agents, or they are not guaranteed to exist. We introduce a new GT-based Risk-Averse Equilibrium (RAE) that always produces a solution that minimises the potential variance in reward accounting for the strategy of other agents. Theoretically and empirically, we show RAE shares many properties with a Nash Equilibrium (NE), establishing convergence properties and generalising to risk-dominant NE in certain cases. To tackle large-scale problems, we extend RAE to the PSRO multi-agent reinforcement learning (MARL) framework. We empirically demonstrate the minimum reward variance benefits of RAE in matrix games with high-risk outcomes. Results on MARL experiments show RAE generalises to risk-dominant NE in a trust dilemma game and that it reduces instances of crashing by 7x in an autonomous driving setting versus the best performing baseline.

Learning of Nash Equilibria in Risk-Averse Games

Risk-Averse No-Regret Learning in Online Convex Games

A Risk-Averse Equilibrium for Multi-Agent Systems

Asymmetric Feedback Learning in Online Convex Games

Taming Equilibrium Bias in Risk-Sensitive Multi-Agent Reinforcement Learning

Risk-Averse Equilibrium for Games

Risk-averse Learning with Non-Stationary Distributions

Tractable Equilibrium Computation in Markov Games through Risk Aversion

Risk-averse learning with delayed feedback

Bandit Learning in Convex Non-Strictly Monotone Games

Model and Reinforcement Learning for Markov Games with Risk Preferences

Convergent Learning Algorithms for Unknown Reward Games

Risk Preferences of Learning Algorithms

Inverse learning of black-box aggregator for robust Nash equilibrium

Model-Based Reinforcement Learning for Offline Zero-Sum Markov Games

Optimal Bayesian Equilibrium for N-Person Credibilistic Non-Cooperative Game with Risk Aversion.

Convergence of Policy Gradient Methods for Nash Equilibria in General-sum Stochastic Games

On Convergence Rates of Robust Adaptive Game Theoretic Learning Algorithms

A Game-Theoretic Framework for Managing Risk in Multi-Agent Systems

A unified stochastic approximation framework for learning in games

Provably Efficient CVaR RL in Low-rank MDPs