Adaptive Incentive Design with Learning Agents

Chinmay Maheshwari,Kshitij Kulkarni,Manxi Wu,Shankar Sastry

2024-09-03

Abstract:How can the system operator learn an incentive mechanism that achieves social optimality based on limited information about the agents' behavior, who are dynamically updating their strategies? To answer this question, we propose an \emph{adaptive} incentive mechanism. This mechanism updates the incentives of agents based on the feedback of each agent's externality, evaluated as the difference between the player's marginal cost and society's marginal cost at each time step. The proposed mechanism updates the incentives on a slower timescale compared to the agents' learning dynamics, resulting in a two-timescale coupled dynamical system. Notably, this mechanism is agnostic to the specific learning dynamics used by agents to update their strategies. We show that any fixed point of this adaptive incentive mechanism corresponds to the optimal incentive mechanism, ensuring that the Nash equilibrium coincides with the socially optimal strategy. Additionally, we provide sufficient conditions that guarantee the convergence of the adaptive incentive mechanism to a fixed point. Our results apply to both atomic and non-atomic games. To demonstrate the effectiveness of our proposed mechanism, we verify the convergence conditions in two practically relevant games: atomic networked quadratic aggregative games and non-atomic network routing games.

Computer Science and Game Theory,Multiagent Systems,Systems and Control,Dynamical Systems

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how system operators can design an incentive mechanism to achieve the socially optimal solution while basing the design on limited information about agent behavior. Specifically, the paper focuses on how, in the case where agents dynamically update their strategies, system operators can adjust the incentive mechanism through learning to ensure that the ultimately reached state is socially optimal. The paper proposes an adaptive incentive mechanism that updates agents' incentives according to the externality of each agent (i.e., the difference between the individual marginal cost and the social marginal cost). This mechanism operates on two time scales: a slower time scale for updating incentives and a faster time scale for the learning dynamics of agents. In this way, even when agents continuously update their strategies, system operators can gradually adjust the incentive mechanism and ultimately achieve the socially optimal solution.

The key contribution of the paper lies in proposing an incentive mechanism that can adapt to changes in agent behavior and proving that under certain conditions, this mechanism can converge to a fixed point, which corresponds to the socially optimal incentive mechanism. In addition, the paper also provides sufficient conditions to ensure the convergence of the adaptive incentive mechanism to a fixed point and applies these results to two practically relevant games: atomic network quadratic aggregation games and non - atomic network routing games, verifying the effectiveness of the proposed method.