Abstract:$ $This paper addresses the inverse problem for Linear-Quadratic (LQ) nonzero-sum $N$-player differential games, where the goal is to learn parameters of an unknown cost function for the game, called observed, given the demonstrated trajectories that are known to be generated by stationary linear feedback Nash equilibrium laws. Towards this end, using the demonstrated data, a synthesized game needs to be constructed, which is required to be equivalent to the observed game in the sense that the trajectories generated by the equilibrium feedback laws of the $N$ players in the synthesized game are the same as those demonstrated trajectories. We show a model-based algorithm that can accomplish this task using the given trajectories. We then extend this model-based algorithm to a model-free setting to solve the same problem in the case when the system's matrices are unknown. The algorithms combine both inverse optimal control and reinforcement learning methods making extensive use of gradient descent optimization for the latter. The analysis of the algorithm focuses on the proof of its convergence and stability. To further illustrate possible solution characterization, we show how to generate an infinite number of equivalent games, not requiring to run repeatedly the complete algorithm. Simulation results validate the effectiveness of the proposed algorithms.

Linear-Quadratic Dynamic Games as Receding-Horizon Variational Inequalities

Feedback and Open-Loop Nash Equilibria for LQ Infinite-Horizon Discrete-Time Dynamic Games

Linear-quadratic-singular stochastic differential games and applications

Two person non-zero-sum linear-quadratic differential game with Markovian jumps in infinite horizon

Reinforcement Learning for Inverse Linear-quadratic Dynamic Non-cooperative Games

Generalized open-loop Nash equilibria in linear-quadratic difference games with coupled-affine inequality constraints

Inverse linear quadratic dynamic games using partial state observations

Linear convergence in time-varying generalized Nash equilibrium problems

Variational Inequalities on Unbounded Domains for Zero-Sum Singular Controller vs. Stopper Games

Variational inequalities on unbounded domains for zero-sum singular-controller vs. stopper games

Nash equilibria in scalar discrete-time linear quadratic games

Reinforcement Learning for Inverse Non-Cooperative Linear-Quadratic Output-feedback Differential Games

Inverse linear-quadratic nonzero-sum differential games

Stochastic linear-quadratic differential game with Markovian jumps in an infinite horizon

Nash Equilibria for Linear Quadratic Discrete-time Dynamic Games via Iterative and Data-driven Algorithms

Linear quadratic differential game problems with constrained open-loop controls

Linear-Quadratic Non-Zero Sum Backward Stochastic Differential Game With Overlapping Information

Differential Variational Inequality Approach to Dynamic Games with Shared Constraints

Some Remarks on Linear-Quadratic Closed-Loop Games with Many Players