Abstract:This paper presents approaches to mean-field control, motivated by distributed control of multi-agent systems. Control solutions are based on a convex optimization problem, whose domain is a convex set of probability mass functions (pmfs). The main contributions follow: 1. Kullback-Leibler-Quadratic (KLQ) optimal control is a special case, in which the objective function is composed of a control cost in the form of Kullback-Leibler divergence between a candidate pmf and the nominal, plus a quadratic cost on the sequence of marginals. Theory in this paper extends prior work on deterministic control systems, establishing that the optimal solution is an exponential tilting of the nominal pmf. Transform techniques are introduced to reduce complexity of the KLQ solution, motivated by the need to consider time horizons that are much longer than the inter-sampling times required for reliable control. 2. Infinite-horizon KLQ leads to a state feedback control solution with attractive properties. It can be expressed as either state feedback, in which the state is the sequence of marginal pmfs, or an open loop solution is obtained that is more easily computed. 3. Numerical experiments are surveyed in an application of distributed control of residential loads to provide grid services, similar to utility-scale battery storage. The results show that KLQ optimal control enables the aggregate power consumption of a collection of flexible loads to track a time-varying reference signal, while simultaneously ensuring each individual load satisfies its own quality of service constraints. Keywords: Mean field games, distributed control, Markov decision processes, Demand Dispatch.

Decomposition and Mean-Field Approach to Mixed Integer Optimal Compensation Problems

On Modeling and Optimal Control of a Class of Hybrid Systems

MF-OMO: An Optimization Formulation of Mean-Field Games

Discrete-time indefinite linear-quadratic mean field games and control: The finite-population case

Mixed Social Optima and Nash equilibrium in Linear-Quadratic-Gaussian Mean-field System

Linear Quadratic Gaussian Mean-Field Controls of Social Optima

Unified Framework of Mean-Field Formulations for Optimal Multi-Period Mean-Variance Portfolio Selection

From Nash Equilibrium to Social Optimum and vice versa: a Mean Field Perspective

Hybrid optimal control with mixed-integer Lagrangian methods

A Class of Discrete-time Mean-field Stochastic Linear-quadratic Optimal Control Problems with Financial Application

On Mean Field Monotonicity Conditions from Control Theoretical Perspective

Kullback-Leibler-Quadratic Optimal Control

Indefinite Linear Quadratic Mean Field Social Control Problems With Multiplicative Noise

A Bilevel Optimization Method for Inverse Mean-Field Games

Direct Approach of Indefinite Linear-Quadratic Mean Field Games

Distributed Control of Linear Quadratic Mean Field Social Systems with Heterogeneous Agents

Equilibrium in Functional Stochastic Games with Mean-Field Interaction

Computing Two-Stage Robust Optimization with Mixed Integer Structures

Robust Mean-Field Games with Partial Observations: A Complementary Strategy

Mean field for Markov Decision Processes: from Discrete to Continuous Optimization