Asynchronous Stochastic Approximation Algorithms for Networked Systems: Regime-Switching Topologies and Multiscale Structure.

G. Yin,Quan Yuan,Le Yi Wang

DOI: https://doi.org/10.1137/120871614

2013-01-01

Abstract:This work develops asynchronous stochastic approximation (SA) algorithms for networked systems with multiagents and regime-switching topologies to achieve consensus control. There are several distinct features of the algorithms: (1) In contrast to most of the existing consensus algorithms, the participating agents compute and communicate in an asynchronous fashion without using a global clock. (2) The agents compute and communicate at random times. (3) The regime-switching process is modeled as a discrete-time Markov chain with a finite state space. (4) The functions involved are allowed to vary with respect to time; hence, nonstationarity can be handled. (5) Multiscale formulation enriches the applicability of the algorithms. In the setup, the switching process contains a rate parameter epsilon > 0 in the transition probability matrix that characterizes how frequently the topology switches. The algorithm uses a step-size mu that defines how fast the network states are updated. Depending on their relative values, three distinct scenarios emerge. Under suitable conditions, it is shown that a continuous-time interpolation of the iterates converges weakly either to a system of randomly switching ordinary differential equations modulated by a continuous-time Markov chain or to a system of differential equations (an average with respect to certain measure). In addition, a scaled sequence of tracking errors converges to either a switching diffusion or a diffusion. Simulation results are presented to demonstrate these findings.