Learning Optimal Scheduling Policy for Remote State Estimation Under Uncertain Channel Condition

Shuang Wu,Xiaoqiang Ren,Qing-Shan Jia,Karl Henrik Johansson,Ling Shi
DOI: https://doi.org/10.1109/tcns.2019.2959162
IF: 4.347
2020-06-01
IEEE Transactions on Control of Network Systems
Abstract:We consider optimal sensor scheduling with unknown communication channel statistics. We formulate two types of scheduling problems with the communication rate being a soft or hard constraint, respectively. We first present some structural results on the optimal scheduling policy using dynamic programming and assuming that the channel statistics is known. We prove that the $Q$-factor is monotonic and submodular, which leads to thresholdlike structures in both problems. Then, we develop a stochastic approximation and parameter learning frameworks to deal with the two scheduling problems with unknown channel statistics. We utilize their structures to design specialized learning algorithms. We, then prove the convergence of these algorithms. Performance improvement compared with the standard $Q$-learning algorithm is shown through numerical examples, which will also discuss an alternative method based on recursive estimation of the channel quality.
What problem does this paper attempt to address?