Learning andSelectingUsers forAchievingReliability in DemandResponse : AMulti-armedBanditApproach

Yingying Li,Qinran Hu,Na Li

2019-01-01

Abstract:One challenge in the optimization and control of societal systems is to handle the unknown and uncertain user behavior. This paper focuses on residential demand response (DR) and proposes a closed-loop learning scheme to address these issues. In particular, we consider DR programs where an aggregator calls upon residential users to change their demand so that the total load adjustment is close to a target value. To learn and select the right customers, we formulate the DR problem as a combinatorial multi-armed bandit (CMAB) problem with a reliability objective. We propose a learning algorithm: CUCBAvg (Combinatorial Upper Confidence Bound-Average), which utilizes both upper confidence bounds and sample averages to balance the tradeoff between exploration (learning) and exploitation (selecting). We consider both a fixed time-invariant target and time-varying targets and show that CUCB-Avg achieves O(log T ) and O( √ T log(T ) regrets respectively. We further numerically test our algorithms using synthetic realistic DR data and demonstrate that our CUCB-Avg performs significantly better than the classic CUCB and better than Thompson Sampling.