Enabling Efficient Model-Free Control of Large-Scale Canals by Exploiting Domain Knowledge
Tao Ren,Jianwei Niu,Lei Shu,Gerhard P. Hancke,Jiyan Wu,Xuefeng Liu,Mingliang Xu
DOI: https://doi.org/10.1109/tie.2020.3013778
IF: 7.7
2021-09-01
IEEE Transactions on Industrial Electronics
Abstract:Canals are constructed worldwidely to divert water from rich to arid areas to mitigate water shortages. Since water resource is fairly limited, it is essential to perform canal control efficiently to improve water-delivery performance. A promising solution is to leverage model predictive control (MPC), which calculates the desired canal action at each time step via reliable predictions of the model. However, the predictive model dependence degrades the practicability and the iterative calculation incurs intensive computations, especially for large-scale canals with high-dimensional state and action spaces (curse of dimensionality). This article presents a new canal control model named efficient model-free canal control (EMCC) that obtains control policies in a model-free way via deep reinforcement learning (DRL) and alleviates the curse of dimensionality via domain knowledge (control experience). EMCC adopts the hidden Markov model with Gaussian mixture densities (GMM-HMM) to model canal system dynamics with control experience, and initializes it according to the actual operation data. Besides, we design a reward generator collaborated with GMM-HMM to supervise the reinforcement learning around control experiences to obtain more efficient control policies. We evaluate EMCC via numerical simulations on Chinese largest water-delivery project (SNWTP). Experimental results show that EMCC leads to significant convergence performances compared with crude applications of DRL on large-scale canals, and achieves desired objectives more satisfactorily than MPC and control-experience under two typical water-delivery tasks.
automation & control systems,engineering, electrical & electronic,instruments & instrumentation