An Influent Responsive Control Strategy with Machine Learning: Q-learning Based Optimization Method for a Biological Phosphorus Removal System.

Ji-Wei Pang,Shan-Shan Yang,Lei He,Yi-Di Chen,Guang-Li Cao,Lei Zhao,Xin-Yu Wang,Nan-Qi Ren
DOI: https://doi.org/10.1016/j.chemosphere.2019.06.103
IF: 8.8
2019-01-01
Chemosphere
Abstract:Biological phosphorus removal (BPR) is an economical and sustainable processes for the removal of phosphorus (P) from wastewater, achieved by recirculating activated sludge through anaerobic and aerobic (An/Ae) processes. However, few studies have systematically analyzed the optimal hydraulic retention times (HRTs) in anaerobic and aerobic reactions, or whether these are the most appropriate control strategies. In this study, a novel optimization methodology using an improved Q-learning (QL) algorithm was developed, to optimize An/Ae HRTs in a BPR system. A framework for QL-based BPR control strategies was established and the improved Q function, Qt+1(st,st+1)=Qt(st,st+1)+k·[R(st,st+1)+γ·maxatQt(st,st+1)-Qt(st,st+1)] was derived. Based on the improved Q function and the state transition matrices obtained under different HRT step-lengths, the optimum combinations of HRTs in An/Ae processes in any BPR system could be obtained, in terms of the ordered pair combinations of the . Model verification was performed by applying six different influent chemical oxygen demand (COD) concentrations, varying from 150 to 600 mg L-1 and influent P concentrations, varying from 12 to 30 mg L-1. Superior and stable effluent qualities were observed with the optimal control strategies. This indicates that the proposed novel QL-based BPR model performed properly and the derived Q functions successfully realized real-time modelling, with stable optimal control strategies under fluctuant influent loads during wastewater treatment processes.
What problem does this paper attempt to address?