Abstract:The real-time control (RTC) of urban drainage systems can make full use of the capabilities of existing infrastructures to mitigate combined sewer overflow (CSO) and urban flooding. Despite the benefits of RTC, it may encounter potential risks and failures, which need further consideration to enhance its robustness. Besides failures of hardware components such as sensors and actuators, the RTC performance is also sensitive to communication failures between the devices that are spatially distributed in a catchment-scale system. This paper proposes a decentralized control strategy based on multi-agent reinforcement learning to enhance communication robustness and coordinate the decentralized control agents through centralized training. To investigate different control structures, a centralized and a fully decentralized strategy are also developed based on reinforcement learning (RL) for comparison. A benchmark drainage model and a real-world drainage model are formulated as two cases, and the control agents are trained to control the orifices or pumps for CSO or flooding mitigation in each case. The three RL strategies reduce the CSO volume by 5.62-9.30% compared with a static baseline in historical rainfalls of the benchmark case and reduce the CSO and flooding volume by 14.39-21.36% compared with currently-used rule-based control in synthetic rainfalls of the real-world case. Benefitting from centralized training, the decentralized agents can achieve similar performance to the centralized agent. The decentralized control also enhances the communication robustness with smaller performance loss than the centralized control when observation communication fails, and provides a robust backup at the local level to limit the uncertainties when action commands from the centralized agent are lost. The results and findings indicate that multi-agent RL contributes to a coordinated and robust solution for RTC of urban drainage systems.

Real-time control of urban drainage systems using neuro-evolution

Stochastic Model Predictive Control for Urban Traffic Networks

Towards Coordinated and Robust Real-Time Control: a Decentralized Approach for Combined Sewer Overflow and Urban Flooding Reduction Based on Multi-Agent Reinforcement Learning

Combined Sewer Overflow and Flooding Mitigation Through a Reliable Real‐Time Control Based on Multi‐Reinforcement Learning and Model Predictive Control

Real-time multi-objective optimization of pump scheduling in water distribution networks using neuro-evolution

Enhancing the Effectiveness of Urban Drainage System Design with an Improved ACO-based Method

Offline Optimization of Sluice Control Rules in the Urban Water System for Flooding Mitigation

Towards the long term implementation of real time control of combined sewer systems: a review of performance and influencing factors

Regulatory Implications of Integrated Real-Time Control Technology under Environmental Uncertainty

Flood mitigation in coastal urban catchments using real-time stormwater infrastructure control and reinforcement learning

Real-time Control Enhanced Blue-Green Infrastructure Towards Torrential Events: A Smart Predictive Solution

A Nonlinear Predictive Control Approach for Urban Drainage Networks Using Data-Driven Models and Moving Horizon Estimation

Combine Evolutionary Optimization with Model Predictive Control in Real-time Flood Control of a River System

Real time control of nature-based solutions: Towards Smart Solutions and Digital Twins in Rangsit Area, Thailand

Optimize the Real-Time Operation Strategy of Urban Reservoirs in Order to Reduce Flooding

Online Sequential Extreme Learning Machine Based Adaptive Control for Wastewater Treatment Plant

Comparison of Real-time Control Methods for CSO Reduction with Two Evaluation Indices: Computing Load Rate and Double Baseline Normalized Distance

A Feedback Simulation Procedure for Real-time Control of Urban Drainage Systems

Machine learning-based surrogate model assisting stochastic model predictive control of urban drainage systems

Design and Evaluation of Control Strategies in Urban Drainage Systems in Kunming City

Quantifying the true potential of Real Time Control in urban drainage systems