Real-time scheduling of renewable power systems through planning-based reinforcement learning

Shaohuai Liu,Jinbo Liu,Weirui Ye,Nan Yang,Guanglun Zhang,Haiwang Zhong,Chongqing Kang,Qirong Jiang,Xuri Song,Fangchun Di,Yang Gao
2023-03-13
Abstract:The growing renewable energy sources have posed significant challenges to traditional power scheduling. It is difficult for operators to obtain accurate day-ahead forecasts of renewable generation, thereby requiring the future scheduling system to make real-time scheduling decisions aligning with ultra-short-term forecasts. Restricted by the computation speed, traditional optimization-based methods can not solve this problem. Recent developments in reinforcement learning (RL) have demonstrated the potential to solve this challenge. However, the existing RL methods are inadequate in terms of constraint complexity, algorithm performance, and environment fidelity. We are the first to propose a systematic solution based on the state-of-the-art reinforcement learning algorithm and the real power grid environment. The proposed approach enables planning and finer time resolution adjustments of power generators, including unit commitment and economic dispatch, thus increasing the grid's ability to admit more renewable energy. The well-trained scheduling agent significantly reduces renewable curtailment and load shedding, which are issues arising from traditional scheduling's reliance on inaccurate day-ahead forecasts. High-frequency control decisions exploit the existing units' flexibility, reducing the power grid's dependence on hardware transformations and saving investment and operating costs, as demonstrated in experimental results. This research exhibits the potential of reinforcement learning in promoting low-carbon and intelligent power systems and represents a solid step toward sustainable electricity generation.
Artificial Intelligence,Machine Learning,Systems and Control
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is the challenges faced by traditional power dispatch systems when dealing with high - proportion renewable energy integration. Specifically, with the increase of renewable energy such as wind and solar energy, the volatility and unpredictability of their power generation have brought great challenges to traditional power dispatch. Traditional power dispatch methods rely on long - term forecasts and day - ahead scheduling plans (Day - Ahead Scheduling, DAS). These methods are difficult to adapt to the short - term changes of renewable energy, resulting in dispatch results that may be inaccurate or infeasible, thus causing problems such as renewable energy curtailment and load shedding. To overcome these problems, the paper proposes a real - time scheduling (Real - Time Scheduling, RTS) method based on reinforcement learning (Reinforcement Learning, RL). This method utilizes ultra - short - term forecasting technology and realizes real - time scheduling decisions through planning - based reinforcement learning algorithms, while taking into account various operational constraints of the power system. This method can generate scheduling plans in a short time, improves the flexibility and accuracy of dispatch, reduces the need for hardware upgrades, lowers investment and operating costs, and helps to realize a low - carbon and intelligent power system. The key innovation points in the paper include: 1. **Real - time Scheduling Framework**: A real - time scheduling framework that combines ultra - short - term forecasting and reinforcement learning is proposed, which can quickly respond to the fluctuations of renewable energy. 2. **Planning - based Reinforcement Learning**: A forward - looking scheduling reinforcement learning method named GridZero is developed. This method draws on the search plus deep - learning framework of the AlphaGo series of work and can make effective scheduling decisions in complex environments. 3. **Safety Assurance**: A safety layer is introduced to prevent the RL agent from causing destructive consequences due to unrestricted operations during the exploration process, ensuring the safety of dispatch. 4. **Simulation Environment**: A highly simulated power grid environment GridSim is constructed for training and testing RL agents to ensure their effectiveness and robustness in practical applications. Through these innovations, the paper demonstrates the potential of RL in promoting low - emission and intelligent power systems and provides a new solution for power dispatch with high - proportion renewable energy integration in the future.