Abstract:This paper presents the application and design of a novel stochastic optimal control methodology based on the Q-learning method for solving the automatic generation control (AGC) under the new control performance standards (CPS) for the North American Electric Reliability Council (NERC). The aims of CPS are to relax the control constraint requirements of AGC plant regulation and enhance the frequency dispatch support effect from interconnected control areas. The NERC's CPS-based AGC problem is a dynamic stochastic decision problem that can be modeled as a reinforcement learning (RL) problem based on the Markov decision process theory. In this paper, the Q-learning method is adopted as the RL core algorithm with CPS values regarded as the rewards from the interconnected power systems; the CPS control and relaxed control objectives are formulated as immediate reward functions by means of a linear weighted aggregative approach. By regulating a closed-loop CPS control rule to maximize the long-term discounted reward in the procedure of online learning, the optimal CPS control strategy can be gradually obtained. This paper also introduces a practical semisupervisory group prelearning method to improve the stability and convergence ability of Q-learning controllers during the prelearning process. Tests on the China Southern Power Grid demonstrate that the proposed control strategy can effectively enhance the robustness and relaxation property of AGC systems while CPS compliances are ensured. DOI:10.1061/(ASCE)EY.1943-7897.0000017. (C) 2011 American Society of Civil Engineers.

Q-learning-based Dynamic Optimal Allocation Algorithm for CPS Order of Interconnected Power Grids

Q-learning Based Dynamic Optimal CPS Control Methodology for Interconnected Power Systems

Multi-objective Dynamic Optimal Dispatch Method for CPS Order of Interconnected Power Grids Using Improved Hierarchical Reinforcement Learning

Stochastic Optimal CPS Relaxed Control Methodology for Interconnected Power Systems Using Q-Learning Method

Optimal CPS Command Dispatch Based on Hierarchically Correlated Equilibrium Reinforcement Learning

Multi-step backtrack Q-learning based dynamic optimal algorithm for auto generation control order dispatch

A Shared Control Approach for Muti-Area Interconnected Power System Via Operational Behaviors Learning

A Reinforcement Learning Approach to Dynamic Optimization of Load Allocation in AGC System

Q-Learning Approach for Hierarchical Agc Scheme of Interconnected Power Grids

A Novel Self-Tuning Cps Controller Based On Q-Learning Method

A Deep Reinforcement Learning Algorithm for the Power Order Optimization Allocation of AGC in Interconnected Power Grids

Hierarchical Correlated Q-Learning For Multi-Layer Optimal Generation Command Dispatch

Dynamic Optimization Model Of Agc Strategy Under Cps For Interconnected Power System

Reinforcement learning based CPS self-tuning control methodology for interconnected power systems

Multi-regional Reactive Power Optimization Based on Correlated Equilibrium Q-learning Collaborative Algorithm

Stochastic Optimal Generation Command Dispatch Based on Improved Hierarchical Reinforcement Learning Approach

An AGC Dynamic Optimization Method Based on Proximal Policy Optimization

Automatic Generation Control Based on Multiple-step Greedy Attribute and Multiple-level Allocation Strategy

Distributed $Q$ -Learning-based Online Optimization Algorithm for Unit Commitment and Dispatch in Smart Grid

A deep reinforcement learning algorithm for the order optimization allocation of total power in the interconnected power grids