Abstract:This paper presents the application and design of a novel stochastic optimal control methodology based on the Q-learning method for solving the automatic generation control (AGC) under the new control performance standards (CPS) for the North American Electric Reliability Council (NERC). The aims of CPS are to relax the control constraint requirements of AGC plant regulation and enhance the frequency dispatch support effect from interconnected control areas. The NERC's CPS-based AGC problem is a dynamic stochastic decision problem that can be modeled as a reinforcement learning (RL) problem based on the Markov decision process theory. In this paper, the Q-learning method is adopted as the RL core algorithm with CPS values regarded as the rewards from the interconnected power systems; the CPS control and relaxed control objectives are formulated as immediate reward functions by means of a linear weighted aggregative approach. By regulating a closed-loop CPS control rule to maximize the long-term discounted reward in the procedure of online learning, the optimal CPS control strategy can be gradually obtained. This paper also introduces a practical semisupervisory group prelearning method to improve the stability and convergence ability of Q-learning controllers during the prelearning process. Tests on the China Southern Power Grid demonstrate that the proposed control strategy can effectively enhance the robustness and relaxation property of AGC systems while CPS compliances are ensured. DOI:10.1061/(ASCE)EY.1943-7897.0000017. (C) 2011 American Society of Civil Engineers.

Multi-step backtrack Q-learning based dynamic optimal algorithm for auto generation control order dispatch

Q-learning-based Dynamic Optimal Allocation Algorithm for CPS Order of Interconnected Power Grids

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

Q-learning Based Dynamic Optimal CPS Control Methodology for Interconnected Power Systems

Multi-objective Dynamic Optimal Dispatch Method for CPS Order of Interconnected Power Grids Using Improved Hierarchical Reinforcement Learning

Q-Learning Approach for Hierarchical Agc Scheme of Interconnected Power Grids

Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step Q(λ) learning

Hierarchical Correlated Q-Learning For Multi-Layer Optimal Generation Command Dispatch

Optimal CPS Command Dispatch Based on Hierarchically Correlated Equilibrium Reinforcement Learning

Reactive Power Optimization Calculation Based On Multi-Step Q(Lambda) Learning Algorithm

A Novel Self-Tuning Cps Controller Based On Q-Learning Method

A Reinforcement Learning Approach to Dynamic Optimization of Load Allocation in AGC System

Optimal power flow for complex power grid using distributed multi-step backtrack Q(λ) learning

Stochastic Optimal CPS Relaxed Control Methodology for Interconnected Power Systems Using Q-Learning Method

Multi-Objective Optimal Power Flow Calculation Based on Multi-Step Q(λ) Learning Algorithm

Optimal Control Method of PSS Based on Multi-Step Backtrack Q(λ) Learning

Collaborative Consensus Transfer Q-learning Based Dynamic Generation Dispatch of Automatic Generation Control With Virtual Generation Tribe

Stochastic Optimal Generation Command Dispatch Based on Improved Hierarchical Reinforcement Learning Approach

Distributed Multi-Step Q(λ) Learning for Optimal Power Flow of Large-Scale Power Grids

Q-Learning Based Dynamic Optimal Relax Automatic Generation Control