Abstract:The reinforcement learning (RL)-based generation control strategies have been widely studied to address the limited adaptability of traditional automatic generation control (AGC) strategies to the load disturbance problem resulting from heterogeneous energy sources. To improve the control accuracy of the RL-based strategy in load frequency control (LFC), a double deep Qnetwork combined with an upper confidence bound (DDQN-UCB)-based strategy is designed to solve the problem of agent decision-making in a nonlinear environment. Firstly, the area control error (ACE) and control performance standard 1 (CPS1) of the LFC power system are considered in the design of the RL reward function. Secondly, the actual and estimated Q-values are calculated using the Q-network and the target Q-network combined with the reward value. Thirdly, the deviation loss of the two Q-values is calculated, and the network is updated based on the loss value using gradient descent. Finally, the UCB algorithm is introduced to equalize the frequency of being selected for each action during the random exploration of the actions, and the agent uses the greedy algorithm in combination with the UCB algorithm to select a power-compensated control action to send to the environment. In this paper, the IEEE multi-area LFC power system is used as an experimental validation model. A comparison of the proposed RL control algorithm with five other algorithms revealed that the pre-learning convergence accuracy was improved by 57.5%. Furthermore, the LFC effectiveness test demonstrated that the DDQN-UCB control strategy enhances LFC accuracy while simultaneously stabilizing the power exchange of the inter-area tie-line to within 1.8972 MW, thereby maintaining the stability of the power system.

Application of Q-learning Approach with Prior Knowledge to Non-linear AGC System

A Shared Control Approach for Muti-Area Interconnected Power System Via Operational Behaviors Learning

Q-Learning Approach for Hierarchical Agc Scheme of Interconnected Power Grids

A Novel Self-Tuning Cps Controller Based On Q-Learning Method

Application of Reinforcement Learning Method in a Hydro-thermal Hybrid Automatic Generation Control System

Q-learning Based Dynamic Optimal CPS Control Methodology for Interconnected Power Systems

A Reinforcement Learning Approach to Dynamic Optimization of Load Allocation in AGC System

A prior knowledge-embedded reinforcement learning method for real-time active power corrective control in complex power systems

Stochastic Optimal CPS Relaxed Control Methodology for Interconnected Power Systems Using Q-Learning Method

Q-learning-based Dynamic Optimal Allocation Algorithm for CPS Order of Interconnected Power Grids

Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step Q(λ) learning

Multi-step backtrack Q-learning based dynamic optimal algorithm for auto generation control order dispatch

Nonlinear Model Based Control with Prior Knowledge Based Learning

Hierarchical Correlated Q-Learning For Multi-Layer Optimal Generation Command Dispatch

Reinforcement learning based CPS self-tuning control methodology for interconnected power systems

Adaptive guaranteed cost control for nonlinear systems with unknown parameters and time delays based on fully actuated system approaches

Stochastic Optimal Generation Command Dispatch Based on Improved Hierarchical Reinforcement Learning Approach

Q-Learning-Based Model Predictive Control for Nonlinear Continuous-Time Systems

A Load Frequency Control Strategy Based on Double Deep Q-network and Upper Confidence Bound Algorithm of Multi-Area Interconnected Power Systems

Artificial emotionnal Q-learning for automatic generation control of interconnected power grids

Fuzzy H∞ Control of Discrete-Time Nonlinear Markov Jump Systems via a Novel Hybrid Reinforcement Q-Learning Method