Abstract:The reinforcement learning (RL)-based generation control strategies have been widely studied to address the limited adaptability of traditional automatic generation control (AGC) strategies to the load disturbance problem resulting from heterogeneous energy sources. To improve the control accuracy of the RL-based strategy in load frequency control (LFC), a double deep Qnetwork combined with an upper confidence bound (DDQN-UCB)-based strategy is designed to solve the problem of agent decision-making in a nonlinear environment. Firstly, the area control error (ACE) and control performance standard 1 (CPS1) of the LFC power system are considered in the design of the RL reward function. Secondly, the actual and estimated Q-values are calculated using the Q-network and the target Q-network combined with the reward value. Thirdly, the deviation loss of the two Q-values is calculated, and the network is updated based on the loss value using gradient descent. Finally, the UCB algorithm is introduced to equalize the frequency of being selected for each action during the random exploration of the actions, and the agent uses the greedy algorithm in combination with the UCB algorithm to select a power-compensated control action to send to the environment. In this paper, the IEEE multi-area LFC power system is used as an experimental validation model. A comparison of the proposed RL control algorithm with five other algorithms revealed that the pre-learning convergence accuracy was improved by 57.5%. Furthermore, the LFC effectiveness test demonstrated that the DDQN-UCB control strategy enhances LFC accuracy while simultaneously stabilizing the power exchange of the inter-area tie-line to within 1.8972 MW, thereby maintaining the stability of the power system.

Regional Power Grid AGC Control Strategy Research Based on Q-Learning Algorithm under WACPB Mode

Multi-regional Reactive Power Optimization Based on Correlated Equilibrium Q-learning Collaborative Algorithm

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Q-Learning Approach for Hierarchical Agc Scheme of Interconnected Power Grids

A Deep Reinforcement Learning Algorithm for the Power Order Optimization Allocation of AGC in Interconnected Power Grids

Research on AGC Performance During Wind Power Ramping Based on Deep Reinforcement Learning.

Power Grid Reactive Voltage Control Method Based on Improved Reinforcement Learning

Automatic Generation Control for Distributed Multi-Region Interconnected Power System with Function Approximation

Q-learning-based Dynamic Optimal Allocation Algorithm for CPS Order of Interconnected Power Grids

A Fast Multi-agent Learning Strategy Base on DWoLF-PHC(λ) for Smart Generation Control of Power Systems

Emergency fault affected wide-area automatic generation control via large-scale deep reinforcement learning

A Load Frequency Control Strategy Based on Double Deep Q-network and Upper Confidence Bound Algorithm of Multi-Area Interconnected Power Systems

Grid-area coordinated load frequency control strategy using large-scale multi-agent deep reinforcement learning

Q-learning Based Dynamic Optimal CPS Control Methodology for Interconnected Power Systems

Automatic generation control for interconnected power grids based on multi-agent correlated equilibrium learning system

Artificial emotionnal Q-learning for automatic generation control of interconnected power grids

Reinforcement learning based CPS self-tuning control methodology for interconnected power systems

A Reinforcement Learning Approach to Dynamic Optimization of Load Allocation in AGC System

Automatic Generation Control Strategy Based on Deep Forest.

Lifelong Learning for Complementary Generation Control of Interconnected Power Grids with High-Penetration Renewables and EVs

Multi-Stage Transmission Line Flow Control Using Centralized and Decentralized Reinforcement Learning Agents