Abstract:The reinforcement learning (RL)-based generation control strategies have been widely studied to address the limited adaptability of traditional automatic generation control (AGC) strategies to the load disturbance problem resulting from heterogeneous energy sources. To improve the control accuracy of the RL-based strategy in load frequency control (LFC), a double deep Qnetwork combined with an upper confidence bound (DDQN-UCB)-based strategy is designed to solve the problem of agent decision-making in a nonlinear environment. Firstly, the area control error (ACE) and control performance standard 1 (CPS1) of the LFC power system are considered in the design of the RL reward function. Secondly, the actual and estimated Q-values are calculated using the Q-network and the target Q-network combined with the reward value. Thirdly, the deviation loss of the two Q-values is calculated, and the network is updated based on the loss value using gradient descent. Finally, the UCB algorithm is introduced to equalize the frequency of being selected for each action during the random exploration of the actions, and the agent uses the greedy algorithm in combination with the UCB algorithm to select a power-compensated control action to send to the environment. In this paper, the IEEE multi-area LFC power system is used as an experimental validation model. A comparison of the proposed RL control algorithm with five other algorithms revealed that the pre-learning convergence accuracy was improved by 57.5%. Furthermore, the LFC effectiveness test demonstrated that the DDQN-UCB control strategy enhances LFC accuracy while simultaneously stabilizing the power exchange of the inter-area tie-line to within 1.8972 MW, thereby maintaining the stability of the power system.

A Novel Self-Tuning Cps Controller Based On Q-Learning Method

Q-learning Based Dynamic Optimal CPS Control Methodology for Interconnected Power Systems

Reinforcement learning based CPS self-tuning control methodology for interconnected power systems

Stochastic Optimal CPS Relaxed Control Methodology for Interconnected Power Systems Using Q-Learning Method

Q-learning-based Dynamic Optimal Allocation Algorithm for CPS Order of Interconnected Power Grids

A Shared Control Approach for Muti-Area Interconnected Power System Via Operational Behaviors Learning

Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step Q(λ) learning

CPS Statistic Information Self-learning Methodology Based Adaptive Automatic Generation Control

Q-Learning Approach for Hierarchical Agc Scheme of Interconnected Power Grids

Multi-step backtrack Q-learning based dynamic optimal algorithm for auto generation control order dispatch

An Average Reward Model Based Whole Process R(λ)-learning for Optimal CPS Control

Q-Learning Based Dynamic Optimal Relax Automatic Generation Control

Multi-Agent Q-Value Mixing Network with Covariance Matrix Adaptation Strategy for the Voltage Regulation Problem

Optimal CPS Control for Interconnected Power Systems Based on SARSA On-Policy Learning Algorithm

Application of Q-learning Approach with Prior Knowledge to Non-linear AGC System

A Reinforcement Learning Approach to Dynamic Optimization of Load Allocation in AGC System

Optimal CPS Command Dispatch Based on Hierarchically Correlated Equilibrium Reinforcement Learning

Hierarchical Correlated Q-Learning For Multi-Layer Optimal Generation Command Dispatch

A Load Frequency Control Strategy Based on Double Deep Q-network and Upper Confidence Bound Algorithm of Multi-Area Interconnected Power Systems

Artificial emotionnal Q-learning for automatic generation control of interconnected power grids

Multi-objective Dynamic Optimal Dispatch Method for CPS Order of Interconnected Power Grids Using Improved Hierarchical Reinforcement Learning