Abstract:This paper introduces a model-free optimization method based on reinforcement learning (RL) aimed at resolving the issues of active power and frequency oscillations present in a traditional virtual synchronous generator (VSG). The RL agent utilizes the active power and frequency response of the VSG as state information inputs and generates actions to adjust the virtual inertia and damping coefficients for an optimal response. Distinctively, this study incorporates a setting-time term into the reward function design, alongside power and frequency deviations, to avoid prolonged system transients due to over-optimization. The soft actor critic (SAC) algorithm is utilized to determine the optimal strategy. SAC, being model-free with fast convergence, avoids policy overestimation bias, thus achieving superior convergence results. Finally, the proposed method is validated through MATLAB/Simulink simulation. Compared to other approaches, this method more effectively suppresses oscillations in active power and frequency and significantly reduces the setting time.

What problem does this paper attempt to address?

Based on the provided paper abstract and partial content, the problems that the paper attempts to solve can be summarized as follows: **Problems the paper attempts to solve**: As environmental pollution and energy shortage problems become increasingly severe, renewable energy power generation technologies represented by wind energy and photovoltaics have received extensive attention. However, these renewable energies are usually connected to the grid through power - electronic converters and lack the inertia and damping characteristics of traditional synchronous generators (SGs), which pose significant challenges to the operation, security, and stability of the grid. To address this problem, researchers introduced the virtual synchronous generator (VSG) control technology. This technology provides inertia characteristics for inverters by simulating the rotor motion equation of SGs to enhance grid stability. Although VSG introduces inertia, it also inherits the low - frequency oscillation problem of SGs. In response to the above problems, this paper proposes a model - free optimization method based on reinforcement learning (RL) to solve the active power and frequency oscillation problems existing in traditional VSGs. Specifically, the RL agent uses the active power and frequency responses of VSGs as state information inputs and generates actions to adjust the virtual inertia and damping coefficients to achieve the optimal response. In particular, this paper adds a settling - time term in the reward function design, considering it together with the active power and frequency deviations, to avoid an overly long system transient process due to excessive optimization. This paper uses the Soft Actor Critic (SAC) algorithm to determine the optimal policy. The SAC algorithm is model - free and has a fast convergence speed, and can avoid policy overestimation bias, thereby achieving better convergence results. **Main contributions**: 1. Transform the optimal adaptive control problem of VSGs into a reinforcement learning task, thereby eliminating the need for complex mathematical models and expert knowledge. Subsequently, use the state - of - the - art SAC algorithm to train the agent so that it can discover the optimal policy. 2. The traditional VSG optimization objective only focuses on reducing active power and frequency fluctuations and ignores the optimization of the system transient response time. This paper introduces an adjustment time component in the reward function design to encourage the agent to further optimize the strategy and improve system performance. Through the above methods, this paper aims to effectively suppress the active power and frequency oscillations in VSGs and significantly reduce the settling time, thereby enhancing the overall performance and stability of the system.

Adaptive Control for Virtual Synchronous Generator Parameters Based on Soft Actor Critic

Deep Q-Network Based Adaptive Robustness Parameters for Virtual Synchronous Generator

Deep Reinforcement Learning Based Adaptive Control Strategy for Virtual Synchronous Generator

Rotor Inertia Adaptive Control and Inertia Matching Strategy Based on Parallel Virtual Synchronous Generators System

A novel reinforcement learning policy optimization based adaptive VSG control technique for improved frequency stabilization in AC microgrids

Data-driven Optimal Control Strategy for Virtual Synchronous Generator via Deep Reinforcement Learning Approach

Adaptive Control Strategies for Improving Frequency Response Parameters in VSG

Deep Reinforcement Learning Based Parameter Self-Tuning Control Strategy for VSG

Online Volt-Var Control Based on Reinforcement Learning in Active Distribution Systems Considering PV Inverters

Improved Adaptive Inertia Control of VSG for Low Frequency Oscillation Suppression

A Distributed Dynamic Inertia-droop Control Strategy Based on Multi-Agent Deep Reinforcement Learning for Multiple Paralleled VSGs

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

Optimal Frequency Control for Virtual Synchronous Generator Based AC Microgrids via Adaptive Dynamic Programming

Adaptive active and reactive power control strategy for virtual synchronous generator

Adaptive damping control strategy of virtual synchronous generator for frequency oscillation suppression

VSG-Based Adaptive Optimal Frequency Regulation for AC Microgrids With Nonlinear Dynamics

A Consistent Dynamic Response Control Strategy for Virtual Synchronous Generator

RBF Neural Network Based Virtual Synchronous Generator Control With Improved Frequency Stability

An Adaptive Control Strategy for Virtual Synchronous Generator to Damp Power System Low Frequency Oscillation

An adaptive control strategy with a mutual damping term for paralleled virtual synchronous generators system

Research on Control Strategy of Virtual Synchronous Generator Based on Self-Adaptive Inertia and Damping