Abstract:This paper presents a novel model-free switching and control method for three-level neutral point clamped (NPC) converter using deep reinforcement learning (DRL). Our approach targets two primary control objectives: voltage balancing and current control. In this method, voltage balancing, current control and selection of optimal switches are achieved using a reward function which is calculated based on various signals measured as observations of the DRL agent. Since the action space is discrete, a deep Q-network (DQN) agent is utilized. DQN is used due to its capability of handling high-dimensional state spaces. In order to highlight its pros and cons, the proposed method is compared with model predictive control (MPC), which is another popular non-linear control method for power electronic converters. The proposed method is evaluated and compared with the MPC method in grid-connected mode using simulations in Matlab/Simulink. To evaluate the practical performance of the DRL method, various experimental results are obtained. The simulation and experimental results demonstrate that the proposed method effectively achieves accurate voltage balancing and ensures steady operation even in the presence of various dynamic changes, including variations in the reference currents and grid voltage. Additionally, the method successfully handles uncertainties, such as sensor measurement noise, and accommodates parameter variations, such as changes in the capacity of the DC-link capacitors and line impedance. The results demonstrate that this method exhibits superior adaptability to real-time changes and uncertainties, delivering more robust performance compared to similar conventional methods like MPC. Thus, this method can be considered a promising approach for intelligent control of power electronic converters, especially when conventional methods such as MPC face challenges in performance and accuracy under severe parameter variations and uncertainties.

Proximal Policy Optimization-Based Reinforcement Learning Approach for DC-DC Boost Converter Control: A Comparative Evaluation Against Traditional Control Techniques

A Digital Dual-Mode Lqr Optimal Controller for Boost Converter over A Wide Operation Range

Digital LQR Steady-State Optimal Control with Feedforward for Nonminimum Phase Boost DC-DC Converter

Dual-Mode Lqr-Feedforward Optimal Control for Non-Minimum Phase Boost Converter

Performance Enhancement of Buck Converter Using Reinforcement Learning Control

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

An Intelligent Control Strategy for buck DC-DC Converter via Deep Reinforcement Learning

Comparative Performance Analysis of the DC-AC Converter Control System Based on Linear Robust or Nonlinear PCH Controllers and Reinforcement Learning Agent

Reinforcement Learning-Based Control of a Power Electronic Converter

Reinforcement learning-based control of improved hybrid current modulated dual active bridge AC/DC converter

Deep Deterministic Policy Gradient Algorithm Based Reinforcement Learning Controller for Single-Inductor Multiple-Output DC–DC Converter

Proximal policy optimization through a deep reinforcement learning framework for remedial action schemes of VSC-HVDC

Reinforcement Learning-Based Event-Triggered FCS-MPC for Power Converters

Combining Data-Driven and Event-Driven for Online Learning Predictive Control in Power Converters

Learning-Based Model Predictive Control of DC-DC Buck Converters in DC Microgrids: A Multi-Agent Deep Reinforcement Learning Approach

A Model-Free Switching and Control Method for Three-Level Neutral Point Clamped Converter Using Deep Reinforcement Learning

Safety-Enhanced Self-Learning for Optimal Power Converter Control

Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System

A New Robust Integral Reinforcement Learning Based Control Algorithm for Interleaved DC/DC Boost Converter

Deep Reinforcement Learning Based Control Strategy for Voltage Regulation of DC-DC Buck Converter Feeding CPLs in DC Microgrid

Deep Reinforcement Learning for Optimizing Inverter Control: Fixed and Adaptive Gain Tuning Strategies for Power System Stability