Abstract:Aiming at a cleaner future power system, many regimes in the world have proposed their ambitious decarbonizing plan, with increasing penetration of renewable energy sources (RES) playing an alternative role to conventional energy. As a result, power system tends to have less backup capacity and operate near their designed limit, thus exacerbating system vulnerability against extreme events. Under this reality, resilient control for the multi-energy micro-grid is facing the following challenges, which are: 1) the effect from the stochastic uncertainties of RES; 2) the need for a model-free and fast-reacting control scheme under extreme events; and 3) efficient exploration and robust performance with limited extreme events data. To deal with the aforementioned challenges, this paper proposes a novel Bayesian Deep Reinforcement Learning-based resilient control approach for multi-energy micro-grid. In particular, the proposed approach replaces the deterministic network in traditional Reinforcement Learning with a Bayesian probabilistic network in order to obtain an approximation of the value function distribution, which effectively solves the Q-value overestimation issue. Compared with the naive Deep Deterministic Policy Gradient (DDPG) method and optimization method, the effectiveness and importance of employing the Bayesian Reinforcement Learning approach are investigated and illustrated across different operating scenarios. Case studies have shown that by using the Monte Carlo posterior mean of the Bayesian value function distribution instead of a deterministic estimation, the proposed Bayesian Deep Deterministic Policy Gradient (BDDPG) method achieves a near-optimum policy in a more stable process, which verifies the robustness and the practicability of the proposed approach.

Deep Reinforcement Learning Based Robust Control Strategy for Power Sharing and Voltage Regulation in DC Grid-Forming Energy Storage System

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

A Control Strategy Based on Deep Reinforcement Learning Under the Combined Wind-Solar Storage System

An Intelligent Control Strategy for buck DC-DC Converter via Deep Reinforcement Learning

Deep Reinforcement Learning Based Approach for Active Power Security Correction Control of Power System

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

A Date-Driven Voltage Control Strategy for Distribution Network Using Deep Reinforcement Learning

Deep Reinforcement Learning Based Control Strategy for Voltage Regulation of DC-DC Buck Converter Feeding CPLs in DC Microgrid

Hybrid Energy Storage Control Method For DC Microgrid Based On Deep Deterministic Policy Gradient

Reinforcement Learning-based District Cooling System Control for Voltage Regulation in Distribution Networks.

Enactment of Deep Reinforcement Learning Control for Power Management and Enhancement of Voltage Regulation in a DC Micro-Grid System

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

A Collaborative Control Strategy of DFIG System with Energy Storage in Weak Grid

An optimal solutions-guided deep reinforcement learning approach for online energy storage control

A Deep Reinforcement Learning Voltage Control Method for Distribution Network

Decentralized multi-objective cloud energy storage operation control with deep reinforcement learning

Reinforcement Learning to Stabilize Singularly Perturbed DC-Side Dynamics of Grid-Connected Voltage-Source Converters in Modern AC–DC Grids Using Singular Perturbation Theory and Adaptive Dynamic Programming

Operation of Distributed Battery Considering Demand Response Using Deep Reinforcement Learning in Grid Edge Control

Deep-Reinforcement-Learning-Based Autonomous Voltage Control for Power Grid Operations

Implementing Deep Reinforcement Learning-Based Grid Voltage Control in Real-World Power Systems: Challenges and Insights

Long-term Deep Reinforcement Learning for Real-Time Economic Generation Control of Cloud Energy Storage Systems with Varying Structures