Abstract:<p>Energy systems undergo major transitions to facilitate the large-scale penetration of renewable energy technologies and improve efficiencies, leading to the integration of many sectors into the energy system domain. As the complexities in this domain increase, it becomes challenging to control energy flows using existing techniques based on physical models. Moreover, although data-driven models, such as reinforcement learning (RL), have gained considerable attention in many fields, a direct shift into RL is not feasible in the energy domain irrespective of the ongoing complexities. To this end, a top-down approach is used to understand this behavior by reviewing the current state of the art.</p><p>We classified RL papers in the literature into seven categories based on their area of application. Subsequently, publications under each category were further examined relative to problem diversity, RL technique employed, performance improvement (compared with other white and gray box models), verification, and reproducibility; many of the articles reported a 10–20% performance improvement with the use of RL. In most studies, however, deep learning techniques and state-of-the-art actor-critic methods (e.g., twin delayed deep deterministic policy gradient and soft actor-critic) were not applied. This has remarkably hindered performance improvements and problems related to complex energy flows have not been considered. Approximately half of the publications reported the use of Q-learning. Furthermore, despite the availability of historical data in the energy system domain, batch RL algorithms have not been exploited. Emerging multi-agent RL applications may be considered as a positive development that can enable the management of complex interactions among multiple parties. Most studies lack proper benchmarking compared to model-based approaches or gray-box models, and a majority cover energy dispatch problems and building energy management. Although RL can adequately solve problems that are considerably integrated in several sectors, only a limited number of publications have discussed its broad application. The present study clearly demonstrates that even without the full utilization of RL capacity, this technique has a considerable potential in resolving the continuously increasing complexity within the energy system domain.</p>

Reinforcement Learning in Adaptive Control of Power System Generation

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

Reinforcement Learning Model-Based and Model-Free Paradigms for Optimal Control Problems in Power Systems: Comprehensive Review and Future Directions

Designing a Fuzzy Q-Learning Power Energy System Using Reinforcement Learning

Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges

Reinforcement Learning for Efficient Power Systems Planning: A Review of Operational and Expansion Strategies

Safe Reinforcement Learning for Power System Control: A Review

Predictive Control of Voltage Source Inverter: an Online Reinforcement Learning Solution

Adaptive Power System Emergency Control using Deep Reinforcement Learning

Reinforcement Learning-Based Intelligent Control Strategies for Optimal Power Management in Advanced Power Distribution Systems: A Survey

Applications of reinforcement learning in energy systems

On Training Effective Reinforcement Learning Agents for Real-time Power Grid Operation and Control

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

Computationally Efficient Safe Reinforcement Learning for Power Systems

Reinforcement Learning Based Power Grid Day-Ahead Planning and AI-Assisted Control

Data-Efficient Deep Reinforcement Learning-Based Optimal Generation Control in DC Microgrids

Optimizing Load Scheduling in Power Grids Using Reinforcement Learning and Markov Decision Processes

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

Real-time scheduling of renewable power systems through planning-based reinforcement learning

Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning