Abstract:<p>Energy systems undergo major transitions to facilitate the large-scale penetration of renewable energy technologies and improve efficiencies, leading to the integration of many sectors into the energy system domain. As the complexities in this domain increase, it becomes challenging to control energy flows using existing techniques based on physical models. Moreover, although data-driven models, such as reinforcement learning (RL), have gained considerable attention in many fields, a direct shift into RL is not feasible in the energy domain irrespective of the ongoing complexities. To this end, a top-down approach is used to understand this behavior by reviewing the current state of the art.</p><p>We classified RL papers in the literature into seven categories based on their area of application. Subsequently, publications under each category were further examined relative to problem diversity, RL technique employed, performance improvement (compared with other white and gray box models), verification, and reproducibility; many of the articles reported a 10–20% performance improvement with the use of RL. In most studies, however, deep learning techniques and state-of-the-art actor-critic methods (e.g., twin delayed deep deterministic policy gradient and soft actor-critic) were not applied. This has remarkably hindered performance improvements and problems related to complex energy flows have not been considered. Approximately half of the publications reported the use of Q-learning. Furthermore, despite the availability of historical data in the energy system domain, batch RL algorithms have not been exploited. Emerging multi-agent RL applications may be considered as a positive development that can enable the management of complex interactions among multiple parties. Most studies lack proper benchmarking compared to model-based approaches or gray-box models, and a majority cover energy dispatch problems and building energy management. Although RL can adequately solve problems that are considerably integrated in several sectors, only a limited number of publications have discussed its broad application. The present study clearly demonstrates that even without the full utilization of RL capacity, this technique has a considerable potential in resolving the continuously increasing complexity within the energy system domain.</p>

A review of reinforcement learning for controlling Building Energy Systems from a computer science perspective

A Review of Deep Reinforcement Learning for Smart Building Energy Management

Applications of reinforcement learning for building energy efficiency control: A review

A review of reinforcement learning for autonomous building energy management

Reinforcement learning building control approach harnessing imitation learning

Deep Reinforcement Learning for Smart Building Energy Management: A Survey.

Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings

Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications

Multi-agent reinforcement learning dealing with hybrid action spaces: A case study for off-grid oriented renewable building energy system

Reinforcement Learning and Its Applications in Modern Power and Energy Systems: A Review

Simulation Studies on Deep Reinforcement Learning for Building Control with Human Interaction

Energy-Efficient HVAC Control Based on Reinforcement Learning and Transfer Learning in a Residential Building

BEAR: Physics-Principled Building Environment for Control and Reinforcement Learning

Data-driven Model Predictive and Reinforcement Learning Based Control for Building Energy Management: a Survey

A review of reinforcement learning methodologies for controlling occupant comfort in buildings

Applications of reinforcement learning in energy systems

Exploring Deep Reinforcement Learning for Holistic Smart Building Control

Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: Simulations and experiments

Real building implementation of a deep reinforcement learning controller to enhance energy efficiency and indoor temperature control

Evaluating Reinforcement Learning Algorithms in Residential Energy Saving and Comfort Management

A Relearning Approach to Reinforcement Learning for Control of Smart Buildings