Abstract:With the rise in electricity, gas and oil prices and the persistently high levels of carbon emissions, there is an increasing demand for effective energy management in energy systems, including electrical grids. Recent literature exhibits large potential for optimizing the behavior of such systems towards energy performance, reducing peak loads and exploiting environmentally friendly ways for energy production. However, the primary challenge relies on the optimization of such systems, which introduces significant complexities since they present quite dynamic behavior. Such cyberphysical frameworks usually integrate multiple interconnected components such as power plants, transmission lines, distribution networks and various types of energy-storage systems, while the behavior of these components is affected by various external factors such as user individual requirements, weather conditions, energy demand and market prices. Consequently, traditional optimal control approaches—such as Rule-Based Control (RBC)—prove inadequate to deal with the diverse dynamics which define the behavior of such complicated frameworks. Moreover, even sophisticated techniques—such as Model Predictive Control (MPC)—showcase model-related limitations that hinder the applicability of an optimal control scheme. To this end, AI model-free techniques such as Reinforcement Learning (RL) offer a fruitful potential for embedding efficient optimal control in cases of energy systems. Recent studies present promising results in various fields of engineering, indicating that RL frameworks may prove the key element for delivering efficient optimal control in smart buildings, electric vehicle charging and smart grid applications. The current paper provides a comprehensive review of RL implementations in energy systems frameworks—such as Renewable Energy Sources (RESs), Building Energy-Management Systems (BEMSs) and Electric Vehicle Charging Stations (EVCSs)—illustrating the benefits and the opportunities of such approaches. The work examines more than 80 highly cited papers focusing on recent RL research applications—between 2015 and 2023—and analyzes the model-free RL potential as regards the energy systems' control optimization in the future.

A review of reinforcement learning based energy management systems for electrified powertrains: Progress, challenge, and potential solution

Progress and summary of reinforcement learning on energy management of MPS-EV

Recent Progress in Energy Management of Connected Hybrid Electric Vehicles Using Reinforcement Learning

Intelligent Learning Algorithm and Intelligent Transportation-Based Energy Management Strategies for Hybrid Electric Vehicles: A Review

Empirical Analysis of AI-based Energy Management in Electric Vehicles: A Case Study on Reinforcement Learning

Reinforcement Learning-Based Energy Management for Hybrid Power Systems: State-of-the-Art Survey, Review, and Perspectives

Towards Optimal Energy Management Strategy for Hybrid Electric Vehicle with Reinforcement Learning

Battery life constrained real-time energy management strategy for hybrid electric vehicles based on reinforcement learning

Reinforcement learning for electric vehicle applications in power systems:A critical review

Reinforcement learning-based energy management strategies of fuel cell hybrid vehicles with multi-objective control

An adaptive hierarchical energy management strategy for hybrid electric vehicles combining heuristic domain knowledge and data-driven deep reinforcement learning

Benchmarking Deep Reinforcement Learning Based Energy Management Systems for Hybrid Electric Vehicles

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

Review and Evaluation of Reinforcement Learning Frameworks on Smart Grid Applications

Online energy management strategy for ammonia-hydrogen hybrid electric vehicles harnessing deep reinforcement learning

Advancements in Electric Vehicle Charging Optimization: A Survey of Reinforcement Learning Approaches

Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle

Deep Reinforcement Learning-Based Energy Management for a Series Hybrid Electric Vehicle Enabled by History Cumulative Trip Information

A Comparative Study of Deep Reinforcement Learning-based Transferable Energy Management Strategies for Hybrid Electric Vehicles

Real-Time Battery Thermal Management for Electric Vehicles Based on Deep Reinforcement Learning

Data-driven transferred energy management strategy for hybrid electric vehicles via deep reinforcement learning