Abstract:Deep reinforcement learning (DRL) has emerged as a promising approach for optimizing energy management strategies (EMS) in new energy vehicles. Nevertheless, existing studies typically focus on evaluating the performance of a particular algorithm, overlooking critical implementation details and lacking comprehensive analysis of their real-world applicability. In this paper, we mitigate this issue by conducting a thorough practicability analysis of DRL-based EMS methods. First, we theoretically analyze the benefits and limitations of existing DRL-based EMS approaches based on taxonomy towards their practicability. Subsequently, a novel EMS method that leverages model-based DRL algorithms that other researchers typically underestimate is proposed. The method newly introduces an uncertainty-aware model-based algorithm known as Probabilistic Ensembles with Trajectory Sampling (PETS) and is validated utilizing a four-wheel-drive (4WD) battery electric vehicle (BEV). After that, we conduct a comprehensive practicability analysis of three state-of-the-art DRL algorithms considering critical aspects for real-world deployment, e.g., hyperparameter sensitivity and algorithm transferability. The results demonstrate that even though the on-policy DRL achieves better asymptotic rewards and the off-policy DRL possesses better convergence, the proposed model-based DRL, PETS-based EMS, outperforms others regarding superior robustness and promising transferability across different extents of relevance between tasks. Besides, energy consumption results demonstrate that the model-based EMS can achieve a considerable 96.6% optimality compared to the baseline dynamic programming (DP). Thus, motivated by the challenges of applying DRL algorithms to real-world EMS, our systematic investigation and new insights contribute to advancing the practical employment of DRL-based EMS for BEVs.

Energy Management of a Plug-in Hybrid Electric Vehicle Using Bayesian Optimization and Soft Actor-Critic Algorithm

High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning

An improved soft actor-critic-based energy management strategy of heavy-duty hybrid electric vehicles with dual-engine system

Soft Actor-Critic Algorithm-Based Energy Management Strategy for Plug-In Hybrid Electric Vehicle

Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle

Deep Reinforcement Learning based Energy Management for Heavy Duty HEV considering Discrete-Continuous Hybrid Action Space

An improved soft actor-critic based energy management strategy of fuel cell hybrid electric vehicle

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

Empirical Analysis of AI-based Energy Management in Electric Vehicles: A Case Study on Reinforcement Learning

An adaptive hierarchical energy management strategy for hybrid electric vehicles combining heuristic domain knowledge and data-driven deep reinforcement learning

Investigation of energy management strategy based on deep reinforcement learning algorithm for multi-speed pure electric vehicles

Safe Deep Reinforcement Learning-Based Constrained Optimal Control Scheme for HEV Energy Management

Optimal Energy Management of Plug-in Hybrid Vehicles Through Exploration-to-Exploitation Ratio Control in Ensemble Reinforcement Learning

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

Practicability analysis of online deep reinforcement learning towards energy management strategy of 4WD-BEVs driven by dual-motor in-wheel motors

An Intelligent Energy Management Strategy for Hybrid Vehicle with Irrational Actions Using Twin Delayed Deep Deterministic Policy Gradient

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

Real-Time Energy Management for Hev Combining Naturalistic Driving Data and Deep Reinforcement Learning with High Generalization

Adaptive real-time energy management strategy using heuristic search for off-road hybrid electric vehicles

Benchmarking Deep Reinforcement Learning Based Energy Management Systems for Hybrid Electric Vehicles

Supplementary Learning Control for Energy Management Strategy of Hybrid Electric Vehicles at Scale