Abstract:Deep reinforcement learning (DRL) has emerged as a promising approach for optimizing energy management strategies (EMS) in new energy vehicles. Nevertheless, existing studies typically focus on evaluating the performance of a particular algorithm, overlooking critical implementation details and lacking comprehensive analysis of their real-world applicability. In this paper, we mitigate this issue by conducting a thorough practicability analysis of DRL-based EMS methods. First, we theoretically analyze the benefits and limitations of existing DRL-based EMS approaches based on taxonomy towards their practicability. Subsequently, a novel EMS method that leverages model-based DRL algorithms that other researchers typically underestimate is proposed. The method newly introduces an uncertainty-aware model-based algorithm known as Probabilistic Ensembles with Trajectory Sampling (PETS) and is validated utilizing a four-wheel-drive (4WD) battery electric vehicle (BEV). After that, we conduct a comprehensive practicability analysis of three state-of-the-art DRL algorithms considering critical aspects for real-world deployment, e.g., hyperparameter sensitivity and algorithm transferability. The results demonstrate that even though the on-policy DRL achieves better asymptotic rewards and the off-policy DRL possesses better convergence, the proposed model-based DRL, PETS-based EMS, outperforms others regarding superior robustness and promising transferability across different extents of relevance between tasks. Besides, energy consumption results demonstrate that the model-based EMS can achieve a considerable 96.6% optimality compared to the baseline dynamic programming (DP). Thus, motivated by the challenges of applying DRL algorithms to real-world EMS, our systematic investigation and new insights contribute to advancing the practical employment of DRL-based EMS for BEVs.

A Data-Driven Reinforcement Learning Based Energy Management Strategy via Bridging Offline Initialization and Online Fine-Tuning for a Hybrid Electric Vehicle

Safe Deep Reinforcement Learning-Based Constrained Optimal Control Scheme for HEV Energy Management

DQL energy management: An online-updated algorithm and its application in fix-line hybrid electric vehicle

Online updating energy management strategy based on deep reinforcement learning with accelerated training for hybrid electric tracked vehicles

An adaptive hierarchical energy management strategy for hybrid electric vehicles combining heuristic domain knowledge and data-driven deep reinforcement learning

A novel health-aware deep reinforcement learning energy management for fuel cell bus incorporating offline high-quality experience

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

Adaptive Energy Management for Real Driving Conditions via Transfer Reinforcement Learning

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

Battery life constrained real-time energy management strategy for hybrid electric vehicles based on reinforcement learning

Towards Optimal Energy Management Strategy for Hybrid Electric Vehicle with Reinforcement Learning

Data-driven transferred energy management strategy for hybrid electric vehicles via deep reinforcement learning

An Intelligent Energy Management Strategy for Hybrid Vehicle with Irrational Actions Using Twin Delayed Deep Deterministic Policy Gradient

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Reinforcement Learning Optimized Look-Ahead Energy Management of a Parallel Hybrid Electric Vehicle

A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning

Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle

Practicability analysis of online deep reinforcement learning towards energy management strategy of 4WD-BEVs driven by dual-motor in-wheel motors

Self-supervised reinforcement learning-based energy management for a hybrid electric vehicle

Heuristic action execution for energy efficient charge-sustaining control of connected hybrid vehicles with model-free double Q-learning

Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle