Abstract:To reduce the fuel consumption of heavy duty logistic vehicles (HDLVs), P2 parallel hybridization is a promising solution, and deep reinforcement learning (DRL) is a promising method to optimize energy management strategies (EMSs). However, the complicated discrete-continuous hybrid action space lying in the P2 system presents a challenge to achieve real-time optimal control. Thus, this paper proposes a novel DRL algorithm combining auto-tune soft actor-critic (ATSAC) with ordinal regression to optimize the engine torque output and gear shifting simultaneously. ATSAC can adjust the update frequency and learning rate of SAC automatically to improve the generalization and ordinal regression can convert discrete variables into samplings in continuous space to handle the hybrid action. Moreover, a multi-dimensional scenario-oriented driving cycle (SODC) is established through naturalistic driving big data (NDBD) as the training cycle to further improve the EMS generalization. By comprehensive comparison with the widely used twin-delayed deep deterministic policy gradient (TD3) based EMSs, ATSAC achieves significant improvement with 53.70% higher computational efficiency and 12.31% lower negative total reward (NTR) in the training process. Application analysis in unseen real-world driving scenarios shows that only ATSAC based EMS can obtain real-time optimal control in the testing process. Furthermore, the EMS trained through SODC obtains 81.73% lower NTR than the standard China World Transient Vehicle Cycle (CWTVC) which demonstrates that SODC can represent the real-world driving scenarios much more accurately than CWTVC, especially in low-speed high-load conditions which are crucial for HDLVs.

Supplementary Learning Control for Energy Management Strategy of Hybrid Electric Vehicles at Scale

Energy Management Strategy Based on Deep Reinforcement Learning and Speed Prediction for Power-Split Hybrid Electric Vehicle with Multidimensional Continuous Control

Deep Reinforcement Learning based Energy Management for Heavy Duty HEV considering Discrete-Continuous Hybrid Action Space

A Comparative Study of Deep Reinforcement Learning Based Energy Management Strategy for Hybrid Electric Vehicle

Energy Management of Hybrid Electric Vehicles Based on Model Predictive Control and Deep Reinforcement Learning

Energy Management Strategy Based on Deep Reinforcement Learning and Speed Prediction for Power‐split Hybrid Electric Vehicle with Multi‐dimensional Continuous Control

Deep Reinforcement Learning Based Hierarchical Energy Management Recognizing Powertrain Dynamics and Enhanced by Transfer Learning for Electrified Vehicles

A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning

Research on Deep Reinforcement Learning-based Intelligent Car-following Control and Energy Management Strategy for Hybrid Electric Vehicles

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

Safe Deep Reinforcement Learning-Based Constrained Optimal Control Scheme for HEV Energy Management

Deep Reinforcement Learning-Based Energy Management for a Series Hybrid Electric Vehicle Enabled by History Cumulative Trip Information

An adaptive hierarchical energy management strategy for hybrid electric vehicles combining heuristic domain knowledge and data-driven deep reinforcement learning

Deep Reinforcement Learning Based Energy Management of Hybrid Electric Vehicles with Expert Knowledge

Rule-interposing Deep Reinforcement Learning Based Energy Management Strategy for Power-Split Hybrid Electric Vehicle

Data-driven transferred energy management strategy for hybrid electric vehicles via deep reinforcement learning

A Comparative Study of Deep Reinforcement Learning-based Transferable Energy Management Strategies for Hybrid Electric Vehicles

Intelligent Learning Algorithm and Intelligent Transportation-Based Energy Management Strategies for Hybrid Electric Vehicles: A Review

High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning

Deep Reinforcement Learning-Based Energy Management for Hybrid Electric Vehicles

Energy Management Strategy for Fuel Cell Electric Vehicles Based on Scalable Reinforcement Learning in Novel Environment