Abstract:To reduce the fuel consumption of heavy duty logistic vehicles (HDLVs), P2 parallel hybridization is a promising solution, and deep reinforcement learning (DRL) is a promising method to optimize energy management strategies (EMSs). However, the complicated discrete-continuous hybrid action space lying in the P2 system presents a challenge to achieve real-time optimal control. Thus, this paper proposes a novel DRL algorithm combining auto-tune soft actor-critic (ATSAC) with ordinal regression to optimize the engine torque output and gear shifting simultaneously. ATSAC can adjust the update frequency and learning rate of SAC automatically to improve the generalization and ordinal regression can convert discrete variables into samplings in continuous space to handle the hybrid action. Moreover, a multi-dimensional scenario-oriented driving cycle (SODC) is established through naturalistic driving big data (NDBD) as the training cycle to further improve the EMS generalization. By comprehensive comparison with the widely used twin-delayed deep deterministic policy gradient (TD3) based EMSs, ATSAC achieves significant improvement with 53.70% higher computational efficiency and 12.31% lower negative total reward (NTR) in the training process. Application analysis in unseen real-world driving scenarios shows that only ATSAC based EMS can obtain real-time optimal control in the testing process. Furthermore, the EMS trained through SODC obtains 81.73% lower NTR than the standard China World Transient Vehicle Cycle (CWTVC) which demonstrates that SODC can represent the real-world driving scenarios much more accurately than CWTVC, especially in low-speed high-load conditions which are crucial for HDLVs.

Reinforcement Learning Energy Management for Hybrid Electric Tracked Vehicle with Deep Deterministic Policy Gradient

Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle

Continuous Reinforcement Learning-Based Energy Management Strategy for Hybrid Electric-Tracked Vehicles

An Intelligent Energy Management Strategy for Hybrid Vehicle with Irrational Actions Using Twin Delayed Deep Deterministic Policy Gradient

Energy management for a hybrid electric vehicle based on prioritized deep reinforcement learning framework

Online updating energy management strategy based on deep reinforcement learning with accelerated training for hybrid electric tracked vehicles

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Hierarchical Rewarding Deep Deterministic Policy Gradient Strategy for Energy Management of Hybrid Electric Vehicles

Deep Deterministic Policy Gradient Based Energy Management Strategy for Hybrid Electric Tracked Vehicle With Online Updating Mechanism

A Data-Driven Reinforcement Learning Based Energy Management Strategy via Bridging Offline Initialization and Online Fine-Tuning for a Hybrid Electric Vehicle

Enhancing Energy Management Strategies for Extended-Range Electric Vehicles through Deep Q-Learning and Continuous State Representation

Data-driven transferred energy management strategy for hybrid electric vehicles via deep reinforcement learning

Deep Reinforcement Learning based Energy Management for Heavy Duty HEV considering Discrete-Continuous Hybrid Action Space

An adaptive hierarchical energy management strategy for hybrid electric vehicles combining heuristic domain knowledge and data-driven deep reinforcement learning

Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle

Real-Time Battery Thermal Management for Electric Vehicles Based on Deep Reinforcement Learning

Battery Health-Aware and Deep Reinforcement Learning-Based Energy Management for Naturalistic Data-Driven Driving Scenarios

DQL energy management: An online-updated algorithm and its application in fix-line hybrid electric vehicle

The application of machine learning-based energy management strategy in a multi-mode plug-in hybrid electric vehicle, part II: Deep deterministic policy gradient algorithm design for electric mode