Abstract:To reduce the fuel consumption of heavy duty logistic vehicles (HDLVs), P2 parallel hybridization is a promising solution, and deep reinforcement learning (DRL) is a promising method to optimize energy management strategies (EMSs). However, the complicated discrete-continuous hybrid action space lying in the P2 system presents a challenge to achieve real-time optimal control. Thus, this paper proposes a novel DRL algorithm combining auto-tune soft actor-critic (ATSAC) with ordinal regression to optimize the engine torque output and gear shifting simultaneously. ATSAC can adjust the update frequency and learning rate of SAC automatically to improve the generalization and ordinal regression can convert discrete variables into samplings in continuous space to handle the hybrid action. Moreover, a multi-dimensional scenario-oriented driving cycle (SODC) is established through naturalistic driving big data (NDBD) as the training cycle to further improve the EMS generalization. By comprehensive comparison with the widely used twin-delayed deep deterministic policy gradient (TD3) based EMSs, ATSAC achieves significant improvement with 53.70% higher computational efficiency and 12.31% lower negative total reward (NTR) in the training process. Application analysis in unseen real-world driving scenarios shows that only ATSAC based EMS can obtain real-time optimal control in the testing process. Furthermore, the EMS trained through SODC obtains 81.73% lower NTR than the standard China World Transient Vehicle Cycle (CWTVC) which demonstrates that SODC can represent the real-world driving scenarios much more accurately than CWTVC, especially in low-speed high-load conditions which are crucial for HDLVs.

Heterogeneous multi-agent deep reinforcement learning for eco-driving of hybrid electric tracked vehicles: A heuristic training framework

Multi-Agent Deep Reinforcement Learning-Based Multi-Objective Cooperative Control Strategy for Hybrid Electric Vehicles

A Deep Reinforcement Learning Framework for Eco-driving in Connected and Automated Hybrid Electric Vehicles

Enabling intelligent transferable energy management of series hybrid electric tracked vehicle across motion dimensions via soft actor-critic algorithm

Cooperative energy management and eco-driving of plug-in hybrid electric vehicle via multi-agent reinforcement learning

Deep reinforcement learning and reward shaping based eco-driving control for automated HEVs among signalized intersections

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

Energy management for a hybrid electric vehicle based on prioritized deep reinforcement learning framework

A Deep Reinforcement Learning Based Hierarchical Eco-Driving Strategy for Connected and Automated HEVs

Safe Deep Reinforcement Learning-Based Constrained Optimal Control Scheme for HEV Energy Management

Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle

Energy Management of Multi-mode Hybrid Electric Vehicles based on Hand-shaking Multi-agent Learning

Online updating energy management strategy based on deep reinforcement learning with accelerated training for hybrid electric tracked vehicles

Deep Reinforcement Learning based Energy Management for Heavy Duty HEV considering Discrete-Continuous Hybrid Action Space

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning

Hybrid Reinforcement Learning-Based Eco-Driving Strategy for Connected and Automated Vehicles at Signalized Intersections

High robustness energy management strategy of hybrid electric vehicle based on improved soft actor-critic deep reinforcement learning

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

Eco-driving at signalized intersections: a parameterized reinforcement learning approach

Energy Management of Multi-mode Plug-in Hybrid Electric Vehicle using Multi-agent Deep Reinforcement Learning