Incentive learning-based energy management for hybrid energy storage system in electric vehicles

Fei Li,Yang Gao,Yue Wu,Yaoxin Xia,Chenglong Wang,Jiajian Hu,Zhiwu Huang

DOI: https://doi.org/10.1016/j.enconman.2023.117480

IF: 10.4

2023-08-18

Energy Conversion and Management

Abstract:Deep reinforcement learning has emerged as a promising candidate for online optimal energy management of multi-energy storage vehicles. However, how to ensure the adaptability and optimality of the reinforcement learning agent under realistic driving conditions is still the main bottleneck. To enable the reinforcement learning agent to efficiently learn the optimal power allocation strategies under diverse driving conditions, this paper proposes an incentive learning-based energy management strategy for battery-supercapacitor electric vehicles to minimize the battery capacity loss cost and power loss cost. First, an incentive reward function based on supercapacitor state-of-charge and vehicle acceleration is proposed for proximal policy optimization-based energy management strategy, which can stimulate the agent to learn for optimal power allocation policy under high load power conditions quickly. Second, a random sampling-based velocity transfer probability surface is constructed for pre-training to guarantee strategy optimality under unfamiliar driving cycles. Third, the generalized advantage estimation and layer normalization of neural networks are incorporated to improve the learning convergence. Results show that the proposed method can reduce the above costs by 5.8%–13.8% and 11.7%–38.8% compared with existing deep reinforcement learning methods under the pre-training driving cycle and test driving cycles, respectively, which yields closer results to offline dynamic programming.

energy & fuels,thermodynamics,mechanics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is in electric vehicles, how to optimize the energy management strategy of the hybrid energy storage system (HESS) through incentive learning methods to reduce battery capacity loss costs and power loss costs. Specifically, the paper focuses on how to make the reinforcement learning agent quickly adapt and learn the optimal power distribution strategy under actual driving conditions, especially to efficiently utilize supercapacitors under high - load power conditions while ensuring the optimality of the strategy under unfamiliar driving cycles. The method proposed in the paper aims to improve the adaptability and optimization ability of energy management and reduce costs compared with existing deep reinforcement learning methods.