Abstract:The efficiency and dynamics of hybrid electric vehicles are inherently linked to effective energy management strategies. However, complexity is heightened due to uncertainty and variations in real driving conditions. This article introduces an innovative strategy for extended-range electric vehicles, grounded in the optimization of driving cycles, prediction of driving conditions, and predictive control through neural networks. First, the challenges of the energy management system are addressed by merging deep reinforcement learning with strongly convex objective optimization, giving rise to a pioneering method called DQL-AMSGrad. Subsequently, the DQL algorithm has been implemented, allowing temporal difference-based updates to adjust Q values to maximize the expected cumulative reward. The loss function is calculated as the mean squared error between the current estimate and the calculated target. The AMSGrad optimization method has been applied to efficiently adjust the weights of the artificial neural network. Hyperparameters such as the learning rate and discount factor have been tuned using data collected during real-world driving tests. This strategy tackles the "curse of dimensionality" and demonstrates a 30% improvement in adaptability to changing environmental conditions. With a 20%-faster convergence speed and a 15%-superior effectiveness in updating neural network weights compared to conventional approaches, it also highlights an 18% reduction in fuel consumption in a case study with the Nissan Xtrail e-POWER system, validating its practical applicability.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: How to optimize the energy management strategy of Extended - Range Electric Vehicles (EREVs) through Deep Q - Learning (DQL) and continuous state representation to deal with the uncertainties and changes in actual driving conditions, thereby improving energy utilization efficiency, reducing fuel consumption and enhancing system adaptability. ### Specific problems include: 1. **The "curse of dimensionality" problem**: Traditional discrete state variables face the challenge of high - dimensional state spaces when dealing with complex driving environments. The paper proposes to effectively manage these discrete state variables through DQL combined with the AMSGrad optimization method. 2. **Dynamic adaptability to environmental conditions**: Existing control strategies are difficult to adapt to changing environmental conditions in real - time. The paper aims to achieve continuous adaptation to changing environmental conditions through the DQL - AMSGrad strategy, improving the efficiency and sustainability of real - time energy management. 3. **Optimization of neural network weight updates**: Traditional DRL algorithms have problems such as Q - value over - estimation, low stability and difficulty in parameter adjustment. By introducing the AMSGrad optimization method, the paper improves the convergence speed and the effectiveness of neural network weight updates, and enhances the robustness and stability of the model. ### Research objectives: - Propose an innovative energy management strategy that combines DQL with the AMSGrad optimization method (DQL - AMSGrad) to optimize the energy management of EREVs. - Prove the effectiveness of this strategy in practical applications through experiments, especially in the Nissan Xtrail e - POWER system, and show its potential in improving driving efficiency and reducing fuel consumption. ### Main contributions: - **Solve the "curse of dimensionality" problem**: Propose a method for effectively managing discrete state variables, combining DQL with AMSGrad, which improves the adaptability and efficiency of the system. - **Improve dynamic adaptability**: Achieve continuous adaptation to changing environmental conditions, improving the efficiency and sustainability of real - time energy management. - **Optimize neural network weight updates**: Through the AMSGrad optimization method, solve the performance gaps in existing DRL algorithms, such as Q - value over - estimation, instability and difficulty in parameter adjustment, and improve the convergence speed and the effectiveness of weight updates. Through these improvements, the paper provides a more efficient, more flexible and more adaptable solution for the energy management of EREVs.

Enhancing Energy Management Strategies for Extended-Range Electric Vehicles through Deep Q-Learning and Continuous State Representation

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Continuous Reinforcement Learning-Based Energy Management Strategy for Hybrid Electric-Tracked Vehicles

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

Transfer Deep Reinforcement Learning-Based Energy Management Strategy for Plug-In Hybrid Electric Heavy-Duty Trucks under Segmented Usage Scenarios

Real-Time Battery Thermal Management for Electric Vehicles Based on Deep Reinforcement Learning

Energy Management Strategy Based on Deep Reinforcement Learning and Speed Prediction for Power-Split Hybrid Electric Vehicle with Multidimensional Continuous Control

The application of machine learning-based energy management strategy in a multi-mode plug-in hybrid electric vehicle, part II: Deep deterministic policy gradient algorithm design for electric mode

Deep reinforcement learning based energy management strategy for range extend fuel cell hybrid electric vehicle

Transfer Deep Reinforcement Learning-enabled Energy Management Strategy for Hybrid Tracked Vehicle

Battery Health-Aware and Deep Reinforcement Learning-Based Energy Management for Naturalistic Data-Driven Driving Scenarios

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

A Double‐Deep Q‐Network‐Based Energy Management Strategy for Hybrid Electric Vehicles under Variable Driving Cycles

DQL energy management: An online-updated algorithm and its application in fix-line hybrid electric vehicle

Battery life constrained real-time energy management strategy for hybrid electric vehicles based on reinforcement learning

Incentive learning-based energy management for hybrid energy storage system in electric vehicles

Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle

An Intelligent Energy Management Strategy for Hybrid Vehicle with Irrational Actions Using Twin Delayed Deep Deterministic Policy Gradient

Longevity-conscious energy management strategy of fuel cell hybrid electric Vehicle Based on deep reinforcement learning

Multi-step Reinforcement Learning for Model-Free Predictive Energy Management of an Electrified Off-Highway Vehicle