Abstract:The modern energy management strategy (EMS) plays a vital role in the energy efficiency of the extended range electric vehicle. However, some modern strategies such as model predictive control (MPC) and dynamic programming (DP) have limited practical potential because they are subject to the pre-known environment information and noise interference. The reinforcement learning (RL)control strategy can be adopted as online control to interact with the vehicle and the environment. In this study, a novel auxiliary power unit (APU) charging strategy with multi-object optimization is proposed to achieve high fuel conversion efficiency while maintaining battery charging health. The state-of-the-art algorithm, Soft Actor-Critic (SAC), is applied to achieve better exploration of the possible APU behaviour and solve the sensitivity and poor convergence problems from the current RL studies. Its performance is further verified by the results of the Deep Deterministic Policy Gradient (DDPG) algorithm and DP. Three innovative targets are selected as the RL rewards for optimization: the engine fuel rate, SOC charging trajectory, and the battery charging rate (C-rate). The first adoption of the battery C-rate monitoring in RL-based energy management strategy helps extend the battery lifespan from excessive discharge. The comparative results show that the SAC had a 36% faster convergence speed than DDPG while providing a smoother and more stable action space. The fuel consumption with SAC also outplays that of DDPG by around 3%, which achieves almost 95% of the global optimization result. The successful deployment of the SAC algorithm as an EMS indicates its standout ability in dealing with wide-range actions and states with high randomness, revealing the practical potential compared with the existing RL strategies.

A Novel Reinforcement-Learning-Based Compensation Strategy for DMPC-Based Day-Ahead Energy Management of Shipboard Power Systems

A Distributed Model Predictive Control Algorithm with the Gap Metric Output Feedback Decoupling

Model and Load Predictive Control for Design and Energy Management of Shipboard Power Systems

Energy optimal dispatching of ship's integrated power system based on deep reinforcement learning

Efficient Load Management in Electric Ships: A Model Predictive Control Approach

Distributed Model Predictive Control with Asynchronous Controller Evaluations

Dynamic joint optimization of power generation and voyage scheduling in ship power system based on deep reinforcement learning

Deep Reinforcement Learning-Based Energy Management for Liquid Hydrogen-Fueled Hybrid Electric Ship Propulsion System

Joint Optimization of Power Generation and Voyage Scheduling in Ship Power System Based on Operating Scene Clustering and Multi-Task Deep Reinforcement Learning

Energy management strategy via maximum entropy reinforcement learning for an extended range logistics vehicle

Multi-objective hierarchical energy management strategy for fuel cell/battery hybrid power ships

An Intelligent Energy Management Framework for Hybrid-Electric Propulsion Systems Using Deep Reinforcement Learning

A Deep Reinforcement Learning Based Approach for Optimal Active Power Dispatch

A Control Strategy Based on Deep Reinforcement Learning Under the Combined Wind-Solar Storage System

Aalborg Universitet Hierarchical Control Design for Shipboard Power System with DC Distribution and Energy Storage aboard Future More-Electric Ships

Research on Multi-Energy Integrated Ship Energy Management System Based on Hierarchical Control Collaborative Optimization Strategy

Dynamic Response Control Strategy for Parallel Hybrid Ships Based on PMP-HMPC

Ship energy scheduling with DQN-CE algorithm combining bi-directional LSTM and attention mechanism

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

Nonlinear Model Predictive Control for Navy Microgrids with Stabilizing Terminal Ingredients

An energy trade-off management strategy for hybrid ships based on event-triggered model predictive control