Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Generalization ability of hybrid electric vehicle energy management strategy based on reinforcement learning method

Hybrid Multi-Agent Reinforcement Learning for Electric Vehicle Resilience Control Towards a Low-Carbon Transition

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

An Energy Management Strategy of Power-Split Hybrid Electric Vehicles Using Reinforcement Learning

A deep reinforcement learning approach to energy management control with connected information for hybrid electric vehicles

Self-supervised reinforcement learning-based energy management for a hybrid electric vehicle

Energy management for hybrid electric vehicles based on imitation reinforcement learning

Hierarchical reinforcement learning based energy management strategy for hybrid electric vehicle

Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences

Multi-step Reinforcement Learning for Model-Free Predictive Energy Management of an Electrified Off-Highway Vehicle

Battery life constrained real-time energy management strategy for hybrid electric vehicles based on reinforcement learning

Bi-level energy management strategy for power-split plug-in hybrid electric vehicles: A reinforcement learning approach for prediction and control

Incentive learning-based energy management for hybrid energy storage system in electric vehicles

Deep reinforcement learning-based energy management of hybrid battery systems in electric vehicles

Continuous Reinforcement Learning-Based Energy Management Strategy for Hybrid Electric-Tracked Vehicles

A Reinforcement Learning Energy Management Strategy for Fuel Cell Hybrid Electric Vehicles Considering Driving Condition Classification

Reinforcement Learning-Based Energy Management for Hybrid Electric Vehicle Enhanced by Velocity Prediction

Reinforcement Learning-Based Energy Management for Hybrid Power Systems: State-of-the-Art Survey, Review, and Perspectives

Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning With Accelerated Gradient Optimization

Cooperative energy management and eco-driving of plug-in hybrid electric vehicle via multi-agent reinforcement learning

Towards Optimal Energy Management Strategy for Hybrid Electric Vehicle with Reinforcement Learning