Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Deep Interactive Teaching-Learning Optimization Algorithm for Generation Command Dispatch of AGC with High-Penetration Electric Vehicles

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Automatic Generation Control Strategy with Aggregated Generation Units Based on Mesh Adaptive Multi-Objective Optimization Algorithm

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Deep Reinforcement Learning Based Charging Scheduling for Household Electric Vehicles in Active Distribution Network

Multi-Objective Optimal Charging Control of Plug-In Hybrid Electric Vehicles in Power Distribution Systems

Synergistic And Priority Control For Electric Vehicles Power Allocation In Participating In Agc

Bi-level decomposition algorithm of real-time AGC command for large-scale electric vehicles in frequency regulation

Hierarchical interactive demand response power profile tracking optimization and control of multiple EV aggregators

Coordinated automatic generation control of interconnected power system with imitation guided exploration multi-agent deep reinforcement learning

A Hierarchical Strategy for Multi-Objective Optimization of Distribution Network Considering DGs and V2G-Enabled EVs Integration

Hierarchical Charge Control of Large Populations of EVs

Optimal Automatic Generation Control (AGC) Dispatching and Its Control Performance Analysis for the Distribution Systems with DGs

Optimization of orderly charging strategy of electric vehicle based on improved alternating direction method of multipliers

Development of an efficient vehicle-to-grid method for massive electric vehicle aggregation

Optimal dispatching of regional power grid considering vehicle network interaction

Hierarchical Coordinated Automatic Generation Control Involving Distributed Energy Resources: an Aggregation Approach

Coordinated Electric Vehicle Charging Control with Aggregator Power Trading and Indirect Load Control

Coordination Of The Charging Of Massive Electric Vehicles Based On Improved Greedy Algorithm

Research of coordinated optimal control strategy between units and grid based on hierarchical event-driving control