Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Hierarchical Bipartite Graph Matching Method for Transactive V2V Power Exchange in Distribution Power System

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Hierarchical transactive power exchange method on expressways for EV energy supplement

Flexible Energy Management Protocol for Cooperative EV-to-EV Charging

Distributed Peer-to-Peer Coordination of Hierarchical Three-Phase Energy Transactions among Electric Vehicle Charging Stations in Constrained Power Distribution and Urban Transportation Networks

Hierarchical Autonomous Cooperative Operation of District Integrated Heating and Power System

Multi-objective optimization strategy for distribution network considering V2G-enabled electric vehicles in building integrated energy system

A Hierarchical Strategy for Multi-Objective Optimization of Distribution Network Considering DGs and V2G-Enabled EVs Integration

Distributed Vehicle-to-Grid (V2G) Coordination: an Efficient and Cost-Effective Framework Based on Multilevel Game Theory

Stable Matching Based Cooperative V2V Charging Mechanism for Electric Vehicles.

Hierarchical Game for Coupled Power System with Energy Sharing and Transportation System

Hierarchical Electricity and Carbon Trading in Transmission and Distribution Networks Based on Virtual Federated Prosumer

Multi-Objective Coordinated Optimal Allocation of DG and EVCSs Based on the V2G Mode

Research on V2G Interaction Strategy of New Power System

An Electric Vehicle Charging Method Considering Multiple Power Exchange Modes’ Coordination

Hierarchical Distributed Framework for EV Charging Scheduling Using Exchange Problem

Decentralized V2G/G2V Scheduling of EV Charging Stations by Considering the Conversion Efficiency of Bidirectional Chargers

Research on the Dispatching of Electric Vehicles Participating in Vehicle-to-Grid Interaction: Considering Grid Stability and User Benefits

A Hierarchical Framework for Vehicle-to-Grid (V2G) Distributed Control Combining Peer-to-peer Blockchain with Systemic Scheduling

A Fully Decentralized Hierarchical Transactive Energy Framework for Charging EVs with Local DERs in Power Distribution Systems

Phev Charging and Discharging Cooperation in V2g Networks: A Coalition Game Approach