Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Two-Stage TSO-DSO Services Provision Framework for Electric Vehicle Coordination

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

A temporal and spatial electric vehicle charging optimization scheme with DSO-EVA coordination framework

Enhanced Transmission and Distribution NetworkCoordination to Host More Electric Vehicles and PV

A Two-Stage Multi-Agent EV Charging Coordination Scheme for Maximizing Grid Performance and Customer Satisfaction

Coordinated Charging and Discharging Management of Electric Vehicles Based on a Multi-function Big Data Service Platform

Deep Reinforcement Learning Based Charging Scheduling for Household Electric Vehicles in Active Distribution Network

Electric Vehicle (EV) Charging Management with Dynamic Distribution System Tariff

Coordinated dispatch of electric vehicles and wind power considering time-of-use pricing

Co-Optimizing Bidding and Power Allocation of an EV Aggregator Providing Real-Time Frequency Regulation Service

Coordinated Electric Vehicle Active and Reactive Power Control for Active Distribution Networks

Network-aware electric vehicle charging/discharging scheduling for grid load management in a hierarchical framework

Two-Stage Optimization for Efficient V2G Coordination in Distribution Power System

Autonomous Vehicle-to-Grid Design for Provision of Frequency Control Ancillary Service and Distribution Voltage Regulation

Time of Use Price based Vehicle to Grid Scheduling of Electric Vehicle Aggregator for Improved Market Operations

Vehicle-to-Grid Control for Supplementary Frequency Regulation Considering Charging Demands

Coordinated Unbalance Compensation and Harmonic Mitigation in the Secondary Distribution Network Through EVs Participation

Two-Stage Coordination Scheme for Multiple EV Charging Stations Connected to an Exclusive DC Feeder Considering Grid-Tie Converter Limitation

Multi-Objective Coordinated Optimal Allocation of DG and EVCSs Based on the V2G Mode

Optimal vehicle-to-grid control for supplementary frequency regulation using deep reinforcement learning

Day-Ahead Distributionally Robust Optimization-Based Scheduling for Distribution Systems with Electric Vehicles