Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Planning a Sustainable Electric Vehicle Infrastructure Considering Battery Life: Modeling and Resolution by the Multi-agent Q-learning Metaheuristic

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

A Novel Mathematical Model for Infrastructure Planning of Dynamic Wireless Power Transfer Systems for Electric Vehicles

Deep Reinforcement Learning Based Charging Scheduling for Household Electric Vehicles in Active Distribution Network

Multi-agent DRL-based Data-Driven Approach for PEVs Charging/discharging Scheduling in Smart Grid.

Intelligent Electric Vehicle Charging Recommendation Based on Multi-Agent Reinforcement Learning

Research on Multi-objective Charging Scheduling Strategy for Electric Vehicles: A Deep Reinforcement Learning Approach

Multi-Agent DDPG Based Electric Vehicles Charging Station Recommendation

Optimal Planning of Dynamic Wireless Charging Infrastructure for Electric Vehicles

Electric vehicle charging scheduling control strategy for the large-scale scenario with non-cooperative game-based multi-agent reinforcement learning

Multiagent Q-Learning Approach for the Recharging Scheduling of Electric Automated Guided Vehicles in Container Terminals

Deep Reinforcement Learning Based Dynamic Charging-Recycling Scheme for Wireless Rechargeable Sensor Networks

Optimized Deep Reinforcement Learning for Smart Charging Scheduling of Plug-in Electric Vehicles

Electric vehicle charging navigation strategy in coupled smart grid and transportation network: A hierarchical reinforcement learning approach

Data-Driven Energy Management of an Electric Vehicle Charging Station Using Deep Reinforcement Learning

Charge Scheduling of Electric Vehicle Fleets: Maximizing Battery Remaining Useful Life Using Machine Learning Models

Deep Reinforcement Learning Based Framework for Mobile Energy Disseminator Dispatching to Charge On-the-Road Electric Vehicles

Bi-level optimization of charging scheduling of a battery swap station based on deep reinforcement learning

Multi-objective Optimal Scheduling of Charging Stations Based on Deep Reinforcement Learning

Attention-Shared Multi-Agent Actor–Critic-Based Deep Reinforcement Learning Approach for Mobile Charging Dynamic Scheduling in Wireless Rechargeable Sensor Networks