Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Transformer-Based Reinforcement Learning for Pickup and Delivery Problems with Late Penalties

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Spatial-temporal Pricing for Ride-Sourcing Platform with Reinforcement Learning

Reinforcement Learning for Practical Express Systems with Mixed Deliveries and Pickups

A Deep Reinforcement Learning Based Real-Time Solution Policy for the Traveling Salesman Problem

MAPDP: Cooperative Multi-Agent Reinforcement Learning to Solve Pickup and Delivery Problems

A Hierarchical Reinforcement Learning Based Optimization Framework for Large-scale Dynamic Pickup and Delivery Problems

A Two-stage Learning-based Method for Large-scale On-demand Pickup and Delivery Services with Soft Time Windows

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

Graph attention reinforcement learning with flexible matching policies for multi-depot vehicle routing problems

Multi-Vehicle Routing Problems with Soft Time Windows: A Multi-Agent Reinforcement Learning Approach

Online Vehicle Routing With Neural Combinatorial Optimization and Deep Reinforcement Learning

Reinforcement Learning for Solving Multiple Vehicle Routing Problem with Time Window

Deep Reinforcement Learning for Solving Vehicle Routing Problems With Backhauls

Learning to Optimize Industry-Scale Dynamic Pickup and Delivery Problems

Reinforcement Learning-based Approach for Dynamic Vehicle Routing Problem with Stochastic Demand

Heterogeneous Attentions for Solving Pickup and Delivery Problem via Deep Reinforcement Learning

DRL4Route: A Deep Reinforcement Learning Framework for Pick-up and Delivery Route Prediction

Optimising Stochastic Routing for Taxi Fleets with Model Enhanced Reinforcement Learning

Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching

Application of Deep Reinforcement Learning Algorithm in Uncertain Logistics Transportation Scheduling