Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Eco-Friendly Crowdsourced Meal Delivery: A Dynamic On-Demand Meal Delivery System with a Mixed Fleet of Electric and Gasoline Vehicles

Modeling and Managing an On-Demand Meal Delivery System with Mixed Autonomy.

On-Demand Automotive Fleet Electrification Can Catalyze Global Transportation Decarbonization and Smart Urban Mobility

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Modeling an On-Demand Meal Delivery System with Human Couriers and Autonomous Vehicles in a Spatial Market

Designing Van-Based Mobile Battery Swapping and Rebalancing Services for Dockless Ebike-Sharing Systems Based on the Dueling Double Deep Q-network

Electric Vehicle E-hailing Fleet Dispatching and Charge Scheduling

Fleet-mix Electric Vehicle Routing Problem for the E-commerce Delivery with Limited Off-Hour Delivery Implementation

Optimal Day-Ahead Transport and Charge Scheduling of Autonomous Electric Vehicle Fleet using Local Renewable Energy Generation

Eco-Mobility-on-Demand Fleet Control with Ride-Sharing

A Low-carbon Dispatching Framework of Electrified Mobility-on-Demand Fleets

Zero-emission vehicle adoption towards sustainable e-grocery last-mile delivery

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

Electric vehicles in urban delivery fleets: How far can they go?

A Physics Model-Guided Online Bayesian Framework for Energy Management of Extended Range Electric Delivery Vehicles

Model for Crowdsourced Parcel Delivery Embedded into Mobility as a Service Based on Autonomous Electric Vehicles

Integrated location and routing optimization for the application of electric vehicles in on-demand delivery

Shared lightweight autonomous vehicles for urban food deliveries: A simulation study

Electric Autonomous Mobility-on-Demand: Jointly Optimal Vehicle Design and Fleet Operation

Crowdsourcing Electrified Mobility for Omni-Sharing Distributed Energy Resources

Mobility-Aware Charging Scheduling for Shared On-Demand Electric Vehicle Fleet Using Deep Reinforcement Learning