Abstract:In order to achieve the target of carbon peak and carbon neutrality, electric vehicles (EVs) have increasingly received a prominent interest to electrify the transportation sector due to their advantages of mobility and flexibility on handling complicated transportation and power networks. However, it is still challenging to realize the significant potential of EVs towards an emerging low-carbon transition. Previous works have focused on vehicle-to-grid (V2G) technology that allows for an increased utilization of EVs to make arbitrage by the temporal differentials of electricity prices. Nevertheless, the economic potential of EVs flexibility may not be fully exploited lacking an appropriate business model. This paper addresses this challenge by developing a coupled power-transportation network for cooperative EVs to optimize the provision of multiple inter-dependent services, including charging service, demand management service, carbon intensity service, and balancing service. In order to unlock this value, the EVs operation problem has already been tackled using model-based optimization approaches, which may raise privacy issues since the requirement for global information and also can be time consuming due to the high variability of transportation and power networks. In this paper, we propose a model-free hierarchical and hybrid multi-agent reinforcement learning method to learn the routing and scheduling decisions of EVs in a coupled power-transportation network with the objective of optimizing multi-service provisions. To this end, EVs do not reply on any knowledge of the simulated environment and are capable of handling system uncertainties via the learning process. Extensive case studies based on a 15-bus radial power distribution network and a 9-node 12-edge transportation network are developed to show that the proposed method outperforms the conventional learning algorithms in terms of policy quality and convergence speed. Finally, the generalizability and scalability are also investigated for different environment circumstances and EV numbers.

Nash equilibrium inspired transfer learning for self-organizing optimal carbon-energy combined-flow

Economic-Emission Coordinated Operation of Transportation-Microgrid Coupled System Based on Dynamic User Equilibrium

Multi-service Provision for Electric Vehicles in Power-Transportation Networks Towards a Low-Carbon Transition: A Hierarchical and Hybrid Multi-Agent Reinforcement Learning Approach

Target-Value-Competition-Based Multi-Agent Deep Reinforcement Learning Algorithm for Distributed Nonconvex Economic Dispatch

Multi-agent Deep Reinforcement Learning Algorithm for Distributed Economic Dispatch in Smart Grid.

Two-stage multiple cooperative games-based joint planning for shared energy storage and local integrated energy systems

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

Reactive power optimization via deep transfer reinforcement learning for efficient adaptation to multiple scenarios

Multiagent Deep Reinforcement Learning for Electric Vehicle Fast Charging Station Pricing Game in Electricity-Transportation Nexus

Deep reinforcement learning based power system optimal carbon emission flow

Multi-Stage Transmission Line Flow Control Using Centralized and Decentralized Reinforcement Learning Agents

Joint Energy and Carbon Trading for Multi-Microgrid System Based on Multi-Agent Deep Reinforcement Learning

Dynamic optimization of an integrated energy system with carbon capture and power-to-gas interconnection: A deep reinforcement learning-based scheduling strategy

Low-carbon optimal learning scheduling of the power system based on carbon capture system and carbon emission flow theory

Encoding Carbon Emission Flow in Energy Management: A Compact Constraint Learning Approach

Decentralized Coordination of Distributed Energy Resources through Local Energy Markets and Deep Reinforcement Learning

A Turbo Q-Learning (TQL) for Energy Efficiency Optimization in Heterogeneous Networks

An adaptive active power rolling dispatch strategy for high proportion of renewable energy based on distributed deep reinforcement learning

Hierarchical Game for Coupled Power System with Energy Sharing and Transportation System

Combined learning for energy efficiency in heterogeneous cellular networks