Abstract:Nowadays, the stationary energy storage systems (ESSs) are widely introduced to recover the regenerative braking energy in urban rail systems. And the multiple ESSs along the line, substations, traction, and braking trains in the traction power system make up a multienergy coupling system, whose energy efficiency is expected to be improved. With the aim of power flow optimization, a cooperative control strategy for multiple ESSs based on multiagent deep reinforcement learning is proposed in this article. Under the distributed control structure, the decision process of multiple ESS agents is formulated as a fully cooperative Markov game, in which each ESS makes independent decision and cooperates to improve the overall energy saving effect. And the value decomposition network, which decomposes the joint state-action value into value functions across agents, is adopted to stabilize the multiagent learning process. Three train operation scenarios are presented in simulation, and the power flow distributions are analyzed quantitatively to better evaluate the performance of the proposed cooperative strategy. A power hardware-in-the-loop (PHIL) experimental platform, which integrates the RT-LAB simulator and physical supercapacitor-based energy storage system (SCESS) is developed in this article to emulate the dc traction power system with multitrain operation. Based on the PHIL platform, the proposed cooperative control strategy is implemented experimentally. Both simulation and experimental results show that in comparison with genetic algorithm (GA), the proposed strategy optimizes the energy distribution between different SCESSs and trains, and improves the overall energy saving effect of the multi-SCESSs.

Decentralized multi-objective cloud energy storage operation control with deep reinforcement learning

Long-term Deep Reinforcement Learning for Real-Time Economic Generation Control of Cloud Energy Storage Systems with Varying Structures

Multi-Agent Deep Reinforcement Learning for Voltage Control with Coordinated Active and Reactive Power Optimization

Deep Reinforcement Learning Based Robust Control Strategy for Power Sharing and Voltage Regulation in DC Grid-Forming Energy Storage System

Large-scale deep reinforcement learning method for energy management of power supply units considering regulation mileage payment

Decentralized Graphical-Representation-Enabled Multi-Agent Deep Reinforcement Learning for Robust Control of Cyber-Physical Systems

Decentralized Cooperative Control of Multiple Energy Storage Systems in Urban Railway Based on Multiagent Deep Reinforcement Learning

Battery and Hydrogen Energy Storage Control in a Smart Energy Network with Flexible Energy Demand Using Deep Reinforcement Learning

Research on Multi-objective Charging Scheduling Strategy for Electric Vehicles: A Deep Reinforcement Learning Approach

Robust Regional Coordination of Inverter-Based Volt/Var Control Via Multi-Agent Deep Reinforcement Learning

Reinforcement Learning-based District Cooling System Control for Voltage Regulation in Distribution Networks.

Multi-agent deep reinforcement learning based distributed control architecture for interconnected multi-energy microgrid energy management and optimization

A Control Strategy Based on Deep Reinforcement Learning Under the Combined Wind-Solar Storage System

Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks

Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

An Improved Two-Stage Deep Reinforcement Learning Approach for Regulation Service Disaggregation in a Virtual Power Plant

A Multi-Agent Deep Reinforcement Learning Approach Enabled Distributed Energy Management Schedule for the Coordinate Control of Multi-Energy Hub with Gas, Electricity, and Freshwater

A Data-Driven Multi-Agent Autonomous Voltage Control Framework Using Deep Reinforcement Learning

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

Multi-agent Deep Reinforcement Learning Strategy for Distributed Energy