Abstract:In large-scale metropolis, it is critical to efficiently allocate various resources such as electricity, medical care, and transportation to meet the living demands of citizens, according to the spatio-temporal distributions of resources and demands. Previous researchers have done plentiful work on such problems by leveraging Multi-Agent Reinforcement Learning (MARL) methods, where multiple agents cooperatively regulate and allocate the resources to meet the demands. However, facing the great number of agents in large cities, existing MARL methods lack efficient parameter sharing strategies among agents to reduce computational complexity. There remain two primary challenges in efficient parameter sharing: (1) during the RL training process, the behavior of agents changes significantly, limiting the performance of group parameter sharing based on fixed role division decided before training; (2) the behavior of agents forms complicated action trajectories, where their role characteristics are implicit, adding difficulty to dynamically adjusting agent role divisions during the training process. In this paper, we propose Dynamic Parameter Sharing (DyPS) to solve the above challenges. We design self-supervised learning tasks to extract the implicit behavioral characteristics from the action trajectories of agents. Based on the obtained behavioral characteristics, we propose a hierarchical MARL framework capable of dynamically revising the agent role divisions during the training process and thus shares parameters among agents with the same role, reducing computational complexity. In addition, our framework can be combined with various typical MARL algorithms, including IPPO, MAPPO, etc. We conduct 7 experiments in 4 representative resource allocation scenarios, where extensive results demonstrate our method's superior performance, outperforming the state-of-the-art baseline methods by up to 31%. Our source codes are available at https://github.com/tsinghua-fib-lab/DyPS.

Deep Reinforcement Learning for Spatial Resource Allocation: A Case Study of School Districting

A Survey on Applications of Reinforcement Learning in Spatial Resource Allocation

Spatial-temporal Pricing for Ride-Sourcing Platform with Reinforcement Learning

Sponet: solve spatial optimization problem using deep reinforcement learning for urban spatial decision analysis

Deep Learning for Data-Driven Districting-and-Routing

Local Search Resource Allocation Algorithm for Space-Based Backbone Network in Deep Reinforcement Learning Method

Dynamic Balancing-Charging Management for Shared Autonomous Electric Vehicle Systems: A Two-Stage Learning-Based Approach

A Dynamic Resource Allocation Strategy with Reinforcement Learning for Multimodal Multi-objective Optimization

Resource Constrained Deep Reinforcement Learning

Spatial planning of urban communities via deep reinforcement learning

DyPS: Dynamic Parameter Sharing in Multi-Agent Reinforcement Learning for Spatio-Temporal Resource Allocation

Learning-based Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation

Two-tiered Online Optimization of Region-wide Datacenter Resource Allocation via Deep Reinforcement Learning

A deep reinforcement learning with dynamic spatio-temporal graph model for solving urban logistics delivery planning problems

DistrictNet: Decision-aware learning for geographical districting

QoE-based Deep Reinforcement Learning for Resource Allocation in Real Time XR Video Transmission

Deep Reinforcement Learning for Dynamic Clustering and Resource Allocation in Smart-Duplex Networks

A Data-Efficient Training Method for Deep Reinforcement Learning

Real-time Allocation of Shared Parking Spaces Based on Deep Reinforcement Learning

ReCovNet: Reinforcement learning with covering information for solving maximal coverage billboards location problem

Research on Resource Allocation Method of Space Information Networks Based on Deep Reinforcement Learning