Abstract:With the rapid development of wireless communication networks, UAVs serving as base stations are increasingly being applied in various scenarios which not only include edge computation and task offloading, but also involve emergency communication, vehicular network enhancement, etc. In order to enhance the utility of UAV base stations’ allocation and deployment, a series of algorithms have been proposed, utilizing heuristic methods, learning-based algorithms or optimization approaches. However, it is intractable for current algorithms to handle the exponential computation increment with UAV base stations increasing, and complicated application scenarios with high dynamic demands. To solve the above issues, we formulate a decision problem with a long sequence to optimize the deployment of multi-UAV base stations for maximizing vehicular networks’ communication coverage ratio, which needs to be subject to co-constraints consisting of moving velocity, energy consumption and communication coverage radius. To solve this optimization problem, we creatively propose an algorithm named dense multi-agent reinforcement learning (DMARL), which is under the dual-layer nested decision-making framework, centralized training with decentralized deployment, and accelerates training by only collecting critical states into the dense sampling buffer. To prove our proposed algorithm’s effectiveness and generalization ability, we conduct experimental simulations in scenarios with different scales. Corresponding results have been provided to verify our algorithm’s superiority in training efficiency and performance metrics, including coverage ratio and energy consumption, compared with other algorithms.

A Graph-Based PPO Approach in Multi-UAV Navigation for Communication Coverage

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

UAV Cooperative Search Based on Multi-agent Generative Adversarial Imitation Learning

Learning Effective Communication for Cooperative Pursuit with Multi-Agent Reinforcement Learning

Multiple-UAV Reinforcement Learning Algorithm Based on Improved PPO in Ray Framework

An Improved PPO for Multiple Unmanned Aerial Vehicles

Muti-Agent Proximal Policy Optimization For Data Freshness in UAV-assisted Networks

Research on the Multiagent Joint Proximal Policy Optimization Algorithm Controlling Cooperative Fixed-Wing UAV Obstacle Avoidance

DTPPO: Dual-Transformer Encoder-based Proximal Policy Optimization for Multi-UAV Navigation in Unseen Complex Environments

On-policy Actor-Critic Reinforcement Learning for Multi-UAV Exploration

Graph Convolutional Multi-Agent Reinforcement Learning For Uav Coverage Control

Proximal Policy Optimization for Multi-rotor UAV Autonomous Guidance, Tracking and Obstacle Avoidance

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

Communication-Efficient Cooperative Multi-Agent PPO via Regulated Segment Mixture in Internet of Vehicles

Mean policy-based proximal policy optimization for maneuvering decision in multi-UAV air combat

MARL-based Design of Multi-Unmanned Aerial Vehicle Assisted Communication System with Hybrid Gaming Mode

UAV Cooperative Air Combat Maneuvering Confrontation Based on Multi-agent Reinforcement Learning

Dense Multi-Agent Reinforcement Learning Aided Multi-UAV Information Coverage for Vehicular Networks

The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games

Multiple Ships Cooperative Navigation and Collision Avoidance using Multi-agent Reinforcement Learning with Communication

The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games