Abstract:Mobile crowdsensing (MCS) is attracting considerable attention in the past few years as a new paradigm for large-scale information sensing. Unmanned aerial vehicles (UAVs) have played a significant role in MCS tasks and served as crucial nodes in the newly-proposed space-air-ground integrated network (SAGIN). In this paper, we incorporate SAGIN into MCS task and present a Space-Air-Ground integrated Mobile CrowdSensing (SAG-MCS) problem. Based on multi-source observations from embedded sensors and satellites, an aerial UAV swarm is required to carry out energy-efficient data collection and recharging tasks. Up to date, few studies have explored such multi-task MCS problem with the cooperation of UAV swarm and satellites. To address this multi-agent problem, we propose a novel deep reinforcement learning (DRL) based method called Multi-Scale Soft Deep Recurrent Graph Network (ms-SDRGN). Our ms-SDRGN approach incorporates a multi-scale convolutional encoder to process multi-source raw observations for better feature exploitation. We also use a graph attention mechanism to model inter-UAV communications and aggregate extra neighboring information, and utilize a gated recurrent unit for long-term performance. In addition, a stochastic policy can be learned through a maximum-entropy method with an adjustable temperature parameter. Specifically, we design a heuristic reward function to encourage the agents to achieve global cooperation under partial observability. We train the model to convergence and conduct a series of case studies. Evaluation results show statistical significance and that ms-SDRGN outperforms three state-of-the-art DRL baselines in SAG-MCS. Compared with the best-performing baseline, ms-SDRGN improves 29.0% reward and 3.8% CFE score. We also investigate the scalability and robustness of ms-SDRGN towards DRL environments with diverse observation scales or demanding communication conditions.

QoI-Aware Mobile Crowdsensing for Metaverse by Multi-Agent Deep Reinforcement Learning

AoI-minimal UAV Crowdsensing by Model-based Graph Convolutional Reinforcement Learning

Age-of-Information Driven Mobile Crowdsensing in Wireless Edge Computing

Mobile Crowdsensing for Data Freshness: A Deep Reinforcement Learning Approach.

Cooperative Data Sensing and Computation Offloading in UAV-Assisted Crowdsensing with Multi-Agent Deep Reinforcement Learning

Exploring Both Individuality and Cooperation for Air-Ground Spatial Crowdsourcing by Multi-Agent Deep Reinforcement Learning.

Space-Air-Ground Integrated Mobile Crowdsensing for Partially Observable Data Collection by Multi-Scale Convolutional Graph Reinforcement Learning

Ensuring Threshold AoI for UAV-Assisted Mobile Crowdsensing by Multi-Agent Deep Reinforcement Learning with Transformer

Mean-Field Multi-Agent Reinforcement Learning for UAV Assisted Secure Data Dissemination.

UAV aided Metaverse over Wireless Communications: A Reinforcement Learning Approach

High-Sample-Efficient Multiagent Reinforcement Learning for Navigation and Collision Avoidance of UAV Swarms in Multitask Environments

Smart Mobile Crowdsensing with Urban Vehicles: A Deep Reinforcement Learning Perspective

Population-Invariant MADRL for AoI-Aware UAV Trajectory Design and Communication Scheduling in Wireless Sensor Networks

Meta-Reinforcement Learning for UAV-Assisted Mobile Edge Computing of Virtual Reality Services.

When Crowdsourcing Meets Unmanned Vehicles: Toward Cost-Effective Collaborative Urban Sensing Via Deep Reinforcement Learning

Energy-Efficient Mobile Crowdsensing by Unmanned Vehicles: A Sequential Deep Reinforcement Learning Approach

A UAV-Assisted Multi-Task Allocation Method for Mobile Crowd Sensing.

Multi-UAV Cooperative Search in Multi-Layered Aerial Computing Networks: A Multi-Agent Deep Reinforcement Learning Approach

Human-Drone Collaborative Spatial Crowdsourcing by Memory-Augmented and Distributed Multi-Agent Deep Reinforcement Learning

UAV-assisted Semantic Communication with Hybrid Action Reinforcement Learning

Distributed UAV Swarm for Device-Free Integrated Sensing and Communication Relying on Multi-Agent Reinforcement Learning