Abstract:Networked robots have become crucial for unmanned applications since they can collaborate to complete complex tasks in remote/hazardous/depopulated areas. Due to the cost inefficiency of deploying cellular network infrastructure in these areas, hybrid satellite-UAV networks emerge as a promising solution. These networks provide seamless and on-demand connectivity for multiple robots with various task requirements, and support computation-intensive and latency-sensitive services through mobile edge computing (MEC)-based offloading. However, to complete tasks in limited times, the rapid collective movement of mobile robots may cause frequent service migration, and a large number of gathered robots may compete for limited bandwidth resources in satellite and UAV communications. As a result, offloading latency may increase significantly. To address this issue, the average completion time of multi-robot offloading in task-oriented satellite-UAV networks with MEC is formulated as an optimization problem. Unlike conventional mobility-aware MEC-based offloading schemes, joint optimization of mobility control, data offloading, and resource allocation is proposed using velocity control of multiple robots. According to Lyapunov optimization, the original optimization problem is simplified into minimizing the average completion time of offloading for all robots within UAV and satellite coverage. A multi-agent $Q$ -learning algorithm, including multi-group dual-agent $Q$ -learning, is proposed based on local state observation and global reward calculation. In each dual-agent $Q$ -learning, one agent is responsible for velocity control and communication resource allocation, while the other is responsible for data offloading and computational resource allocation. The convergence of the proposed multi-agent $Q$ -learning algorithm is also theoretically analyzed. Simulation results show that the proposed scheme can effectively reduce the offloading latency by up to 35% in the multi-robot environment over its conventional counterparts.

Mobility Management for Cellular-Connected UAVs: Model Based Versus Learning Based Approaches for Service Availability

Mobile-Aware Service Offloading for UAV-Assisted IoVs: A Multi-Agent Tiny Distributed Learning Approach

Mobility for Cellular-Connected UAVs: Challenges for the Network Provider

Machine Learning assisted Handover and Resource Management for Cellular Connected Drones

Radio Resource Management for Cellular-Connected UAV: A Learning Approach

Task-Oriented Satellite-UAV Networks with Mobile-Edge Computing

Mobility State Detection of Cellular-Connected UAVs based on Handover Count Statistics

Efficient Drone Mobility Support Using Reinforcement Learning

Multi-Agent Reinforcement Learning for Offloading Cellular Communications with Cooperating UAVs

Cellular-Connected Multi-UAV MEC Networks: an Online Stochastic Optimization Approach

Fuzzy Logic Based Handover under Multi-Connectivity in Cellular-Connected UAV Communications

A Deep Q-Learning based, Base-Station Connectivity-Aware, Decentralized Pheromone Mobility Model for Autonomous UAV Networks

A Reinforcement Learning Based User Association Algorithm for UAV Networks

Stable Matching with Evolving Preference for Adaptive Handover in Cellular-Connected UAV Networks

Dynamic UAV routing for multi-access edge computing

Reinforcement learning vs rule-based dynamic movement strategies in UAV assisted networks

A Deep Reinforcement Learning Approach to Efficient Drone Mobility Support

Learning-based Resilience Guarantee for Multi-Uav Collaborative QoS Management.

RSS-Based UAV-BS 3-D Mobility Management via Policy Gradient Deep Reinforcement Learning

Joint Design of Access Point Selection and Path Planning for UAV-Assisted Cellular Networks

Handover Experiments with UAVs: Software Radio Tools and Experimental Research Platform