Abstract:The advent of Urban Air Mobility (UAM) presents the scope for a transformative shift in the domain of urban transportation. However, its widespread adoption and economic viability depends in part on the ability to optimally schedule the fleet of aircraft across vertiports in a UAM network, under uncertainties attributed to airspace congestion, changing weather conditions, and varying demands. This paper presents a comprehensive optimization formulation of the fleet scheduling problem, while also identifying the need for alternate solution approaches, since directly solving the resulting integer nonlinear programming problem is computationally prohibitive for daily fleet scheduling. Previous work has shown the effectiveness of using (graph) reinforcement learning (RL) approaches to train real-time executable policy models for fleet scheduling. However, such policies can often be brittle on out-of-distribution scenarios or edge cases. Moreover, training performance also deteriorates as the complexity (e.g., number of constraints) of the problem increases. To address these issues, this paper presents an imitation learning approach where the RL-based policy exploits expert demonstrations yielded by solving the exact optimization using a Genetic Algorithm. The policy model comprises Graph Neural Network (GNN) based encoders that embed the space of vertiports and aircraft, Transformer networks to encode demand, passenger fare, and transport cost profiles, and a Multi-head attention (MHA) based decoder. Expert demonstrations are used through the Generative Adversarial Imitation Learning (GAIL) algorithm. Interfaced with a UAM simulation environment involving 8 vertiports and 40 aircrafts, in terms of the daily profits earned reward, the new imitative approach achieves better mean performance and remarkable improvement in the case of unseen worst-case scenarios, compared to pure RL results.

Online Learning Based Joint Gateway Selection and User Scheduling in Non-Stationary Air-Ground Networks

Traffic Priority-Aware Multi-User Distributed Dynamic Spectrum Access: A Multi-Agent Deep RL Approach

Reinforcement Learning-Based Resource Allocation and Energy Efficiency Optimization for a Space–Air–Ground-Integrated Network

Learning to Optimize Resource in Dynamic Wireless Environment Via Meta-Gating Graph Neural Network

Deep Reinforcement Learning for Multi-User Access Control in Non-Terrestrial Networks.

Machine Learning-Based User Scheduling in Integrated Satellite-HAPS-Ground Networks

Traffic-Aware Online Network Selection in Heterogeneous Wireless Networks

Wireless Resource Scheduling in Virtualized Radio Access Networks Using Stochastic Learning.

Network Selection Based on Evolutionary Game and Deep Reinforcement Learning in Space-Air-Ground Integrated Network

DRL-based Underlay Dynamic Spectrum Access for Cognitive Satellite Networks under Spectrum Sensing Errors

Joint Channel And Power Allocation In Dynamic Cognitive Small Cell Networks Using Asymmetric Graphical Game

Online Frequency Scheduling by Learning Parallel Actions

Intelligent Action Selection for NGSO Networks with Interference Constraints: A Modified Q-Learning Approach

Primary-User-Friendly Dynamic Spectrum Anti-Jamming Access: A GAN-Enhanced Deep Reinforcement Learning Approach

Deep Reinforcement Learning based Routing for Non-cooperative Multi-flow Games in Dynamic AANETs

Toward Intelligent Non-Terrestrial Networks Through Symbiotic Radio: A Collaborative Deep Reinforcement Learning Scheme

HAPS-UAV-Enabled Heterogeneous Networks: A Deep Reinforcement Learning Approach

Meta-Gating Framework for Fast and Continuous Resource Optimization in Dynamic Wireless Environments

Online Bipartite Matching for HAP Access in Space-Air-Ground Integrated Networks using Graph Neural Network-Enhanced Reinforcement Learning

A Graph-based Adversarial Imitation Learning Framework for Reliable & Realtime Fleet Scheduling in Urban Air Mobility

Reinforcement Learning Assisted Bandwidth Aware Virtual Network Resource Allocation