Abstract:This paper studies an integrated sensing and communications (ISAC) system for low-altitude economy (LAE), where a ground base station (GBS) provides communication and navigation services for authorized unmanned aerial vehicles (UAVs), while sensing the low-altitude airspace to monitor the unauthorized mobile target. The expected communication sum-rate over a given flight period is maximized by jointly optimizing the beamforming at the GBS and UAVs' trajectories, subject to the constraints on the average signal-to-noise ratio requirement for sensing, the flight mission and collision avoidance of UAVs, as well as the maximum transmit power at the GBS. Typically, this is a sequential decision-making problem with the given flight mission. Thus, we transform it to a specific Markov decision process (MDP) model called episode task. Based on this modeling, we propose a novel LAE-oriented ISAC scheme, referred to as Deep LAE-ISAC (DeepLSC), by leveraging the deep reinforcement learning (DRL) technique. In DeepLSC, a reward function and a new action selection policy termed constrained noise-exploration policy are judiciously designed to fulfill various constraints. To enable efficient learning in episode tasks, we develop a hierarchical experience replay mechanism, where the gist is to employ all experiences generated within each episode to jointly train the neural network. Besides, to enhance the convergence speed of DeepLSC, a symmetric experience augmentation mechanism, which simultaneously permutes the indexes of all variables to enrich available experience sets, is proposed. Simulation results demonstrate that compared with benchmarks, DeepLSC yields a higher sum-rate while meeting the preset constraints, achieves faster convergence, and is more robust against different settings.

An Ensemble Learning and Problem Solving Architecture for Airspace Management.

An Ensemble Architecture for Learning Complex Problem-Solving Techniques from Demonstration

A Deep Ensemble Multi-Agent Reinforcement Learning Approach for Air Traffic Control

General real-time three-dimensional multi-aircraft conflict resolution method using multi-agent reinforcement learning

Autonomous Air Traffic Controller: A Deep Multi-Agent Reinforcement Learning Approach

Human-AI Collaboration in Real-World Complex Environment with Reinforcement Learning

Integrated Sensing and Communications for Low-Altitude Economy: A Deep Reinforcement Learning Approach

Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering

General multi-agent reinforcement learning integrating adaptive manoeuvre strategy for real-time multi-aircraft conflict resolution

Over-the-Horizon Air Combat Environment Modeling and Deep Reinforcement Learning Application

Explaining deep reinforcement learning decisions in complex multiagent settings: towards enabling automation in air traffic flow management

Joint Offloading and Resource Allocation for Hybrid Cloud and Edge Computing in SAGINs: A Decision Assisted Hybrid Action Space Deep Reinforcement Learning Approach

Discovering Expert-Level Air Combat Knowledge via Deep Excitatory-Inhibitory Factorized Reinforcement Learning

Autonomous air traffic separation assurance through machine learning

A Human-Centered Data-Driven Planner-Actor-Critic Architecture via Logic Programming

A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance

Intelligent air defense task assignment based on hierarchical reinforcement learning

Game of Drones: Intelligent Online Decision Making of Multi-UAV Confrontation

D-HAL: Distributed Hierarchical Adversarial Learning for Multi-Agent Interaction in Autonomous Intersection Management

Deep Reinforcement-Learning-Based Air-Combat-Maneuver Generation Framework