Abstract:Cooperative transmission is a promising solution for reliable communication in underwater acoustic sensor networks. To fully exploit the benefits of cooperative system, efficient management of relay resources is required. However, the challenging working conditions, including imprecise channel information, limited observation feedback and dynamic underwater environment, call for an adaptive relay selection strategy for improving system capacity performance. In this paper, we present an innovative Partial Expert-based Reinforcement Adversarial Learning (PER-AL) strategy that unifies expert assisted learning and self-adjusting learning in a integral adversarial multi-armed bandit framework. The user constantly interacts with the environment and optimizes the relay decision by analyzing all gathered observation rewards, rather than relying on prior channel information. By introducing a novel expert learning mechanism into the initial learning stage, expert generator and expert learner are proposed to control the performance of experts and provide optimal expert advice to reinforce the initial relay learning for efficiency. With the deepening learning, the relay quality is then evaluated by analyzing the historical observable rewards directly through a Exp3 policy without engaging complexity too much. The proposed algorithm could automatically adapt to the underwater conditions and detect the optimal relay within a short time, significantly improving system capacity without jeopardizing the adversarial regret guarantee. Finally, extensive simulation results demonstrated the superior of PER-AL algorithm compared with recent existing CSI-based and CSI-free solution benchmarks. Particularly, PER-AL algorithm has a 28.3% improvement of the total network capacity and achieves a 44.5% decrease of implementation time.

Communication-Aware Mobile Relaying via an AUV for Minimal Wait Time: A Broad Learning-Based Solution

Collaboration-Aware Relay Selection for AUV in Internet of Underwater Network: Evolving Contextual Bandit Learning Approach

Low-Complexity Joint Resource Allocation and Trajectory Design for UAV-Aided Relay Networks With the Segmented Ray-Tracing Channel Model

Effective Relay Selection for Underwater Cooperative Acoustic Networks

Relay-aided cooperative Underwater acoustic communications: Selective relaying

Asynchronous Amplify-and-forward Relay Communications for Underwater Acoustic Networks.

A Novel Sensor Scheduling Algorithm Based on Deep Reinforcement Learning for Bearing-Only Target Tracking in UWSNs

On Sparse Bayesian Spreading Function Estimation Based Iterative Detection in Multiple-Input Multiple-Output Underwater Acoustic Communications

Joint Resource Allocation and Trajectory Optimization for UAV-Aided Relay Networks

A Channel-Aware Routing Protocol With Nearest Neighbor Regression For Underwater Sensor Networks

Enabling Reliable Water-Air Direct Optical Wireless Communication for Uncrewed Vehicular Networks: A Deep Reinforcement Learning Approach

Belief-Propagation-Based Low-Complexity Channel Estimation and Detection for Underwater Acoustic Communications With Moving Transceivers

Partial Expert-Based Adversarial Relay Learning Strategy for Underwater Acoustic Sensor Networks

Efficient Beacon-Aided AUV Localization: A Reinforcement Learning Based Approach

A Learning Method for AUV Collision Avoidance Through Deep Reinforcement Learning

Dynamic Aerial Base Station Placement for Minimum-Delay Communications

Prediction-Based Submarine Cable-Tracking Strategy for Autonomous Underwater Vehicles with Side-Scan Sonar

Advancing reliability and efficiency of urban communication: Unmanned aerial vehicles, intelligent reflection surfaces, and deep learning techniques

AUV-Aided Optical-Acoustic Hybrid Data Collection Based on Deep Reinforcement Learning

A novel intelligent collision avoidance algorithm based on deep reinforcement learning approach for USV

Relay Selection in Underwater Acoustic Cooperative Networks: A Contextual Bandit Approach.