Abstract:Providing qualified and sustainable communications is one of the key challenges for the Internet of Underwater Things (IoUT) facing constrained energy supplements, nonstationary environments, and severe communication interference. Owing to spatial separation, several nodes can (and are often required to) make transmissions simultaneously to maximize network capacity. However, existing transmission solutions often face the dilemma between maximizing local capacity and global concurrency. We break this dilemma via UDARMF, an underwater distributed and adaptive resource management framework, which maximizes network capacity by supporting an increased number of communications in the network. It is a distributed deep multiagent reinforcement learning framework that uses an observation encoder and a local utility network to coordinate the collaboration among underwater nodes by adaptively tuning its transmit parameters. We designed experiments to compare UDARMF with baselines in network capacity, concurrency, and energy efficiency. Extensive experiments were conducted to find the appropriate hyperparameters to achieve the optimal network performances. We also analyze the performance of UDARMF and baselines over diverse communication and lifetime requirements, communication environment, and energy storage. Simple closed-form approximations of UDARMF are given to reveal that an energy-constrained network's capacity increases with available energy, following a linear trend on the logarithmic scale. Experimental results demonstrate that compared to other methods, UDARMF achieves a much better tradeoff between network capacity and concurrency, at which the lifetime requirements are satisfied. The proposed framework and the closed-form approximations are likely to become valuable tools in designing and analyzing IoUT.

A Scalable and Fair Power Allocation Scheme Based on Deep Multi-Agent Reinforcement Learning in Underwater Wireless Sensor Networks

Achieving Time-Sharing and Spatial-Reuse Underwater Wireless Sensor Networks with Communication Fairness: A Distributed Deep Multi-Agent Reinforcement Learning Approach

A Deep MARL-Based Power-Management Strategy for Improving the Fair Reuse of UWSNs

Joint Link Scheduling and Power Allocation in Imperfect and Energy-Constrained Underwater Wireless Sensor Networks

Achieving Fair-Effective Communications and Robustness in Underwater Acoustic Sensor Networks: A Semi-Cooperative Approach

A Centralized Cross-Layer Protocol for Joint Power Control, Link Scheduling, and Routing in UWSNs

Power Allocation Based on Multi-Agent Deep Deterministic Policy Gradient for Underwater Acoustic Communication Networks

An end-to-end sensor scheduling method based on D3QN for underwater passive tracking in UWSNs

A Novel Sensor Scheduling Algorithm Based on Deep Reinforcement Learning for Bearing-Only Target Tracking in UWSNs

An Efficient Depth-Adjustment Deployment Scheme for Underwater Wireless Sensor Networks

Traffic Load-Aware Resource Management Strategy for Underwater Wireless Sensor Networks

A Robust Power Allocation Algorithm for Underwater Acoustic Cooperative Communication Network

A Hierarchical Deep-Reinforcement Approach

An Efficient Routing Access Method Based on Multi-Agent Reinforcement Learning in UWSNs

Routing Protocol Design for Underwater Optical Wireless Sensor Networks: A Multiagent Reinforcement Learning Approach.

A distributed routing-aware power control scheme for underwater wireless sensor networks

Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks.

Optimal Cooperative Relaying and Power Control for IoUT Networks with Reinforcement Learning

Optimal Relay Node Placement and Flow Allocation in Underwater Acoustic Sensor Networks

UDARMF: An Underwater Distributed and Adaptive Resource Management Framework

Reinforcement-Learning-Based Relay Mobility and Power Allocation for Underwater Sensor Networks Against Jamming