Abstract:Underwater Wireless Sensor Networks (UWSNs) have aroused increasing interest of many researchers in industry, military, commerce and academe recently. Due to the harsh underwater environment, energy efficiency is a significant theme should be considered for routing in UWSNs. Underwater positioning is also a particularly tricky task since the high attenuation of radio-frequency signals in UWSNs. In this paper, we propose an energy-efficient depth-based opportunistic routing algorithm with Q-learning (EDORQ) for UWSNs to guarantee the energy-saving and reliable data transmission. It combines the respective advantages of Q-learning technique and opportunistic routing (OR) algorithm without the full-dimensional location information to improve the network performance in terms of energy consumption, average network overhead and packet delivery ratio. In EDORQ, the void detection factor, residual energy and depth information of candidate nodes are jointly considered when defining the Q-value function, which contributes to proactively detecting void nodes in advance, meanwhile, reducing energy consumption. In addition, a simple and scalable void node recovery mode is proposed for the selection of candidate set so as to rescue packets that are stuck in void nodes unfortunately. Furthermore, we design a novel method to set the holding time for the schedule of packet forwarding base on Q-value so as to alleviate the packet collision and redundant transmission. We conduct extensive simulations to evaluate the performance of our proposed algorithm and compare it with other three routing algorithms on Aqua-sim platform (NS2). The results show that the proposed algorithm significantly improve the performance in terms of energy efficiency, packet delivery ratio and average network overhead without sacrificing too much average packet delay.

An end-to-end sensor scheduling method based on D3QN for underwater passive tracking in UWSNs

A Novel Sensor Scheduling Algorithm Based on Deep Reinforcement Learning for Bearing-Only Target Tracking in UWSNs

Adaptive Sensor Scheduling for Target Tracking in Underwater Wireless Sensor Networks

Coordinated Target Localization and Tracking with Node Scheduling for a Network of Autonomous Underwater Vehicles

An Artificial Measurements-Based Adaptive Filter for Energy-Efficient Target Tracking Via Underwater Wireless Sensor Networks

Energy-efficient Localization and Target Tracking Via Underwater Mobile Sensor Networks

End-to-end multi-sensor fusion method based on deep reinforcement learning in UASNs

A Multi-Node Cooperative Bearing-Only Target Passive Tracking Algorithm Via UWSNs

Energy-Efficient Depth-Based Opportunistic Routing with Q-Learning for Underwater Wireless Sensor Networks

Tracking a Duty-Cycled Autonomous Underwater Vehicle by Underwater Wireless Sensor Networks

An Efficient Depth-Adjustment Deployment Scheme for Underwater Wireless Sensor Networks

A Q-Learning-Based Hierarchical Routing Protocol with Unequal Clustering for Underwater Acoustic Sensor Networks

ADRP-DQL: An adaptive distributed routing protocol for underwater acoustic sensor networks using deep Q-learning

Distributed Target Tracking in UWSNs under Stochastic Node Communication Scheme

An Efficient Routing Access Method Based on Multi-Agent Reinforcement Learning in UWSNs

SQMCR: Stackelberg Q-Learning-Based Multi-Hop Cooperative Routing Algorithm for Underwater Wireless Sensor Networks

Deep Reinforcement Learning-Based Joint Sequence Scheduling and Trajectory Planning in Wireless Rechargeable Sensor Networks

Deep Reinforcement Learning-based Multi-AUV Task Allocation Algorithm in Underwater Wireless Sensor Networks

QMCR: A Q-learning-based Multi-Hop Cooperative Routing Protocol for Underwater Acoustic Sensor Networks

Mobile Charging Scheduling Approach for Wireless Rechargeable Sensor Networks Based on Multiple Discrete-Action Space Deep Q-Network

A Scalable and Fair Power Allocation Scheme Based on Deep Multi-Agent Reinforcement Learning in Underwater Wireless Sensor Networks