Abstract:The dynamic topology, narrow transmission bandwidth, and limited energy of sensor nodes in mobile underwater acoustic sensor networks (UASNs) pose challenges to design an efficient and robust network for underwater communications. In this paper, we propose a novel machine learning-based clustering and routing scheme, named energy-efficient clustering and cooperative routing based on improved K-means and Q-learning (ECRKQ), to reduce and balance energy consumption among sensor nodes in a mobile UASN and improve the bandwidth utilization. In the cluster head (CH) selection stage, ECRKQ modifies the K-means algorithm to dynamically select a CH based on the residual energy of the node and the distance from the node to the centroid in a cluster. In the clustering stage, ECRKQ adopts the Q-learning algorithm by incorporating the residual energy of the CH, the energy consumption of data transmission from the node to the CH, and the energy consumption of the data transmission from the CH to the base station into the Q-value function. In the data transmission stage, ECRKQ applies the dynamic coded cooperation (DCC) transmission to improve the bandwidth utilization and the robustness of the underwater communications. In the DCC transmission, cooperative nodes are also dynamically selected based on the residual energy and the energy consumption of transmitting a packet to their destinations. In the simulation, we apply the ocean current drifting model to emulate the position variation of nodes caused by ocean currents in a mobile UASN. The simulation results show that the proposed ECRKQ scheme can achieve more balanced energy consumption among sensor nodes in a mobile UASN than that of the existing scheme.

QLACO: Q-learning Aided Ant Colony Routing Protocol for Underwater Acoustic Sensor Networks

A Q-Learning-Based Hierarchical Routing Protocol with Unequal Clustering for Underwater Acoustic Sensor Networks

A Cooperative Routing Protocol Based on Q-Learning for Underwater Optical-Acoustic Hybrid Wireless Sensor Networks

Q-Learning Based Multi-Channel Aware Routing Protocol for Optical-Acoustic Hybrid Underwater Wireless Sensor Network

Sea Experiment for Mobile Underwater Acoustic Networks

Sea Experiment for Mobile Underwater Acoustic Networks Extended Abstract

ADRP-DQL: An adaptive distributed routing protocol for underwater acoustic sensor networks using deep Q-learning

SQMCR: Stackelberg Q-Learning-Based Multi-Hop Cooperative Routing Algorithm for Underwater Wireless Sensor Networks

UWIA-AODV: Improved Ad-Hoc On-Demand Distance Vector Routing Protocol for Underwater Acoustic Sensor Networks

ECRKQ: Machine Learning-Based Energy-Efficient Clustering and Cooperative Routing for Mobile Underwater Acoustic Sensor Networks

Cross-Layer Routing Protocol Based on Channel Quality for Underwater Acoustic Communication Networks

Anypath Routing Protocol Design via Q-Learning for Underwater Sensor Networks

Secure Routing in Underwater Acoustic Sensor Networks Based on AFSA-ACOA Fusion Algorithm

Delay and Queue Aware Adaptive Scheduling- Based MAC Protocol for Underwater Acoustic Sensor Networks

Source-based Disjoint Multi-Path Routing Protocol for Underwater Acoustic Networks

Routing Protocol Design for Underwater Optical Wireless Sensor Networks: A Multiagent Reinforcement Learning Approach.

Asynchronous Amplify-and-forward Relay Communications for Underwater Acoustic Networks.

An Adaptive Multi-Zone Geographic Routing Protocol for Underwater Acoustic Sensor Networks

A Novel Sensor Scheduling Algorithm Based on Deep Reinforcement Learning for Bearing-Only Target Tracking in UWSNs

Effective Relay Selection for Underwater Cooperative Acoustic Networks

DR-ALOHA-Q: A Q-Learning-Based Adaptive MAC Protocol for Underwater Acoustic Sensor Networks