Abstract:The dynamic topology, narrow transmission bandwidth, and limited energy of sensor nodes in mobile underwater acoustic sensor networks (UASNs) pose challenges to design an efficient and robust network for underwater communications. In this paper, we propose a novel machine learning-based clustering and routing scheme, named energy-efficient clustering and cooperative routing based on improved K-means and Q-learning (ECRKQ), to reduce and balance energy consumption among sensor nodes in a mobile UASN and improve the bandwidth utilization. In the cluster head (CH) selection stage, ECRKQ modifies the K-means algorithm to dynamically select a CH based on the residual energy of the node and the distance from the node to the centroid in a cluster. In the clustering stage, ECRKQ adopts the Q-learning algorithm by incorporating the residual energy of the CH, the energy consumption of data transmission from the node to the CH, and the energy consumption of the data transmission from the CH to the base station into the Q-value function. In the data transmission stage, ECRKQ applies the dynamic coded cooperation (DCC) transmission to improve the bandwidth utilization and the robustness of the underwater communications. In the DCC transmission, cooperative nodes are also dynamically selected based on the residual energy and the energy consumption of transmitting a packet to their destinations. In the simulation, we apply the ocean current drifting model to emulate the position variation of nodes caused by ocean currents in a mobile UASN. The simulation results show that the proposed ECRKQ scheme can achieve more balanced energy consumption among sensor nodes in a mobile UASN than that of the existing scheme.

Mitigating jamming attacks in underwater sensor networks using M‐Qubed‐based opportunistic routing protocol

SQMCR: Stackelberg Q-Learning-Based Multi-Hop Cooperative Routing Algorithm for Underwater Wireless Sensor Networks

A Q-Learning-Based Hierarchical Routing Protocol with Unequal Clustering for Underwater Acoustic Sensor Networks

Energy Efficient Routing for Multi-modal Underwater Wireless Sensor Networks

A Cooperative Routing Protocol Based on Q-Learning for Underwater Optical-Acoustic Hybrid Wireless Sensor Networks

Q-Learning Based Multi-Channel Aware Routing Protocol for Optical-Acoustic Hybrid Underwater Wireless Sensor Network

ECRKQ: Machine Learning-Based Energy-Efficient Clustering and Cooperative Routing for Mobile Underwater Acoustic Sensor Networks

Energy-aware Routing for Delay-Sensitive Underwater Wireless Sensor Networks.

Anypath Routing Protocol Design via Q-Learning for Underwater Sensor Networks

An end-to-end sensor scheduling method based on D3QN for underwater passive tracking in UWSNs

Multi-Objective Routing Optimization to Support IoUT Applications in UWSNs

Penalized Reinforcement Learning-Based Energy-Efficient UAV-RIS Assisted Maritime Uplink Communications Against Jamming

Routing Protocol Design for Underwater Optical Wireless Sensor Networks: A Multiagent Reinforcement Learning Approach.

Reinforcement Learning-Based Opportunistic Routing Protocol using Depth Information for Energy-efficient Underwater Wireless Sensor Networks

Reinforcement-Learning-Based Relay Mobility and Power Allocation for Underwater Sensor Networks Against Jamming

OMUS: Efficient Opportunistic Routing in Multi-Modal Underwater Sensor Networks

Source-based Disjoint Multi-Path Routing Protocol for Underwater Acoustic Networks

QoS and Jamming-Aware Wireless Networking Using Deep Reinforcement Learning

A Novel Sensor Scheduling Algorithm Based on Deep Reinforcement Learning for Bearing-Only Target Tracking in UWSNs

An Efficient Metaheuristic-Based Clustering with Routing Protocol for Underwater Wireless Sensor Networks

QLACO: Q-learning Aided Ant Colony Routing Protocol for Underwater Acoustic Sensor Networks