Robust underwater image enhancement with cascaded multi-level sub-networks and triple attention mechanism

Dehuan Zhang,Chenyu Wu,Jingchun Zhou,Weishi Zhang,Zifan Lin,Kemal Polat,Fayadh Alenezi

DOI: https://doi.org/10.1016/j.neunet.2023.11.008

IF: 7.8

2024-01-01

Neural Networks

Abstract:With the growing exploration of marine resources, underwater image enhancement has gained significant attention. Recent advances in convolutional neural networks (CNN) have greatly impacted underwater image enhancement techniques. However, conventional CNN-based methods typically employ a single network structure, which may compromise robustness in challenging conditions. Additionally, commonly used UNet networks generally force fusion from low to high resolution for each layer, leading to inaccurate contextual information encoding. To address these issues, we propose a novel network called Cascaded Network with Multi-level Sub-networks (CNMS), which encompasses the following key components: (a) a cascade mechanism based on local modules and global networks for extracting feature representations with richer semantics and enhanced spatial precision, (b) information exchange between different resolution streams, and (c) a triple attention module for extracting attention-based features. CNMS selectively cascades multiple sub-networks through triple attention modules to extract distinct features from underwater images, bolstering the network's robustness and improving generalization capabilities. Within the sub-network, we introduce a Multi-level Sub-network (MSN) that spans multiple resolution streams, combining contextual information from various scales while preserving the original underwater images' high-resolution spatial details. Comprehensive experiments on multiple underwater datasets demonstrate that CNMS outperforms state-of-the-art methods in image enhancement tasks.

computer science, artificial intelligence,neurosciences

What problem does this paper attempt to address?

The paper attempts to address the challenges in underwater image enhancement, particularly the issues of robustness and detail feature extraction in existing convolutional neural network (CNN)-based methods under extreme underwater conditions. Specifically, the paper points out:

1. **Limitations of existing methods**:
- Conventional single network structure methods may perform poorly under extreme conditions, with less robustness.
- Commonly used UNet networks typically enforce fusion from low resolution to high resolution layer by layer, leading to inaccurate context information encoding.

2. **Proposed new method**:
- The paper proposes a new network structure called Cascaded Multi-Stage Subnetworks (CNMS), aiming to address the above issues through the following key components:
- **Cascading mechanism**: A cascading mechanism based on local modules and global networks to extract feature representations with rich semantics and enhanced spatial precision.
- **Information exchange between different resolution streams**: Information exchange between streams of different resolutions to better capture context information.
- **Triple attention mechanism**: A triple attention module to extract attention-based features, selectively cascading multiple subnetworks to extract different features of underwater images, enhancing the network's robustness and generalization ability.

3. **Objectives**:
- Improve the quality of underwater images, including increasing contrast, removing color bias, restoring image details, and eliminating uneven bright spots.
- Demonstrate through experiments on multiple underwater datasets that CNMS outperforms existing state-of-the-art methods in image enhancement tasks.

In summary, the paper aims to address the shortcomings of existing underwater image enhancement methods in terms of robustness and detail feature extraction by proposing a new network structure, CNMS, thereby improving the visual quality of underwater images.