Enhancing Underwater Imaging for Robot Through Embedded Polarization Neural Network

Yuanzheng Ma,Shu Pan,Peter Marsh,Xun Guan

DOI: https://doi.org/10.1364/cleo_at.2024.af2d.4

2024-01-01

Abstract:Transforming underwater imaging, our integrated neural network and polarization camera enhance PSNR by 40.61%, enabling the robot to effectively discern weak signals in highly scattering environments and facilitating swift imaging and detection.

What problem does this paper attempt to address?

Predicting Polarization Beyond Semantics for Wearable Robotics.

Kailun Yang,Luis M. Bergasa,Eduardo Romera,Xiao Huang,Kaiwei Wang

DOI: https://doi.org/10.1109/humanoids.2018.8625005

2018-01-01

Abstract:Semantic perception is a key enabler in robotics, which supposes a very resourceful and efficient manner of applying vision information for upper-level navigation and manipulation tasks. Given the challenges on specular semantics such as water hazards, transparent glasses and metallic surfaces, polarization imaging has been explored to complement the RGB-based pixel-wise semantic segmentation because it reflects surface characteristics and provides additional attributes. However, polarimetric measurements generally entail prohibitively expensive cameras and highly accurate calibrations. Inspired by the representation power of Convolutional Neural Networks (CNNs), we propose to predict polarization information from monocular RGB images, precisely per-pixel polarization difference. The core of our approach is a cluster of efficient deep architectures building on factorized convolutions, hierarchical dilations and pyramid representations, aimed to produce both semantic and polarimetric estimations in real time. Comprehensive experiments demonstrate the qualified accuracy on a wearable exoskeleton humanoid robot.
Research on Underwater Imaging Technology Based on Polarization Enhancement

Jiahao Liu,Shouhu Cao,Di Zhang,Yujie Chen,Fang Kong

DOI: https://doi.org/10.1088/1742-6596/2832/1/012009

2024-01-01

Journal of Physics Conference Series

Abstract:Abstract In response to the problem of unclear representation of underwater polarization image information, this paper first improves the pulse coupled neural network (PCNN) model. The polarization image is used as model input, and ultimately obtaining the target polarization degree feature representation (e-DOP) image. The polarization angle image and polarization degree feature representation image are used as the decomposition source images for non-subsampled contourlet transform (NSCT) decomposition. The decomposed high-frequency sub bands are used as inputs to the PCNN model to obtain the ignition map, and the high-frequency coefficients are computed by analyzing their regional energy distribution. The low-frequency parts are fused using principal component analysis. Finally, the final resultant image is obtained by NSCT inverse transformation. The outcomes of the experiment demonstrate that the image of fusion obtained through the method in this article has better visual effects, more prominent detail features, and effectively improves the contrast between the target and the background.
Full Scene Underwater Imaging with Polarization and an Untrained Network

Yanmin Zhu,Tianjiao Zeng,Kewei Liu,Zhenbo Ren,Edmund Y. Lam

DOI: https://doi.org/10.1364/oe.444755

IF: 3.8

2021-01-01

Optics Express

Abstract:The veiling effect caused by the scattering and absorption of suspending particles is a critical challenge of underwater imaging. It is possible to combine the image formation model (IFM) with the optical polarization characteristics underwater to effectively remove the veiling effect and recover a clear image. The performance of such methods, to a great extent, depends on the settings of the global parameters in the application scenarios. Meanwhile, learning-based methods can fit the underwater image information degradation process nonlinearly to restore the images from scattering. Here, we propose for the first time a method for full scene underwater imaging that synergistically makes use of an untrained network and polarization imaging. By mounting a Stokes mask polarizer on the CMOS camera, we can simultaneously obtain images with different polarization states for IFM calculation and optimize the imaging automatically by an untrained network without requiring extra training data. This method makes full use of the nonlinear fitting ability of a neural network and corrects the undesirable imaging effect caused by imperfect parameter settings of the classical IFM in different scenes . It shows good performance in removing the impact of water scattering and preserving the object information, making it possible to achieve clear full scene underwater imaging.
Underwater Image Recovery Utilizing Polarimetric Imaging Based on Neural Networks.

Ran Zhang,Xinyuan Gui,Haoyuan Cheng,Jinkui Chu

DOI: https://doi.org/10.1364/ao.431299

IF: 1.9

2021-01-01

Applied Optics

Abstract:Underwater imaging faces challenges due to complex optical properties in water. Our purpose is to explore the application of polarimetric imaging in image recovery under turbid water based on deep learning. A polarization camera is used to capture the polarization images of objects under water as datasets. The method used in our study aims to explore a structure and loss function that is suitable for the model. In terms of the model structure, four pairs of models consisting of polarized version and gray version based on the idea of dense U-Net and information flow were proposed. In the aspect of loss function, the method of combining weighted mean squared error with perceptual loss was proposed and a proper set of loss weights was selected through comparison experiments. Comparing the model outputs, it is found that adding polarized information along with the light intensity information to the model at the very front of the model structure brings about better recovering image. The model structure proposed can be used for image recovery in turbid water or other scattering environments. Since the polarization characteristics are considered, the recovered image has more detailed features than that where only intensity is considered. The results of comparison with other methods show the effectiveness of the proposed method.
Multi-Turbidity Underwater Image Restoration Based on Neural Network and Polarization Imaging

Gui Xinyuan,Zhang Ran,Cheng Haoyuan,Tian Lianbiao,Chu Jinkui

DOI: https://doi.org/10.3788/lop202259.0410001

2022-01-01

Laser & Optoelectronics Progress

Abstract:Underwater imaging is one of the most commonly used methods for ocean exploration, and a growing number of studies have shown that polarization is the key to the underwater creatures having vision in low illumination. In this paper, a multi-turbidity underwater image recovery method based on deep learning and polarization imaging is proposed. Multiple turbidity underwater polarization data sets are obtained by capturing images of clean water and underwater polarization images with different turbidity. A small size neural network is proposed to better learn the mapping relation between underwater polarization information under different turbidity and clear underwater images. A sliding window superposition method with different steps is proposed for different circumstances. The results show that the polarization method proposed in this paper can effectively recover the underwater image, and the peak signal to noise ratio recovered under different turbidity is 47.39% higher than that of the original image on average. The proposed method combining deep learning and polarization imaging technology can restore underwater images in multi-turbidity environment and overcome the problem of poor restoration effect of ordinary underwater images.
A Dual-Branch Autoencoder Network for Underwater Low-Light Polarized Image Enhancement

Chang Xue,Qingyu Liu,Yifan Huang,En Cheng,Fei Yuan

DOI: https://doi.org/10.3390/rs16071134

IF: 5

2024-03-25

Remote Sensing

Abstract:Underwater detection faces uncomfortable illumination conditions, and traditional optical images sensitive to intensity often cannot work well in these conditions. Polarization imaging is a good solution for underwater detection under adverse lighting conditions. However, the process of obtaining polarization information causes it to be more sensitive to noise; serious noise reduces the quality of polarized images and subsequent performance in advanced visual tasks. Unfortunately, the flourishing low-light image enhancement methods applied to intensity images have not demonstrated satisfactory performance when transferred to polarized images. In this paper, we propose a low-light image enhancement paradigm based on the antagonistic properties of polarization parameters. Furthermore, we develop a dual-branch network that relies on a gradient residual dense feature extraction module (GRD) designed for polarized image characteristics and polarization loss, effectively avoiding noise introduced during the direct amplification of brightness, and capable of restoring target contour details. To facilitate a data-driven learning method, we propose a simulation method for underwater low-light polarized images. Extensive experimental results on real-world datasets demonstrate the effectiveness of our proposed approach and its superiority against other state-of-the-art methods.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
UNDERWATER IMAGE ENHANCEMENT BASED ON POLARIZATION IMAGING

L. Shen,Y. Zhao

DOI: https://doi.org/10.5194/isprs-archives-xliii-b1-2020-579-2020

2020-08-06

Abstract:Abstract. The need of high-quality underwater imaging is obviously required in many underwater applications. For example, underwater archaeology, underwater ecological research, underwater object detection and tracking. This paper presents a joint enhancing and denoising scheme for an image taken in underwater conditions. Conventional image enhancing methods may amplify the noise depending on the distance and density of the particles in the water. To suppress the noise and improve the enhancement performance, an imaging model is modified by adding the process of amplifying the noise in underwater conditions. This model offers depth-chromaticity compensation regularization for the transmission map and chromaticity-depth compensation regularization for enhancing the image. The proposed iterative underwater image enhancing method with polarization uses these two joint regularization schemes and the relationship between the transmission map and enhanced irradiance image. The transmission map and irradiance image are used to promote each other. To verify the effectiveness of the algorithm, polarizing images of different scenes in different conditions are collected. Different algorithms are applied to the original images. Experimental results demonstrate that the proposed scheme increases visibility in extreme conditions without amplifying the noise.
An End-to-End Underwater-Image-Enhancement Framework Based on Fractional Integral Retinex and Unsupervised Autoencoder

Yang Yu,Chenfeng Qin

DOI: https://doi.org/10.3390/fractalfract7010070

IF: 3.577

2023-01-01

Fractal and Fractional

Abstract:As an essential low-level computer vision task for remotely operated underwater robots and unmanned underwater vehicles to detect and understand the underwater environment, underwater image enhancement is facing challenges of light scattering, absorption, and distortion. Instead of using a specific underwater imaging model to mitigate the degradation of underwater images, we propose an end-to-end underwater-image-enhancement framework that combines fractional integral-based Retinex and an encoder–decoder network. The proposed variant of Retinex aims to alleviate haze and color distortion in the input image while preserving edges to a large extent by utilizing a modified fractional integral filter. The encoder–decoder network with channel-wise attention modules trained in an unsupervised manner to overcome the lack of paired underwater image datasets is designed to refine the output of the Retinex. Our framework was evaluated under qualitative and quantitative metrics on several public underwater image datasets and yielded satisfactory enhancement results on the evaluation set.
Polarization-based underwater image enhancement using the neural network of Mueller matrix images

Haoyuan Cheng,Jinkui Chu,Yongtai Chen,Jianying Liu,Wenzhe Gong

DOI: https://doi.org/10.1080/09500340.2021.2024902

IF: 1.3

2022-01-01

Journal of Modern Optics

Abstract:In the paper, a method of underwater image enhancement based on the neural network of Mueller matrix images is presented. The novelty of our study lies in the employment of the neural network based on Mueller matrix imaging, which can provide complete information of the target, to conduct underwater image enhancement. To establish the dataset, we obtain the Mueller matrix images of different objects under different water turbidity. We utilize an improved neural network based on U-net and a loss function using the high-level feature extractor to enhance the underwater images. The method does not require the physical model of the underwater scene because of the use of deep learning. The enhancement of objects with different materials and textures is obvious, which illustrates that the proposed method is effective and robust. Our method is superior compared with the state-of-the-art methods in terms of quantitative measures and visual quality.
Robust polarization-based underwater image enhancement method using anchor brightness adaptation

Yuehan Chen,Yafeng Li,Yulin Wang,Zetian Mi,Yujia Wang,Xianping Fu

DOI: https://doi.org/10.1016/j.optlaseng.2023.107737

IF: 5.666

2023-07-24

Optics and Lasers in Engineering

Abstract:Most of the polarization-based image enhancement methods do not take into account the inaccurate parameter estimation caused by the amplified camera noise at low illumination conditions. To solve this problem, this paper proposes a robust polarization-based underwater image enhancement method using anchor brightness adaptation (ABA). Relying on the relationship between the Stokes Vector and the angle of polarization (AoP), the proposed neighborhood high fidelity constraint (NHFC) can robustly select the region of the background light that is most suitable for parameter estimation, greatly reducing the interference of camera random noise under low illumination conditions and making full use of the characteristics of polarized optical imaging. The background light intensity at infinity and the scene transmission map are then estimated based on the polarization characteristics of the scene, thus effectively enhancing the image. Finally, ABA is introduced with the help of greyscale information of the original image to ensure the best exposure of the enhanced image. The experiments simulate underwater imaging at night, and the results show that the enhancement performance of this method is stable and can effectively solve the problem of being susceptible to random noise interference at low light, which has better applicability for real-time underwater image enhancement.

optics
Image Descattering with Synthetic Polarization Imaging and Untrained Network

Yanmin Zhu,Tianjiao Zeng,Kewei Liu,Zhenbo Ren,Chok Hang Yeung,Edmund Y. Lam

DOI: https://doi.org/10.1117/12.2601098

2021-01-01

Abstract:Water scattering is a significant limiting factor for underwater imaging quality. It changes the transportation direction of the original light path, causes the attenuation of light intensity, and so on. In this work, we use a synthetic polarizing camera to capture the images with different polarization states and reduce the impact of water scattering in one step with the underwater light propagation model and the Stokes vector. In addition, an untrained deep network is designed to complete the image descattering processing. Compared with the methods based on deep learning or physical model prior, it is more efficient. This technology is suitable for use in portable underwater imaging optical systems for real-time imaging and detecting particulate matter such as microplastics and microbial particles. It also broadens the application of underwater polarization imaging.
Visibility Enhancement of Underwater Images Based on Polarization Common-Mode Rejection of a Highly Polarized Target Signal.

Haoxiang Li,Jingping Zhu,Jinxin Deng,Fengqi Guo,Liu Yue,Jian Sun,Yunyao Zhang,Xun Hou

DOI: https://doi.org/10.1364/oe.474365

IF: 3.8

2022-01-01

Optics Express

Abstract:Underwater active polarization imaging is promising due to its effect of significantly descattering. Polarization-difference is commonly used to filter out backscattered noise. However, the polarization common-mode rejection of target signal has rarely been utilized. In this paper, via taking full advantage of this feature of Stokes vectors S2 which ably avoids interference from target light, the spatial variation of the degree of polarization of backscattered light is accurately estimated, and the whole scene intensity distribution of background is reconstructed by Gaussian surface fitting based on least square. Meanwhile, the underwater image quality measure is applied as optimization feedback, through iterative computations, not only sufficiently suppresses backscattered noise but also better highlights the details of the target. Experimental results demonstrate the effectiveness of the proposed method for highly polarized target in strongly scattering water.
U2PNet: an Unsupervised Underwater Image-Restoration Network Using Polarization

Linghao Shen,Haisheng Xia,Xun Zhang,Yongqiang Zhao,Ning Li,Seong G. Kong,Binglu Wang,Zhijun Li

DOI: https://doi.org/10.1109/tcyb.2024.3365693

IF: 11.8

2024-01-01

IEEE Transactions on Cybernetics

Abstract:This article presents U(2)PNet, a novel unsupervised underwater image restoration network using polarization for improving signal-to-noise ratio and image quality in underwater imaging environments. Traditional methods for underwater image restoration using polarization require specific cues or pairs of underwater polarization datasets, which limit their practical applications. Our proposed method requires only one mosaicked polarized image of the scene and does not require datasets for pretraining or specific cues. We design two subnetworks (T-net and B-infinity -net) to accurately estimate the transmission map and background light, and unique nonreference loss functions to ensure effective restoration. Our experiments are based on an indoor polarization simulated dataset and a real polarization image dataset constructed from our underwater robotic platform equipped with polarization cameras. Experiment results demonstrate that our proposed method achieves state-of-the-art performance on both simulated and real underwater polarization images.
Underwater active polarization imaging algorithm based on low-rank sparse decomposition

Xiaohuan Li,Xia Wang,Zihang Su

DOI: https://doi.org/10.1117/12.2603863

2021-11-24

Abstract:Underwater optical imaging has important application value, but it is also challenging. In traditional underwater imaging, the problems of uneven illumination, blurred texture details and low contrast often exist, in this paper we propose an underwater active polarization imaging algorithm based on low-rank sparse decomposition aiming to solve the problems above. According to the principle of underwater polarization imaging, the algorithm first performs target information enhancement on the acquired polarization images. Then combining with the low-rank characteristics of backscatter images in the scattered light field, the background information and target information could be separated from the captured images by the low-rank sparse decomposition principle, the high-quality image could be recovered from turbid water as a result. The results of experimental treatments with different turbidity levels demonstrate that the underwater polarization imaging algorithm based on low-rank sparse decomposition can improve the contrast of images, maintain the details of images and remove the background scattering at the same time. Moreover, the proposed method can effectively recover multiple targets and significantly improve the imaging quality which provides a new idea for underwater polarization clear imaging detection.
Underwater motion scene image restoration based on an improved U-Net network

Jianye Liu,Hu Luo,Dawei Tu

DOI: https://doi.org/10.1364/AO.505198

2024-01-01

Abstract:Active underwater polarization imaging is a common underwater imaging method, which uses the polarization difference between the reflected light and the scattered light in the underwater scene to suppress the scattered light, so as to improve the imaging quality of the underwater scene. However, the implementation often requires the acquisition of multiple polarization images, which is not suitable for the restoration of images of underwater motion scenes. To address the problem, a U-AD-Net deep learning network model based on a single polarized image is proposed, taking the polarization information of the single polarized image as the feature input, based on the classic U-Net network model, and introducing Dense-Net and spatial attention module. The learning ability and generalization ability of the proposed model for deep features are enhanced, and the polarization information that is most helpful to the image restoration is extracted, so as to restore the scene image more comprehensively. IE, AG, UCIQE, and SSIM are selected as evaluation metrics to assess the quality of the restored images. Experimental results show that the images restored through this proposed method contain richer detail information, having an obvious advantage to the existing network models. Since only a single polarized image is needed for restoration, this method has dynamic adaptability to underwater moving scene restoration.
Optical correlation assists to enhance underwater polarization imaging performance

Pingli Han,Fei Liu,Yi Wei,Xiaopeng Shao

DOI: https://doi.org/10.1016/j.optlaseng.2020.106256

IF: 5.666

2020-11-01

Optics and Lasers in Engineering

Abstract:Polarization imaging is a promising technology with great potential. Besides light intensity, it provides information in another dimension, polarization. When imaging through water, two frames of the target at two orthogonally polarized states, Imax and Imin, are needed. In most experiments, they are determined by searching for the darkest and brightest moments during polarizer rotation. But in cases where no noticeable difference can be observed, the final detection may fail. To cope with this problem, this study exploits optical correlation to assist to determine the optimal image pair. It provides reference information by comparing the similarity between two images and quantify it with parameter PCE (Peak-to-correlation energy). Final determination is based upon the fact that the optimal image pair should be in the least similarity. Experiments demonstrate that the location of optimal images can be accurately indicated through the method. Final recovery with the selected images presents the best contrast and image quality. The established strategy will enhance performance of underwater polarization imaging with better vision, higher resolution and accuracy.

optics
Underwater Image Enhancement based on Deep Learning and Image Formation Model

Xuelei Chen,Pin Zhang,Lingwei Quan,Chao Yi,Cunyue Lu

DOI: https://doi.org/10.48550/arXiv.2101.00991

2021-01-04

Image and Video Processing

Abstract:Underwater robots play an important role in oceanic geological exploration, resource exploitation, ecological research, and other fields. However, the visual perception of underwater robots is affected by various environmental factors. The main challenge now is that images captured by underwater robots are color-distorted. The hue of underwater images tends to be close to green and blue. In addition, the contrast is low and the details are fuzzy. In this paper, a new underwater image enhancement algorithm based on deep learning and image formation model is proposed. Experimental results show that the advantages of the proposed method are that it eliminates the influence of underwater environmental factors, enriches the color, enhances details, achieves higher scores in PSNR and SSIM metrics, and helps feature key-point point matching get better results. Another significant advantage is that its computation speed is much faster than other methods.
Polarization-Enhanced Underwater Laser Range-Gated Imaging for Subaquatic Applications

Shuaibao Chen,Peng Liu,Wei He,Dong Luo,Yuguang Tan,Liangpei Chen,Jue Wang,Qi Zhao,Guohua Jiao,Wei Chen

DOI: https://doi.org/10.3390/s24206681

IF: 3.9

2024-10-18

Sensors

Abstract:Laser range-gated underwater imaging technology, by removing most of the backscattering noise, can effectively increase image contrast and extend the detection range. The optical signal captured by a range-gated imaging system primarily comprises reflected light from the object and backscattered light from the surrounding water. Consequently, surfaces with low reflectivity or highly turbid water environments substantially constrain the applicability of the range-gated imaging system. To enhance the detection capability of underwater laser range-gated imaging, this paper proposes the incorporation of underwater polarized light imaging technology as an enhancement method. Based on polarization differences, backscattered light and reflected light from an object can be distinguished. Experimental results indicate that, compared to images obtained using a conventional range-gated laser imaging system, those captured with a polarization-enhanced system exhibit an increase of up to 47% for the corresponding Enhancement Measure Evaluation (EME) index. The proposed approach, which integrates polarization imaging with range-gated laser imaging, has the potential to broaden the applicability of underwater laser imaging scenarios, such as deep-sea exploration and military applications.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
U 2 PNet: An Unsupervised Underwater Image-Restoration Network Using Polarization

Linghao Shen,Haisheng Xia,Xun Zhang,Yongqiang Zhao,Ning Li,Seong G. Kong,Binglu Wang,Zhijun Li

DOI: https://doi.org/10.1109/tcyb.2024.3365693

IF: 11.8

2024-01-01

IEEE Transactions on Cybernetics

Abstract:This article presents U<sup>2</sup>PNet, a novel unsupervised underwater image restoration network using polarization for improving signal-to-noise ratio and image quality in underwater imaging environments. Traditional methods for underwater image restoration using polarization require specific cues or pairs of underwater polarization datasets, which limit their practical applications. Our proposed method requires only one mosaicked polarized image of the scene and does not require datasets for pretraining or specific cues. We design two subnetworks (T-net and B ∞ -net) to accurately estimate the transmission map and background light, and unique nonreference loss functions to ensure effective restoration. Our experiments are based on an indoor polarization simulated dataset and a real polarization image dataset constructed from our underwater robotic platform equipped with polarization cameras. Experiment results demonstrate that our proposed method achieves state-of-the-art performance on both simulated and real underwater polarization images. The code and datasets will be available at https://github.com/polwork/U-2Pnet.

automation & control systems,computer science, cybernetics, artificial intelligence
Adaptive underwater image recovery using polarimetric imaging

Zheng Liang,Congcong Zhao,Yafei Wang,Xueyan Ding,Zetian Mi,Xianping Fu

DOI: https://doi.org/10.1117/12.2524394

2019-05-06

Abstract:Underwater image generally suffers from color degradation and poor visibility due to the attenuation and scattering of the propagated light. To restore the real environment of the underwater, most of previous methods only consider the scattering in the polarimetric image recovery model. To compensate for the absorption in different wavelengths, a simple but effective underwater image enhancement method that based on the adaptive polarimetric image recovery model is proposed in this paper. In order to achieve automatic estimation of model parameters, the backscattered light estimation is based on the quad-tree decomposition to extract the fully backscattered area automatically. The experimental results demonstrate that the visual quality of underwater images are efficiently enhanced.

Enhancing Underwater Imaging for Robot Through Embedded Polarization Neural Network

Predicting Polarization Beyond Semantics for Wearable Robotics.

Research on Underwater Imaging Technology Based on Polarization Enhancement

Full Scene Underwater Imaging with Polarization and an Untrained Network

Underwater Image Recovery Utilizing Polarimetric Imaging Based on Neural Networks.

Multi-Turbidity Underwater Image Restoration Based on Neural Network and Polarization Imaging

A Dual-Branch Autoencoder Network for Underwater Low-Light Polarized Image Enhancement

UNDERWATER IMAGE ENHANCEMENT BASED ON POLARIZATION IMAGING

An End-to-End Underwater-Image-Enhancement Framework Based on Fractional Integral Retinex and Unsupervised Autoencoder

Polarization-based underwater image enhancement using the neural network of Mueller matrix images

Robust polarization-based underwater image enhancement method using anchor brightness adaptation

Image Descattering with Synthetic Polarization Imaging and Untrained Network

Visibility Enhancement of Underwater Images Based on Polarization Common-Mode Rejection of a Highly Polarized Target Signal.

U2PNet: an Unsupervised Underwater Image-Restoration Network Using Polarization

Underwater active polarization imaging algorithm based on low-rank sparse decomposition

Underwater motion scene image restoration based on an improved U-Net network

Optical correlation assists to enhance underwater polarization imaging performance

Underwater Image Enhancement based on Deep Learning and Image Formation Model

Polarization-Enhanced Underwater Laser Range-Gated Imaging for Subaquatic Applications

U 2 PNet: An Unsupervised Underwater Image-Restoration Network Using Polarization

Adaptive underwater image recovery using polarimetric imaging