Abstract:At present, deep learning has demonstrated outstanding performance in the area of underwater image enhancement. However, these approaches often demand substantial computational resources and extended training time. Knowledge distillation is a widely used technique for model compression, and nowadays it has delivered outstanding results across various fields. However, it has not been utilized in the field of underwater image enhancement. To tackle the aforementioned issues, this paper introduces a knowledge distillation technique for underwater image enhancement for the first time. It is a semi-supervised self-inter feature distillation and unsupervised self-domain adversarial distillation approach. It specifically includes adaptive local self-feature distillation technique, information lossless multi-scale inter-feature distillation technique, and self-domain adversarial distillation approach in LAB-RGB space. Self-feature distillation enhances the performance of the student network by correcting other lossy feature maps with the maximum effective feature map. Inter-feature distillation enables the student network to maximize the potential information learned from the teacher network. Furthermore, an information loss-free pooling approach is suggested to achieve multi-scale loss-free information extraction. Self-domain adversarial distillation boosts the performance of student networks through unsupervised adaptive enhancement in LAB space and unsupervised domain adversarial distillation in RGB space. Finally, a self-inter alternate knowledge distillation training measure is proposed, aiming to maximize the respective benefits of self-inter knowledge distillation. Through extensive comparative experiments, it can be found that student networks with dissimilar structures trained using the knowledge distillation technique designed in this paper achieve outstanding underwater image enhancement results.

Image Quality Assessment Based on Self-Supervised Learning and Knowledge Distillation

DCCD: Reducing Neural Network Redundancy Via Distillation

Self-Paced Knowledge Distillation for Real-Time Image Guided Depth Completion

Image Quality Assessment Using Deep Convolutional Networks

Deep Supervised Dictionary Learning for No-Reference Image Quality Assessment

Tolerant Self-Distillation for Image Classification

Image Quality Assessment Based Label Smoothing in Deep Neural Network Learning.

Image Quality Assessment Guided Deep Neural Networks Training

Image Classification Based on Self-Distillation

Soft Hybrid Knowledge Distillation Against Deep Neural Networks

Unsupervised Barcode Image Reconstruction Based on Knowledge Distillation.

Teacher-student collaborative knowledge distillation for image classification

Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbones

Self-Distillation: Towards Efficient and Compact Neural Networks

Improved Knowledge Distillation via Teacher Assistant

Toward Efficient Image Denoising: A Lightweight Network with Retargeting Supervision Driven Knowledge Distillation

Self-Referenced Deep Learning

Knowledge Distillation Meets Self-Supervision

Semi-Supervised Feature Distillation and Unsupervised Domain Adversarial Distillation for Underwater Image Enhancement

Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation