T-KD: Two-Tier Knowledge Distillation for a Lightweight Underwater Fish Species Classification Model
Beibei Li,Yiran Liu,Qingling Duan
DOI: https://doi.org/10.1007/s10499-023-01314-1
2023-01-01
Aquaculture International
Abstract:The classification of underwater fish species holds significant importance for fisheries management. Nevertheless, existing deep fish classification models require high computational resources, which hamper their deployment on underwater devices. Additionally, the complex underwater environment, the camouflaged appearance of fish, and the similarity among fish species pose challenges to the accuracy of lightweight fish classification models. To address the above issues, this paper proposes a novel two-tier knowledge distillation (T-KD) method to improve the accuracy and reduce the parameters of the underwater fish species classification models. Specifically, the T-KD involves the following key steps. Firstly, a new fish species dataset, Fish37, is constructed to augment the diversity of fish species present in existing datasets. Subsequently, we introduce a novel interlayer mapping similarity-preserving (IMSP), to facilitate the learning of richer discriminative features by capturing the mapping relationships between teacher and student network layers. Moreover, a new layer tail response (LTR) is proposed to mimic the predictions of the teacher network, efficiently improving classification performance and generalization capability. The proposed T-KD approach demonstrates remarkable performance in fish species classification, surpassing that of well-known lightweight models. The effectiveness of T-KD is extensively validated across various network depths, including ResNet and EfficientNet, and compared to other knowledge distillation methods like KD, PKT, RKD, and SP. Notably, T-KD outperforms MobileNetv3-large and obtains an impressive Top-1 accuracy of 97.20% on the Fish37 dataset only using about 1/15 model size of Vision Transformer. Furthermore, detailed generalization experiments are conducted to assess T-KD’s performance on popular benchmark datasets, such as A_Large_Scale_Fish_Dataset, Fish4knowledge, WildFish and WildFish++. In conclusion, the results indicate the potential of the T-KD to facilitate underwater fish species classification with limited computational resources on underwater devices. This research also opens up promising avenues for the practical implementation of lightweight fish classification models.