Contrastive Learning Based on Multiscale Hard Features for Remote-Sensing Image Scene Classification.

Zhihao Li,Biao Hou,Xianpeng Guo,Siteng Ma,Yanyu Cui,Shuang Wang,Licheng Jiao
DOI: https://doi.org/10.1109/tgrs.2023.3291878
IF: 8.2
2023-01-01
IEEE Transactions on Geoscience and Remote Sensing
Abstract:The overwhelming majority of models for remote-sensing image (RSI) scene classification generally require the weights pretrained on natural images for initialization before formal training. However, differences in imaging mechanisms lead to huge discrepancies between natural images and RSIs, and the strong visual representation learned from massive natural images limits the performance of models when inferencing RSIs. To address this issue, the well-established self-supervised contrastive learning paradigm in the natural image field is introduced to the RSI field. We propose a contrastive learning method based on multiscale hard features (MHCL), which aims to use finite RSIs to learn sufficient visual representations in an unsupervised contrastive manner, thus providing a powerful upstream pretrained model for fine-tuning downstream scene classification task. Multilevel features extracted by intermediate layers of each encoder's backbone are first gathered and then a hard features transformation method (HFT) is proposed to create hard positive features and diverse queues that save hard negatives, thereby enriching the finite scene information in small-scale RSIs. Furthermore, we redesign the multiscale hard features joint contrastive loss to boost the model to explore sufficient invariant representations by additionally pulling hard positive pairs closer and pushing hard negative pairs farther away in the embedding space. Extensive experiments demonstrate that the upstream pretraining model generated by MHCL achieves competitive transferred performance on three popular scene classification datasets, outperforming the traditional model pretrained on ImageNet and models pretrained by other state-of-the-art contrastive learning methods. Our code will be released at: https://github.com/benesakitam/MHCL.
What problem does this paper attempt to address?