Abstract:Self-Supervised Contrastive Learning has proven effective in deriving high-quality representations from unlabeled data. However, a major challenge that hinders both unimodal and multimodal contrastive learning is feature suppression, a phenomenon where the trained model captures only a limited portion of the information from the input data while overlooking other potentially valuable content. This issue often leads to indistinguishable representations for visually similar but semantically different inputs, adversely affecting downstream task performance, particularly those requiring rigorous semantic comprehension. To address this challenge, we propose a novel model-agnostic Multistage Contrastive Learning (MCL) framework. Unlike standard contrastive learning which inherently captures one single biased feature distribution, MCL progressively learns previously unlearned features through feature-aware negative sampling at each stage, where the negative samples of an anchor are exclusively selected from the cluster it was assigned to in preceding stages. Meanwhile, MCL preserves the previously well-learned features by cross-stage representation integration, integrating features across all stages to form final representations. Our comprehensive evaluation demonstrates MCL's effectiveness and superiority across both unimodal and multimodal contrastive learning, spanning a range of model architectures from ResNet to Vision Transformers (ViT). Remarkably, in tasks where the original CLIP model has shown limitations, MCL dramatically enhances performance, with improvements up to threefold on specific attributes in the recently proposed MMVP benchmark.

Meta Supervised Contrastive Learning for Few-Shot Open-Set Modulation Classification with Signal Constellation

FSOS-AMC: Few-Shot Open-Set Learning for Automatic Modulation Classification

MetaCL: a semi-supervised meta learning architecture via contrastive learning

Achieving Efficient Feature Representation for Modulation Signal: A Cooperative Contrast Learning Approach

Self-Contrastive Learning based Semi-Supervised Radio Modulation Classification

Open-ICL: Open-Set Modulation Classification via Incremental Contrastive Learning

SOAMC: A Semi-Supervised Open-Set Recognition Algorithm for Automatic Modulation Classification

Tackling Few-Shot Challenges in Automatic Modulation Recognition: A Multi-Level Comparative Relation Network Combining Class Reconstruction Strategy

Joint Signal Detection and Automatic Modulation Classification via Deep Learning

Contrastive Language-Signal Prediction for Automatic Modulation Recognition

SPContrastNet: A Self-Paced Contrastive Learning Model for Few-Shot Text Classification

MCML: A Novel Memory-based Contrastive Meta-Learning Method for Few Shot Slot Tagging

Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning

CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection

A complex-valued convolutional fusion-type multi-stream spatiotemporal network for automatic modulation classification

Multimodal CLIP Inference for Meta-Few-Shot Image Classification

MAML is a Noisy Contrastive Learner in Classification

Supervised Momentum Contrastive Learning for Few-Shot Classification

Self-HCL: Self-Supervised Multitask Learning with Hybrid Contrastive Learning Strategy for Multimodal Sentiment Analysis

Interventional Contrastive Learning with Meta Semantic Regularizer

Meta-Learning Guided Label Noise Distillation for Robust Signal Modulation Classification