Abstract:Self-Supervised Contrastive Learning has proven effective in deriving high-quality representations from unlabeled data. However, a major challenge that hinders both unimodal and multimodal contrastive learning is feature suppression, a phenomenon where the trained model captures only a limited portion of the information from the input data while overlooking other potentially valuable content. This issue often leads to indistinguishable representations for visually similar but semantically different inputs, adversely affecting downstream task performance, particularly those requiring rigorous semantic comprehension. To address this challenge, we propose a novel model-agnostic Multistage Contrastive Learning (MCL) framework. Unlike standard contrastive learning which inherently captures one single biased feature distribution, MCL progressively learns previously unlearned features through feature-aware negative sampling at each stage, where the negative samples of an anchor are exclusively selected from the cluster it was assigned to in preceding stages. Meanwhile, MCL preserves the previously well-learned features by cross-stage representation integration, integrating features across all stages to form final representations. Our comprehensive evaluation demonstrates MCL's effectiveness and superiority across both unimodal and multimodal contrastive learning, spanning a range of model architectures from ResNet to Vision Transformers (ViT). Remarkably, in tasks where the original CLIP model has shown limitations, MCL dramatically enhances performance, with improvements up to threefold on specific attributes in the recently proposed MMVP benchmark.

Multi-level Contrastive Learning for Keyphrase Generation

Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation

Enhancing Pre-Trained Language Representations Based on Contrastive Learning for Unsupervised Keyphrase Extraction

MCLF: A Multi-grained Contrastive Learning Framework for ASR-robust Spoken Language Understanding

Learning the Unlearned: Mitigating Feature Suppression in Contrastive Learning

Towards Better Multi-modal Keyphrase Generation via Visual Entity Enhancement and Multi-granularity Image Noise Filtering

Large Language Models can Contrastively Refine their Generation for Better Sentence Representation Learning

Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learning

MCL: Multi-Granularity Contrastive Learning Framework for Chinese NER

On the Comparison between Multi-modal and Single-modal Contrastive Learning

Multi-level cross-modal contrastive learning for review-aware recommendation

Multi-level Graph Knowledge Contrastive Learning

Improving Knowledge Graph Completion with Structure-Aware Supervised Contrastive Learning

GLEU-Guided Multi-resolution Network for Short Text Conversation

MULTI-LEVEL CONTRASTIVE LEARNING FOR CROSS-LINGUAL ALIGNMENT

MoleMCL: a multi-level contrastive learning framework for molecular pre-training

Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning

A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU

Multi-Scale Subgraph Contrastive Learning

Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding

Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure