Abstract:Contrastive learning has been demonstrated effective in unsupervised sentence representation learning. Given one sentence, positive pairs are obtained by passing the sentence to the encoder twice using the different dropout masks, and negative pairs are obtained by taking another sentence in the same mini-batch. However, the method suffers from the surface structure bias, i.e., sentences with similar surface structures will be regarded as close in semantics while sentences with dissimilar surface structures will be viewed as distinct in semantics. This leads to the result that paraphrasing a sentence that is dissimilar in surface structure will receive a lower semantic similarity score than inserting a negative word into the sentence. In this paper, we first verify the bias by collecting a sentence transformation testset. Then we systematically probe the existing models by proposing novel splits based on benchmark datasets in accordance with semantic and surface structure similarity. We tackle the bias in two aspects: balancing the learning target by augmenting with data that counters the bias, and meanwhile preserving word semantics by leveraging recall loss to prevent catastrophic forgetting. We evaluate our model on standard semantic textual similarity (STS) tasks using different pre-trained backbones and achieve state-of-the-art averaged performance across the STS benchmarks. Particularly, our models that are fine-tuned with RoBERTa base and RoBERTa large achieve significantly better performance on most benchmark datasets.

SETE: Syntax-Enhanced Triplet Extraction with Semantic Consistency

Negation Triplet Extraction with Syntactic Dependency and Semantic Consistency

OssCSE: Overcoming Surface Structure Bias in Contrastive Learning for Unsupervised Sentence Embedding

Decomposing Relational Triple Extraction with Large Language Models for Better Generalization on Unseen Data

Dual Encoder: Exploiting the Potential of Syntactic and Semantic for Aspect Sentiment Triplet Extraction

Bridging Text and Knowledge with Multi-Prototype Embedding for Few-Shot Relational Triple Extraction.

A Pairing Enhancement Approach for Aspect Sentiment Triplet Extraction

Speculation and negation identification via unified Machine Reading Comprehension frameworks with lexical and syntactic data augmentation

SynET: Synonym Expansion Using Transitivity

Generating paraphrase sentences for multimodal entity-category-sentiment triple extraction

Semantic and Syntactic Enhanced Aspect Sentiment Triplet Extraction

On the Form of Parsed Sentences for Relation Extraction

More than Syntaxes: Investigating Semantics to Zero-shot Cross-lingual Relation Extraction and Event Argument Role Labelling

EGDE: A Framework for Bridging the Gap in Medical Zero-shot Relation Triplet Extraction.

BiSECT: Learning to Split and Rephrase Sentences with Bitexts

MESED: A Multi-modal Entity Set Expansion Dataset with Fine-grained Semantic Classes and Hard Negative Entities

A semantically enhanced dual encoder for aspect sentiment triplet extraction

Learning Better Sentence Representation with Syntax Information

Explicit Interaction Network for Aspect Sentiment Triplet Extraction

Synchronously tracking entities and relations in a syntax-aware parallel architecture for aspect-opinion pair extraction

Span-based semantic syntactic dual enhancement for aspect sentiment triplet extraction