Abstract:Sentence semantic matching is a core research area in natural language processing, which is widely used in various natural language tasks. In recent years, attention mechanism has shown good performance in deep neural networks for sentence semantic matching. Most of the attention-based deep neural networks focus on sentences interaction which ignore modeling the core semantic of the sentence. In other words, they do not consider the importance of the relative distance of words when modeling the sentence semantics, which leads to deviations in modeling the core semantics of the sentence and unstable sentence interaction. Usually, people tend to associate words that are relatively close together when they read and believe that there is a deeper connection between them. Besides, the current interactive matching method after sentence modeling is relatively simple and it may be inadequate. In this paper, we build a well-performed distance-aware self-attention and multi-level matching model (DSSTM) for sentence semantic matching tasks. By considering the importance of different distance tokens, it can get the better original semantics of sentences and hold interactive matching method in multiple level after sentence modeling. To be specific, given two input sentences, we first encode them as contextual embeddings. Then, the contextual embeddings are handled by enhanced distance-aware self-attention to further strengthen the sentence semantic modeling from the whole and local aspect. At the same time, we apply the co-attention layer to extract cross-sentence interaction features while simplifying all the remaining components. Finally, we fuse them into the multi-level matching function to obtain the aggregation vector and learn divers matching representations, which is helpful to capture the diversity of sentence pairs. We conduct experiments on three sentence semantic matching tasks. Experimental results on these public datasets demonstrate that our model outperforms competitive baseline methods and our model has fewer parameters. Our source code is publicly available at https://github.com/xiaodeng-1/DSSTM.(c) 2022 Elsevier B.V. All rights reserved.

Text Semantic Matching with an Enhanced Sample Building Method Based on Contrastive Learning

OssCSE: Overcoming Surface Structure Bias in Contrastive Learning for Unsupervised Sentence Embedding

Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding

SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samples

A Contrastive Framework to Enhance Unsupervised Sentence Representation Learning

Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning

CLSESSP: Contrastive learning of sentence embedding with strong semantic prototypes

A Study of Contrastive Learning Algorithms for Sentence Representation Based on Simple Data Augmentation

SimCSE++: Improving Contrastive Learning for Sentence Embeddings from Two Perspectives

Unsupervised Sentence Embedding Model Based on Contrastive Learning

HNCSE: Advancing Sentence Embeddings Via Hybrid Contrastive Learning with Hard Negatives

Contrastive sentence representation learning with adaptive false negative cancellation

Learning to Attentively Represent Distinctive Information for Semantic Text Matching

Semantic-aware Contrastive Learning for More Accurate Semantic Parsing

A Sentence is Worth 128 Pseudo Tokens: A Semantic-Aware Contrastive Learning Framework for Sentence Embeddings

The Short Text Matching Model Enhanced with Knowledge via Contrastive Learning

Semantic-Enhanced Supervised Contrastive Learning.

Enhanced Distance-Aware Self-Attention and Multi-Level Match for Sentence Semantic Matching

WhitenedCSE: Whitening-based Contrastive Learning of Sentence Embeddings

Whitening-based Contrastive Learning of Sentence Embeddings

Enhanced Text Matching Based on Semantic Transformation