Abstract:The goal of sentence matching is to determine the semantic relation between two sentences, which is the basis of many downstream tasks in natural language processing, such as question answering and information retrieval. Recent studies using attention mechanism to align the elements of two sentences have shown promising results in capturing semantic similarity/relevance. Most existing methods mainly focus on the design of multi-layer attention network, however, some critical issues have not been dealt with well: 1) the higher attention layer is easily affected by error propagation because it relies on the alignment results of preceding attentions; 2) models have the risk of losing low-layer semantic features with the increase of network depth; and 3) the approach of capturing global matching information brings about large computing complexity for model training. To this end, we propose a Deep Bi-Directional Interaction Network (DBDIN) to solve these issues, which captures semantic relatedness from two directions and each direction employs multiple attention-based interaction units. To be specific, the attention of each interaction unit will repeatedly focus on the original sentence representation of another one for semantic alignment, which alleviates the error propagation problem by attending to a fixed semantic representation. Then we design deep fusion to aggregate and propagate attention information from low layers to high layers, which effectively retains low-layer semantic features for subsequential interactions. Moreover, we introduce a self-attention mechanism at last to enhance global matching information with smaller model complexity. We conduct experiments on natural language inference and paraphrase identification tasks with three benchmark datasets SNLI, SciTail and Quora. Experimental results demonstrate that our proposed method can achieve significant improvements over baseline systems without using any external knowledge. Additionally, we conduct interpretable study to disclose how our deep interaction network with attention can benefit sentence matching, which provides a reference for future model design. Ablation studies and visualization analyses further verify that our model can better capture interactive information between two sentences, and the proposed components are indeed able to help modeling semantic relation more precisely.

Multiway Attention Networks for Modeling Sentence Pairs.

ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs.

Enhanced Attentive Convolutional Neural Networks for Sentence Pair Modeling

Two-Step Multi-Factor Attention Neural Network For Answer Selection

Multiway Attention for Neural Machine Translation

Word-Pair Relevance Modeling with Multi-View Neural Attention Mechanism for Sentence Alignment

Multi-Modality Cross Attention Network for Image and Sentence Matching

One Vs. Many QA Matching with Both Word-level and Sentence-level Attention Network.

A Multi-Size Neural Network with Attention Mechanism for Answer Selection

Inter-Weighted Alignment Network for Sentence Pair Modeling.

Modelling Sentence Pairs with Tree-structured Attentive Encoder

Multi-Matching Network for Multiple Choice Reading Comprehension

Multimodal Unified Attention Networks for Vision-and-Language Interactions

Deep Bi-Directional Interaction Network for Sentence Matching

Attention-Fused Deep Matching Network for Natural Language Inference

AHNN: an Attention-Based Hybrid Neural Network for Sentence Modeling

Multiway Dynamic Mask Attention Networks for Natural Language Inference.

Sentence Pair Semantic Enhanced Matching Network for Text Information Retrieval

Attention-Based Memory Network for Sentence-Level Question Answering.

Word-level Dual Channel with Multi-Head Semantic Attention Interaction for Community Question Answering

Attention-Based Multi-level Network for Text Matching with Feature Fusion