Abstract:Semantic matching of natural language sentences or identifying the relationship between two sentences is a core research problem underlying many natural language tasks. Depending on whether training data is available, prior research has proposed both unsupervised distance-based schemes and supervised deep learning schemes for sentence matching. However, previous approaches either omit or fail to fully utilize the ordered, hierarchical, and flexible structures of language objects, as well as the interactions between them. In this paper, we propose Hierarchical Sentence Factorization---a technique to factorize a sentence into a hierarchical representation, with the components at each different scale reordered into a "predicate-argument" form. The proposed sentence factorization technique leads to the invention of: 1) a new unsupervised distance metric which calculates the semantic distance between a pair of text snippets by solving a penalized optimal transport problem while preserving the logical relationship of words in the reordered sentences, and 2) new multi-scale deep learning models for supervised semantic training, based on factorized sentence hierarchies. We apply our techniques to text-pair similarity estimation and text-pair relationship classification tasks, based on multiple datasets such as STSbenchmark, the Microsoft Research paraphrase identification (MSRP) dataset, the SICK dataset, etc. Extensive experiments show that the proposed hierarchical sentence factorization can be used to significantly improve the performance of existing unsupervised distance-based metrics as well as multiple supervised deep learning models based on the convolutional neural network (CNN) and long short-term memory (LSTM).

Learning to Order Natural Language Texts.

Neural Sentence Ordering.

Deep Attentive Sentence Ordering Network.

Exploring hypergraph-based semi-supervised ranking for query-oriented summarization

Evaluating Text Coherence at Sentence and Paragraph Levels.

Learning to Organize a Bag of Words into Sentences with Neural Networks: an Empirical Study

Non-Autoregressive Sentence Ordering

Hierarchical Attention Networks for Sentence Ordering.

Local and Global Context-Based Pairwise Models for Sentence Ordering

Learning to Order Facts for Discourse Planning in Natural Language Generation

Topological Sort for Sentence Ordering

Order-Planning Neural Text Generation from Structured Data

Functional Analytics for Document Ordering for Curriculum Development and Comprehension

Machine Reading Comprehension: Matching and Orders

Text Reading Order in Uncontrolled Conditions by Sparse Graph Segmentation

Learning to Rank Semantic Coherence for Topic Segmentation.

Improving Sentence Similarity Estimation for Unsupervised Extractive Summarization

Automatic Text Summarization Based on Word Order

Learning to Rank Utterances for Query-Focused Meeting Summarization

Matching Natural Language Sentences with Hierarchical Sentence Factorization

Word Order's Impacts: Insights from Reordering and Generation Analysis