Abstract:Background: Text Matching (TM) is a fundamental task of natural language processing widely used in many application systems such as information retrieval, automatic question answering, machine translation, dialogue system, reading comprehension, etc. In recent years, a large number of deep learning neural networks have been applied to TM, and have refreshed benchmarks of TM repeatedly. Among the deep learning neural networks, convolutional neural network (CNN) is one of the most popular networks, which suffers from difficulties in dealing with small samples and keeping relative structures of features. In this paper, we propose a novel deep learning architecture based on capsule network for TM, called CapsTM, where capsule network is a new type of neural network architecture proposed to address some of the short comings of CNN and shows great potential in many tasks. Methods: CapsTM is a five-layer neural network, including an input layer, a representation layer, an aggregation layer, a capsule layer and a prediction layer. In CapsTM, two pieces of text are first individually converted into sequences of embeddings and are further transformed by a highway network in the input layer. Then, Bidirectional Long Short-Term Memory (BiLSTM) is used to represent each piece of text and attention-based interaction matrix is used to represent interactive information of the two pieces of text in the representation layer. Subsequently, the two kinds of representations are fused together by BiLSTM in the aggregation layer, and are further represented with capsules (vectors) in the capsule layer. Finally, the prediction layer is a connected network used for classification. CapsTM is an extension of ESIM by adding a capsule layer before the prediction layer. Results: We construct a corpus of Chinese medical question matching, which contains 36,360 question pairs. This corpus is randomly split into three parts: a training set of 32,360 question pairs, a development set of 2000 question pairs and a test set of 2000 question pairs. On this corpus, we conduct a series of experiments to evaluate the proposed CapsTM and compare it with other state-of-the-art methods. CapsTM achieves the highest F-score of 0.8666. Conclusion: The experimental results demonstrate that CapsTM is effective for Chinese medical question matching and outperforms other state-of-the-art methods for comparison.

Capsule Networks For Chinese Opinion Questions Machine Reading Comprehension

Bi-directional Capsule Network Model for Chinese Biomedical Community Question Answering

Hybrid Embedding and Joint Training of Stacked Encoder for Opinion Question Machine Reading Comprehension.

Multi-Matching Network for Multiple Choice Reading Comprehension

DCMN+: Dual Co-Matching Network for Multi-choice Reading Comprehension

Robust triple extraction with cascade bidirectional capsule network

Keyword-based Query Comprehending Via Multiple Optimized-Demand Augmentation.

CapsTM: capsule network for Chinese medical text matching

Interactive capsule network for implicit sentiment analysis

A question-oriented propagation network for news reading comprehension

Reinforced Mnemonic Reader for Machine Reading Comprehension

Dual Head-wise Coattention Network for Machine Comprehension with Multiple-Choice Questions

Research on a Capsule Network Text Classification Method with a Self-Attention Mechanism

Learn a prior question-aware feature for machine reading comprehension

Consensus Attention-based Neural Networks for Chinese Reading Comprehension

iCapsNets: Towards Interpretable Capsule Networks for Text Classification

An Ensemble Model For Winning A Chinese Machine Reading Comprehension Competition

Reinforced Mnemonic Reader for Machine Comprehension

Multi-paragraph Reading Comprehension with Token-level Dynamic Reader and Hybrid Verifier

To Answer or Not to Answer? Improving Machine Reading Comprehension Model with Span-based Contrastive Learning

Sentiment Analysis by Capsules