Abstract:1. INTRODUCTION Reusing information redundancy in question-answer pairs is one of the alternative approaches to question answering (QA) system. If the same question has been asked by other users, the QA system responses to such question using the answer associated with the redundant question. Nevertheless, the task of identifying similarity of questions is not trivial. Traditional text similarity measures are neither effective nor efficient in distinguishing the similarity of sentence-level text. Document similarity techniques are not effective since the length of sentence text is rather short and contains very little word overlap. Furthermore, the similarity and relevance of sentences can be characterized into different levels, which is difference than a standard topicality notion used in document retrieval. In this paper, we focus on the problem of identifying questions that express the same information need. The main goal is to match questions with their paraphrases. To achieve this, we propose a hybrid question similarity approach that combines semantic, syntactic, and question type similarity. Semantic and syntactic information is measured by taking into account word similarity, word ordering, and parts of speech information. Information about the types of question is derived from a Support Vector Machine classifier. The experimental results have shown that our approach is highly effective in detecting redundant questions. For many years, knowledge-sharing community sites, such as Yahoo! Answers, have been accepting a large amount of questions from millions of users. Given the current magnitude of questions and answers in their archive, it is plausible that a newly submitted question has already been asked by the other users. However, finding such similar questions is ineffective due to the inherited limitation of the current search engines. Standard text retrieval approaches that compute the similarity of a document-level text are neither effective nor efficient for matching natural language questions. First, the fundamental principle of document similarity techniques is based on the degree of word overlaps. This notion works well in distinguishing similar documents since they are likely to contain sufficient number of words in common. On the other hand, the length of question phrases is relatively short and often contains very few word overlap. Furthermore, due to the generative power of natural language, the same question can be expressed in various ways. Hence, most questions are likely to receive a low similarity score from document similarity measures. The notion of topical relevance, which is central to the standard information retrieval systems, …

Research on the Text Similarity Algorithms in Automatic Scoring of Subjective Questions

An adaptive method for text domain similarity calculation

Automated Correcting Algorithm for Subjective Questions Based on Segmentation Technology and Sentence Similarity

Measurement of Text Similarity: A Survey

A Method of Computer Automatic Scoring for Subjective Questions

Design of Exercise Grading System Based on Text Similarity Computing

Research on Chinese Semantic Similarity Algorithm

A survey on the techniques, applications, and performance of short text semantic similarity

Research on Word Similarity and Its Application in English Auxiliary Writing

A Survey of Text-Matching Techniques

Question Similarity Identification in Automatic Generation of Test Papers

Utilizing Sentence Similarity and Question Type Similarity to Response to Similar Questions in Knowledge-Sharing Community

Automated Chinese-English translation scoring based on answer knowledge base

Automatically Measuring Subjectivity of Chinese Sentences for Sentiment Analysis to Reviews on the Internet

Survey on Sentiment Orientation Analysis of Texts

A Study on Automatic Scoring for Machine Translation Systems

Study on Topic Partition in Automatic Abstracting System

Study on Topic Partition Based on Sequential Paragraphic Similarity

Short Text Similarity Calculation Using Semantic Information

A Survey on Deep Text Matching

An automated essay scoring systems: a systematic literature review