Abstract:Latent Semantic Analysis (LSA) is widely used for finding the documents whose semantic is similar to the query of keywords. Although LSA yield promising similar results, the existing LSA algorithms involve lots of unnecessary operations in similarity computation and candidate check during on-line query processing, which is expensive in terms of time cost and cannot efficiently response the query request especially when the dataset becomes large. In this paper, we study the efficiency problem of on-line query processing for LSA towards efficiently searching the similar documents to a given query. We rewrite the similarity equation of LSA combined with an intermediate value called partial similarity that is stored in a designed index called partial index. For reducing the searching space, we give an approximate form of similarity equation, and then develop an efficient algorithm for building partial index, which skips the partial similarities lower than a given threshold θ. Based on partial index, we develop an efficient algorithm called ILSA for supporting fast on-line query processing. The given query is transformed into a pseudo document vector, and the similarities between query and candidate documents are computed by accumulating the partial similarities obtained from the index nodes corresponds to non-zero entries in the pseudo document vector. Compared to the LSA algorithm, ILSA reduces the time cost of on-line query processing by pruning the candidate documents that are not promising and skipping the operations that make little contribution to similarity scores. Extensive experiments through comparison with LSA have been done, which demonstrate the efficiency and effectiveness of our proposed algorithm.

LATENT SEMANTIC INDEXING (LSI) AND ITS APPLICATIONS IN CHINESE TEXT PROCESSING

Improving Multi-Faceted Book Search by Incorporating Sparse Latent Semantic Analysis of Click-Through Logs

Query Subtopic Mining Via Subtractive Initialization of Non-negative Sparse Latent Semantic Analysis

Document Clustering Using Locality Preserving Indexing

An adaptive method for text domain similarity calculation

Text Information Retrieval Based on Latent Semantic Analysis

A Simplified Latent Semantic Indexing Approach for Multi-Linguistic Information Retrieval.

The Theory and Application of Latent Semantic Analysis

Improving Text Classification Using Local Latent Semantic Indexing

Locality Preserving Indexing for Document Representation.

Supervised latent semantic indexing for document categorization

Clustering-based Semantic Retrieval Algorithm

An Application Of Lexical Semantics Of Chinese In Webpage Keyword Extraction Algorithm

Modeling Image Data for Effective Indexing and Retrieval in Large General Image Databases.

An Index-Based Algorithm for Fast On-Line Query Processing of Latent Semantic Analysis

Understanding and Enhancing the Folding-In Method in Latent Semantic Indexing.

A latent image semantic indexing scheme for image retrieval on the web

AN APPROACH TO SEMANTICS-BASED IMAGE RETRIEVAL AND BROWSING

To Predict L2 Writing Quality Using Lexical Richness Indices: an Investigation of Learners of Chinese As a Foreign Language

An Ontology-Based Latent Semantic Indexing Approach Using Long Short-Term Memory Networks.

A Linkage Clustering Based Query Expansion Algorithm