Hybrid Strategy to Distributed Index Organization in Search Engine

CHEN Wei,LIU Kang-miao,BU Jia-jun,CHEN Chun,ZHANG Li-jun
DOI: https://doi.org/10.3785/j.issn.1008-973x.2009.08.002
2009-01-01
Abstract:A hybrid index organization strategy named Loc-Glob was proposed to enhance the query performance and scalability in search engine.Loc-Glob integrates two well-studied index partitioning schemes,which are widely used in search engines.Firstly,index is partitioned according to local(or global) index organization strategy,taking cluster of some index servers as a single machine.Then,index distributed to certain cluster are further partitioned to index servers according to the global(or local) index organization strategy inside the cluster.Loc-Glob is more scalable than the traditional strategies to accommodate the explosively growing web pages.Experimental results indicate that the throughput of Loc-Glob outperforms the global index organization while it is very close to the local index organization,and Loc-Glob provides good load-balancing level.
What problem does this paper attempt to address?