Dynamic Snippet Algorithm for Search Engine

SHU Bai-han,YAN Hong-fei

2009-01-01

Abstract:Selection principles and algorithm rules of search engine snippets are not clearly stipulated.Providing a formal principle of algorithm implementation based on a query of dynamic snippet,dynamic snippet algorithm is completed under this framework.Compared with Baidu,the experimental results show that the coherence of this algorithm exceeds 6%.

What problem does this paper attempt to address?

Query Biased Snippet Generation in XML Search

Yu Huang,Ziyang Liu,Yi Chen

DOI: https://doi.org/10.1145/1376616.1376651

2008-01-01

Abstract:Snippets are used by almost every text search engine to complement ranking scheme in order to effectively handle user searches, which are inherently ambiguous and whose relevance semantics are difficult to assess. Despite the fact that XML is a standard representation format of web data, research on generating result snippets for XML search remains untouched. In this paper we present a system, eXtract, which addresses this important yet open problem. We identify that a good XML result snippet should be a self-contained meaningful information unit of a small size that effectively summarizes this query result and differentiates it from others, according to which users can quickly assess the relevance of the query result. We have designed and implemented a novel algorithm to satisfy these requirements and verified its efficiency and effectiveness through experiments.
Improving XML Search by Generating and Utilizing Informative Result Snippets

Ziyang Liu,Yu Huang,Yi Chen

DOI: https://doi.org/10.1145/1806907.1806911

IF: 1.6289

2010-01-01

ACM Transactions on Database Systems

Abstract:Snippets are used by almost every text search engine to complement the ranking scheme in order to effectively handle user searches, which are inherently ambiguous and whose relevance semantics are difficult to assess. Despite the fact that XML is a standard representation format of Web data, research on generating result snippets for XML search remains limited. To tackle this important yet open problem, in this article, we present a system eXtract which generates snippets for XML search results. We identify that a good XML result snippet should be a meaningful information unit of a small size that effectively summarizes this query result and differentiates it from others, according to which users can quickly assess the relevance of the query result. We have designed and implemented a novel algorithm to satisfy these requirements. Furthermore, we propose to cluster the query results based on their snippets. Since XML result clustering can only be done at query time, snippet-based clustering significantly improves the efficiency while compromising little clustering accuracy. We verified the efficiency and effectiveness of our approach through experiments.
Improving Search Snippets in Context-Aware Web Search Scenarios.

Jia Chen,Jiaxin Mao,Yiqun Liu,Min Zhang,Shaoping Ma

DOI: https://doi.org/10.1007/978-3-030-56725-5_1

2020-01-01

Abstract:As an essential part in web search, search snippets usually provide result previews for users to either gather useful information or make click-through decisions. In complex search scenarios, users may need to submit multiple queries to search systems until their information needs are satisfied. As user intents tend to be ambiguous, incorporating contextual information for user modeling has been proved effective in many session-level tasks. Therefore, the generation of search snippets may also benefit from the integration of context information. However, to our best knowledge, most existing snippet generation methods ignore user interaction and focus merely on the query content. Whether it is useful of exploiting session contexts to improve search snippets still remains inscrutable. To this end, we propose a snippet generation model which considers session contexts. The proposed method utilizes the query sequence as well as users’ interaction behaviors within a session to model users’ session-level information needs. We also adopt practical log-based search data to evaluate the performance of the proposed method. Experiment results based on both expert annotation and user preference test show the effectiveness of considering contextual information in search snippet generation.
Refine Software Q&A Document Search Results Based on Code Pattern

Chenyan HUA,Yanzhen ZOU,Zixiao ZHU,Bing XIE

DOI: https://doi.org/10.3778/j.issn.1673-9418.1609028

2017-01-01

Abstract:Developers often need to search related software Q&A documents in Q&A website. In the search results, the Q&A documents which contain good code snippets (usage examples) are preferred. However, how to metric those code snippets in document is still a big challenge. To address this issue, this paper proposes an approach for refining software Q&A document search results based on code pattern. Firstly, code snippets are extracted from each document in the search results. Then, the common code patterns are mined and used to measure the quality of those code snippets. Finally, the documents with high quality are recommended and ranked at the top of the search results. In the experiments, this paper carries out some evaluations with 10 real problems that software developers meet in practice. Compared to the search results of StackOverflow, the proposed approach has an increment of 40% at NDCG@5.
Semantic snippet construction for search engine results based on segment evaluation

K. S. Kuppusamy,G. Aghila

DOI: https://doi.org/10.48550/arXiv.1202.2614

2012-02-13

Abstract:The result listing from search engines includes a link and a snippet from the web page for each result item. The snippet in the result listing plays a vital role in assisting the user to click on it. This paper proposes a novel approach to construct the snippets based on a semantic evaluation of the segments in the page. The target segment(s) is/are identified by applying a model to evaluate segments present in the page and selecting the segments with top scores. The proposed model makes the user judgment to click on a result item easier since the snippet is constructed semantically after a critical evaluation based on multiple factors. A prototype implementation of the proposed model confirms the empirical validation.

Information Retrieval
An online clustering algorithm for Chinese web snippets based on Generalized Suffix Array

Zhang Hui,Wang Han,Yang Gao,Zhou Jingmin

DOI: https://doi.org/10.1109/CYBERC.2009.5342183

2009-01-01

Abstract:As the information on the Internet increases dramatically, the web search engine has become an indispensable tool to search and locate the required information. Web snippets clustering can classify the search results and help users to narrow the search scope. This paper presents an online clustering algorithm for Chinese web snippets using common substrings. The algorithm firstly preprocesses the results of a search engine and extracts common substrings using Generalized Suffix Array. Then it builds a snippet-snippet similarity matrix by calculating similarities between every two snippets using common substring-based dimensional model. At last, the algorithm groups the web snippets using an improved hierarchical clustering algorithm. Theoretical analysis and experiments show that compared to traditional Chinese web snippet clustering algorithms based on Chinese word segmentation, our algorithm performs better both in the efficiency of clustering and the readability of the generated cluster labels. © 2009 IEEE.
Detailed Technique Report on XMLSnippet Yiqi

Lu,Sheng Huang,Yanghua Xiao

2011-01-01

Abstract:XMLSnippet is a tool to recommend XML snippets automatically by mining structural patterns from application to assist programmer to generate proper XML configuration during product phase. These actions are realized by mining closed frequent tree patterns from existing projects without awareness of the syntax of the specific frameworks. XMLSnippet is built on two major techniques that are borrowed from database communities: closed frequent XML tree pattern mining and sub-tree query. In this paper, we will describe those major techniques used in XMLSnippet offline mining in detail. 1 The framework of XMLSnippet In the design of XMLSnippet, we want to achieve following objectives. 1. The tool should be syntax neutral to support frameworks with different syntax. The tool should be able to extract generalized patterns over all kinds of frameworks. 2. The tool should be able to provide online intelligent coding suggestions with reusable XML Snippet when the programmer is editing the XML configuration files. The architecture of XMLSnippet is shown in Figure 1. The architecture consists of three major phases: pre-processing, offline mining and online query. In phase 1, application repositories are first preprocessed, and then the result (XML Repository) will be fed to closed frequent sub-tree mining. Closed sub-tree patterns in the XML configuration files are of special interest to us since they contains enough information of editing rules in framework-based programming. All the resulting patterns are stored in XML tree pattern database, which will be indexed by prefix tree. Sample source codes will be indexed by inverted index that will be used to speedup CQ. The cores of XMLSnippet are closed frequent tree mining and prefix tree/source code indexing. Frequent tree pattern mining distinguishes our tool from other code recommendation tools that utilizes frequent item set patterns[3], or sequential patterns[4]. Online query phase is responsible for providing effective online assistance for programmers.
PKU at INEX 2011 XML Snippet Track.

Songlin Wang,Yihong Hong,Jianwu Yang

DOI: https://doi.org/10.1007/978-3-642-35734-3_33

2012-01-01

Abstract:Snippets are used by almost every text search engine to complement ranking scheme in order to effectively handle user's searching, which are inherently ambiguous and whose relevance semantics are difficult to assess. In this paper, we present our participation in the INEX 2011 Snippet track. A efficiently retrieval system has been used, by which the semi-structured of document has been considered, and we present a snippet generate system, which effectively summarize the query results and document content, according to which users can quickly assess the relevance of the query results. © Springer-Verlag Berlin Heidelberg 2012.
Effect of Snippet Text Bolding in Search User Behavior

ZHANG Hui,SU Ning,LIU Yiqun,MA Shaoping

DOI: https://doi.org/10.16511/j.cnki.qhdxxb.2018.25.033

2018-01-01

Abstract:Search users rely on result captions including titles, snippets and URLs to decide whether they should click a particular result.Snippets usually serve as a query-dependent summary of its corresponding landing page and are, therefore, one of the most important factors in the search interaction process.At present, commercial search engines use query bolding strategies, but these have various problems and lack useful information.This paper presents a bolding strategy that improves user search efficiency.The method includes three bolding strategies based on crowd sourcing results which differ from the query terms strategy.Tests show that the search behavior is affected by the term bolding strategies without changes in the snippet contents.The tests also show that the responses to the three bolding strategies are better than responses to the query terms bolding strategy to produce a better bolding strategy.The appropriate bolding numbers, bolding ratio, and targeted information have a very positive impact on the user's search behavior.
Effective and Efficient Query-aware Snippet Extraction for Web Search

Jingwei Yi,Fangzhao Wu,Chuhan Wu,Xiaolong Huang,Binxing Jiao,Guangzhong Sun,Xing Xie

DOI: https://doi.org/10.48550/arXiv.2210.08809

2022-10-27

Abstract:Query-aware webpage snippet extraction is widely used in search engines to help users better understand the content of the returned webpages before clicking. Although important, it is very rarely studied. In this paper, we propose an effective query-aware webpage snippet extraction method named DeepQSE, aiming to select a few sentences which can best summarize the webpage content in the context of input query. DeepQSE first learns query-aware sentence representations for each sentence to capture the fine-grained relevance between query and sentence, and then learns document-aware query-sentence relevance representations for snippet extraction. Since the query and each sentence are jointly modeled in DeepQSE, its online inference may be slow. Thus, we further propose an efficient version of DeepQSE, named Efficient-DeepQSE, which can significantly improve the inference speed of DeepQSE without affecting its performance. The core idea of Efficient-DeepQSE is to decompose the query-aware snippet extraction task into two stages, i.e., a coarse-grained candidate sentence selection stage where sentence representations can be cached, and a fine-grained relevance modeling stage. Experiments on two real-world datasets validate the effectiveness and efficiency of our methods.

Artificial Intelligence
A Framework for Evaluating Snippet Generation for Dataset Search

Xiaxia Wang,Jinchi Chen,Shuxin Li,Gong Cheng,Jeff Z. Pan,Evgeny Kharlamov,Yuzhong Qu

DOI: https://doi.org/10.48550/arXiv.1907.01183

2019-07-02

Abstract:Reusing existing datasets is of considerable significance to researchers and developers. Dataset search engines help a user find relevant datasets for reuse. They can present a snippet for each retrieved dataset to explain its relevance to the user's data needs. This emerging problem of snippet generation for dataset search has not received much research attention. To provide a basis for future research, we introduce a framework for quantitatively evaluating the quality of a dataset snippet. The proposed metrics assess the extent to which a snippet matches the query intent and covers the main content of the dataset. To establish a baseline, we adapt four state-of-the-art methods from related fields to our problem, and perform an empirical evaluation based on real-world datasets and queries. We also conduct a user study to verify our findings. The results demonstrate the effectiveness of our evaluation framework, and suggest directions for future research.

Information Retrieval,Databases
Dynamic Analysis of Chinese Search Engine User Behavior

MA Shaoping,LIU Yiqun,LIU Jian,ZHANG Min,ZHU Jianhua,RU Liyun

DOI: https://doi.org/10.3969/j.issn.1003-0077.2011.06.012

2011-01-01

Abstract:Search engine has been one of the most important information acquisition tools on the Web.To meet users' information needs,most commercial search engines rely on user behavior analysis to improve the performance of result ranking,data quality estimation,Web spam detection and other related techniques.However,these works seldom focus on long-term dynamic analysis of user behavior,which may be essential for both system architecture and user interface designing of future search techniques.Based on a large-scale user behavior data provided by a most popular Chinese search engine,search behavior between 2006 and 2011 was studied,producing many findings which may help us better understand how users grow with search engines.
Effective Time Ratio: A Measure for Web Search Engines with Document Snippets

Jing He,Baihan Shu,Xiaoming Li,Hongfei Yan

DOI: https://doi.org/10.1007/978-3-642-17187-1_7

2010-01-01

Abstract:The dominant method for evaluating search engines is the Cranfield paradigm, but the existing metrics do not consider some modern search engines features, such as document snippets. In this paper, we propose a new metric effective time ratio for search engine evaluation. Effective time ratio measures the ratio between effective time a user spent on getting relevant information and the total search time. For retrieval system without presenting document snippet, its value is identical to precision. For search engine with snippet, some theoretical analysis proves that its value can reflect both retrieval system performance and snippet quality. We further deploy a real user study, showing that effective time ratio can reflect users’ satisfaction better than the existing metrics based on document relevance and/or snippet relevance.
On the Extraction of Search-Oriented Snips for Popular Files from the Internet

GUO Hua-nan,LEI Kai,LI Xiao-ming

DOI: https://doi.org/10.3969/j.issn.1000-5862.2008.02.002

2008-01-01

Abstract:This paper describes a method for generating a set of keywords for a file,which is supposed to have strong similarity with the set of queries that a user most probably uses to search for the file. The generating procedure is presented and the algorithm is described in detail. Experiment is conducted to show the effectiveness of the method.
Beyond Query-Oriented Highlighting: Investigating the Effect of Snippet Text Highlighting in Search User Behavior

Hui Zhang

DOI: https://doi.org/10.1155/2018/7836969

IF: 3.12

2018-01-01

Computational Intelligence and Neuroscience

Abstract:Search users rely on result captions including titles, snippets, and URLs to decide whether they should read and click a particular result or not. Snippet usually serves as a query-dependent summary of its corresponding landing page and is therefore treated as one of the most important factors in search interaction process. Although there exist many efforts in improving snippet generation algorithms and incorporating more powerful interaction functions into snippets, little attention is paid to the effect of text highlighting in user behaviors. The highlighting of query terms in search snippets has been regarded as a matter of course and whether there exists a better way in snippet text highlighting remains uninvestigated. In this paper, we try to find out whether the default strategy of highlighting query terms employed by most commercial search engines is the best for search users. Through carefully designed experiments, we show that the retrieval efficiency can be affected by different term-highlighting strategies without changes in snippet contents. We also propose an automatic method which adopts CRF to learn to highlight terms based on word embedding, Wikipedia, and snippet content information. Experimental results show that the proposed method could predict highlighted terms selected by crowd workers with moderate performance.
Deep Semantics-Enhanced Neural Code Search

Ying Yin,Longfei Ma,Yuqi Gong,Yucen Shi,Fazal Wahab,Yuhai Zhao

DOI: https://doi.org/10.3390/electronics13234704

IF: 2.9

2024-11-30

Electronics

Abstract:Code search uses natural language queries to retrieve code snippets from a vast database, identifying those that are semantically similar to the query. This enables developers to reuse code and enhance software development efficiency. Most existing code search algorithms focus on capturing semantic and structural features by learning from both text and code graph structures. However, these algorithms often struggle to capture deeper semantic and structural features within these sources, leading to lower accuracy in code search results. To address this issue, this paper proposes a novel semantics-enhanced neural code search algorithm called SENCS, which employs graph serialization and a two-stage attention mechanism. First, the code program dependency graph is transformed into a unique serialized encoding, and a bidirectional long short-term memory (LSTM) model is used to learn the structural information of the code in the graph sequence to generate code vectors rich in structural features. Second, a two-stage attention mechanism enhances the embedded vectors by assigning different weight information to various code features during the code feature fusion phase, capturing significant feature information from different code feature sequences, resulting in code vectors rich in semantic and structural information. To validate the performance of the proposed code search algorithm, extensive experiments were conducted on two widely used code search datasets, CodeSearchNet and JavaNet. The experimental results show that the proposed SENCS algorithm improves the average code search accuracy metrics by 8.30 % (MRR) and 17.85% (DCG) and compared to the best baseline code search model in the literature, with an average improvement of 14.86% in the SR@1 metric. Experiments with two open-source datasets demonstrate SENCS achieves a better search effect than state of-the-art models.

engineering, electrical & electronic,computer science, information systems,physics, applied
BANDAR: Benchmarking Snippet Generation Algorithms for (RDF) Dataset Search

Xiaxia Wang,Gong Cheng,Jeff Z. Z. Pan,Evgeny Kharlamov,Yuzhong Qu

DOI: https://doi.org/10.1109/tkde.2021.3095309

IF: 9.235

2021-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:The large volume of open data on the Web is expected to be reused and create value. Finding the right data to reuse is a non-trivial task addressed by the recent dataset search systems, which retrieve datasets relevant to a keyword query. An important component of such systems is snippet generation, extracting data from a retrieved dataset to exemplify its content and explain its relevance to the query. Snippet generation algorithms have emerged but were mainly evaluated by user studies. More efficient and reproducible evaluation methods are needed. To meet this challenge, in this article, we present a set of quality metrics for assessing the usefulness of a snippet from different perspectives, and we select and aggregate them into quality profiles for different stages of a dataset search process. Furthermore, we create a benchmark from thousands of collected real-world data needs and datasets, on which we apply the presented quality metrics and profiles to evaluate snippets generated by two existing algorithms and three adapted algorithms. The results, which are reproducible as they are automatically computed without human interaction, show the pros and cons of the tested algorithms and highlight directions for future research. The benchmark data is publicly available.
Research and Evaluation of Near-replicas of Web Pages Detection Algorithms

Jian-yong WANG,Zheng-mao XIE,Ming LEI,Xiao-ming LI

DOI: https://doi.org/10.3321/j.issn:0372-2112.2000.Z1.033

2000-01-01

Tien Tzu Hsueh Pao/Acta Electronica Sinica

Abstract:Many documents are replicated across the World-wide Web. How to efficiently and accurately find the near-replicas of web pages becomes an important topic in the search engine research area, which can be used to improve the quality of searching service. We propose five near-replicas detection algorithms for search engines that rely on keyword matching, and evaluate them using the WebGather search engine system. In addition, we also compare our method with one of the most popular copy detection mechanisms. Our method is successfully adopted to remove the near-replicas of web pages in WebGather, and it can also be widely used to build library.
Paraphrasing with search engine query logs

Shiqi Zhao,Haifeng Wang,Ting Liu

2010-01-01

Abstract:This paper proposes a method that extracts paraphrases from search engine query logs. The method first extracts paraphrase query-title pairs based on an assumption that a search query and its corresponding clicked document titles may mean the same thing. It then extracts paraphrase query-query and title-title pairs from the query-title paraphrases with a pivot approach. Paraphrases extracted in each step are validated with a binary classifier. We evaluate the method using a query log from Baidu, a Chinese search engine. Experimental results show that the proposed method is effective, which extracts more than 3.5 million pairs of paraphrases with a precision of over 70%. The results also show that the extracted paraphrases can be used to generate high-quality paraphrase patterns.
How Do Users Describe Their Information Need: Query Recommendation Based on Snippet Click Model

Yiqun Liu,Junwei Miao,Min Zhang,Shaoping Ma,Liyun Ru

DOI: https://doi.org/10.1016/j.eswa.2011.04.188

IF: 8.5

2011-01-01

Expert Systems with Applications

Abstract:Query recommendation helps users to describe their information needs more clearly so that search engines can return appropriate answers and meet their needs. State-of-the-art researches prove that the use of users’ behavior information helps to improve query recommendation performance. Instead of finding the most similar terms previous users queried, we focus on how to detect users’ actual information need based on their search behaviors. The key idea of this paper is that although the clicked documents are not always relevant to users’ queries, the snippets which lead them to the click most probably meet their information needs. Based on analysis into large-scale practical search behavior log data, two snippet click behavior models are constructed and corresponding query recommendation algorithms are proposed. Experimental results based on two widely-used commercial search engines’ click-through data prove that the proposed algorithms outperform practical recommendation methods of these two search engines. To the best of our knowledge, this is the first time that snippet click models are proposed for query recommendation task.

Dynamic Snippet Algorithm for Search Engine

Query Biased Snippet Generation in XML Search

Improving XML Search by Generating and Utilizing Informative Result Snippets

Improving Search Snippets in Context-Aware Web Search Scenarios.

Refine Software Q&A Document Search Results Based on Code Pattern

Semantic snippet construction for search engine results based on segment evaluation

An online clustering algorithm for Chinese web snippets based on Generalized Suffix Array

Detailed Technique Report on XMLSnippet Yiqi

PKU at INEX 2011 XML Snippet Track.

Effect of Snippet Text Bolding in Search User Behavior

Effective and Efficient Query-aware Snippet Extraction for Web Search

A Framework for Evaluating Snippet Generation for Dataset Search

Dynamic Analysis of Chinese Search Engine User Behavior

Effective Time Ratio: A Measure for Web Search Engines with Document Snippets

On the Extraction of Search-Oriented Snips for Popular Files from the Internet

Beyond Query-Oriented Highlighting: Investigating the Effect of Snippet Text Highlighting in Search User Behavior

Deep Semantics-Enhanced Neural Code Search

BANDAR: Benchmarking Snippet Generation Algorithms for (RDF) Dataset Search

Research and Evaluation of Near-replicas of Web Pages Detection Algorithms

Paraphrasing with search engine query logs

How Do Users Describe Their Information Need: Query Recommendation Based on Snippet Click Model