Research on Expert Search at Enterprise Track of TREC 2005.

Yunbo Cao,Jingjing Liu,Shenghua Bao,Hang Li

2005-01-01

Abstract:We (MSRA team) participated in the Expert Search task at the Enterprise Track of TREC 2005. This document reports our experimental results on expert search.

What problem does this paper attempt to address?

THUIR at TREC 2007: Enterprise Track

Yupeng Fu,Yufei Xue,Tong Zhu,Yiqun Liu,Min Zhang,Shaoping Ma

2007-01-01

Abstract:We participate in document search and expert search of Enterprise Track in TREC2007. The motive behind the TREC Enterprise Track is to study the issues searching the documents and experts inside an enterprise environment, which has not been sufficiently addressed in research. In document search, we focus on the key overview page pre-selection methods and link analysis algorithms. In expert search, we develop methods to detect expert identifiers and experimented based on our previous PDD model.
THUIR at TREC2007: Enterprise track 1

Yupeng Fu,Yufei Xue,Tong Zhu,Yiqun Liu,Min Zhang,Shaoping Ma

2007-01-01

Abstract:We participate in document search and expert search of Enterprise Track in TREC2007. The motive behind the TREC Enterprise Track is to study the issues searching the documents and experts inside an enterprise environment, which has not been sufficiently addressed in research. In document search, we focus on the key overview page pre-selection methods and link analysis algorithms. In expert search, we develop methods to detect expert identifiers and experimented based on our previous PDD model.
THUIR at TREC2008: Enterprise track

Yufei Xue,Tong Zhu,Guichun Hua,Min Zhang,Yiqun Liu,Shaoping Ma

2008-01-01

Abstract:We participate in document search and expert search of Enterprise Track in TREC2008. The corpus and tasks are same as the year before. Different from TREC 2007, the topics come from CSIRO Enquiries, and the topic statements are richer and more colloquial.. In document search, we look into the key resource page pre-selection, the use of anchor text, query classification, and multi-field search. In expert search, we develop methods to detect expert identifiers and experimented based on our previous PDD (personal description documents) model.
CNDS Expert Finding System for TREC 2005.

Conglei Yao,Bo Peng,Jing He,Zhifeng Yang

DOI: https://doi.org/10.6028/nist.sp.500-266.enterprise-pekingu.yan

2005-01-01

Abstract:This paper describes our system developed for Expert Finding task of Enterprise Track for TREC2005. This system employs 3 methods, traditional IR method, email clustering method and entry page finding method, to find experts related to a specific topic in W3C corpus. Experiment indicates that traditional IR method is useful to expert finding if the query is well generated, email clustering method is helpful when the mail list is relevant to a unique work group or committee, and entry page finding method is valuable while the topic is the theme of a special group. We use result aggregation methods of linear synthesis to combine the results generated by the three methods,. Of our 5 runs submitted for Expert Finding task, the best run is the one generated by linear synthesis, providing a MAP score of 0.2174(Bpref of 0.4299 and p@10 of 0.3460)
A Two-Stage Model for Expert Search

Yunbo Cao,Jingjing Liu,Shenghua Bao,Hang Li,Nick Craswell

2008-01-01

Abstract:This paper is concerned with expert search, a search task where the user types a query representing a topic and the search system returns a ranked list of people who are considered experts on the topic. The system does this using the evidence existing in a document collection. We proposed a model for performing the task in our TREC 2005 submission, referred to as two-stage model. Since that, a number of groups adopted the model in their systems and achieved good performances in their TREC submissions. This paper aims to give a comprehensive description on the two-stage model and provide more experimental results on the use of the model. Two-stage model is capable of employing many types of association relationships among query terms, documents and people (experts). The model consists of two parts: relevance model and co-occurrence model. The relevance model characterizes the relevance of documents to queries. The co- occurrence model characterizes the co-occurrence between people and terms (i.e., queries) in various types. In this paper, we report our new experimental results on two-stage model using the data in both TREC 2005 and TREC 2006 expert search tasks. We also show that our approach is applicable beyond TREC W3C corpus, by introducing experimental results on expert finding at Microsoft Research1. Our experimental results, once again, demonstrate the effectiveness of the two-stage model.
THUIR at TREC 2005: Enterprise Track.

Yupeng Fu,Wei Yu,Yize Li,Yiqun Liu,Min Zhang,Shaoping Ma

2008-01-01

Abstract:IR group of Tsinghua University participated in the expert finding task of TREC2005 enterprise track this year. We developed a novel method which is called document reorganization to solve the problem of locating expert for certain query topics. This method collects and combines related information from different media formats to organize a document which describes an expert candidate. This method proves both effective and efficient for expert finding task. Our submitted run (THUENT0505) obtains the best performance in all participants with evaluation metric MAP. The reorganized documents are also significantly smaller in size than the original corpus.
TREC-10 Web Track Experiments at MSRA.

Jianfeng Gao,Guihong Cao,Hongzhao He,Min Zhang,Jian-Yun Nie,Steve Walker,Stephen E. Robertson

DOI: https://doi.org/10.6028/nist.sp.500-250.web-microsoft-china

2001-01-01

Abstract:In TREC-10, Microsoft Research Asia (MSRA) participated in the Web track (ad hoc retrieval task and homepage finding task). The latest version of the Okapi system (Windows 2000 version) was used. We focused on the developing of content-based retrieval and linkbased retrieval, and investigated the suitable combination of the two. For content-based retrieval, we examined the problems of weighting scheme, re-weighting and pseudo-relevance feedback (PRF). Then we developed a method called collection refinement (CE) for QE. We investigated the use of two kinds of link information, link anchor and link structure. We used anchor descriptions instead of content text to build index. Furthermore, different search strategies, such as spreading activation and PageRank, have been tested. Experimental results show: (1) Okapi system is robust and effective for web retrieval. (2) In ad hoc task, content-based retrieval achieved much better performance, and the impact of anchor text can be neglected; while for homepage finding task, both anchor text and content text provide useful information contributing more on precision and recall respectively. (3) Although query expansion does not show any improvement in our web retrieval experiments, we believe that there are still potential for CE.
MSRA Att TRECVID 2008: High-Level Feature Extraction and Automatic Search.

Tao Mei,Zheng-Jun Zha,Yuan Liu,Meng Wang,Guo-Jun Qi,Xinmei Tian,Jingdong Wang,Linjun Yang,Xian-Sheng Hua

2008-01-01

Abstract:This paper describes the MSRA experiments for TRECVID 2008. We performed the experiments in high-level feature extraction and automatic search tasks. For high-level feature extraction, we representatively investigated the benefit of global and local low-level features by a variety of learning-based methods, including supervised and semi-supervised learning algorithms. For automatic search, we focused on text and visual baseline, query-independent learning, and various reranking methods.
WIM at TREC 2005.

Junyu Niu,Lin Sun,Luqun Lou,Fang Deng,Chen Lin,Haiqing Zheng,Xuanjing Huang

2005-01-01

Abstract:This paper describes the three TREC tasks we participated in this year, which are, Genomics track’s categorization task and ad hoc task, and Enterprise track’s known item search task. For the categorization task, we adopt a domain-specific terms extraction method and an ontology-based method for feature selection. A SVM classifier and a Rocchio based two staged classifier were also used in this experiment. For the ad-hoc task, we used BM25 algorithm, probabilistic model and query expansion. For the Enterprise track, language model was adopted, and entity recognition was also implemented in our experiment.
TREC 2006 at Maryland: Blog, Enterprise, Legal and QA Tracks

Douglas W. Oard,Tamer Elsayed,Jianqiang Wang,Yejun Wu,Pengyi Zhang,Eileen G. Abels,Jimmy Lin,Dagobert Soergel

DOI: https://doi.org/10.21236/ada463747

2006-01-01

Abstract:Abstract : In TREC 2006, teams from the University of Maryland participated in the Blog track, the Expert Search task of the Enterprise track, the Complex Interactive Question Answering task of the Question Answering track, and the Legal track. This paper reports our results.
Peking University at the TREC 2006 Terabyte Track

Li Jinging,Yan Hongfei

DOI: https://doi.org/10.6028/nist.sp.500-272.terabyte-pekingu.yan

2006-01-01

Abstract:paper details the experiments carried out at TREC 2006 Terabyte Track using Indri Search Engine. There were three tasks in the Terabyte track of TREC 2006, i.e. efficiency task, ad hoc task and named page finding task. We participated in two tasks, and submitted 5 runs for ad hoc task and 3 runs for named page task respectively. In ad hoc task, we looked at the importance of term proximity. In named page finding task, we cared more about the information of document structure and document prior.
MSRA-USTC-SJTU at TRECVID 2007: High-Level Feature Extraction and Search.

Tao Mei,Xian-Sheng Hua,Wei Lai,Linjun Yang,Zheng-Jun Zha,Yuan Liu,Zhiwei Gu,Guo-Jun Qi,Meng Wang,Jinhui Tang,Xun Yuan,Zheng Lu,Jingjing Liu

2007-01-01

Abstract:This paper describes the MSRA-USTC-SJTU experiments for TRECVID 2007. We performed the experiments in high-level feature extraction and automatic search tasks. For high-level feature extraction, we investigated the benefit of unlabeled data by semi-supervised learning, and the multi-layer (ML) multi-instance (MI) relation embedded in video by MLMI kernel, as well as the correlations between concepts by correlative multi-label learning. For automatic search, we fuse text, visual example, and concept-based models while using temporal consistency and face information for re-ranking and result refinement.
Search Task Success Evaluation by Exploiting Multi-View Active Semi-Supervised Learning.

Ling Chen,Alin Fan,Hongyu Shi,Gencai Chen

DOI: https://doi.org/10.1016/j.ipm.2019.102180

IF: 7.466

2019-01-01

Information Processing & Management

Abstract:Search task success rate is an important indicator to measure the performance of search engines. In contrast to most of the previous approaches that rely on labeled search tasks provided by users or third-party editors, this paper attempts to improve the performance of search task success evaluation by exploiting unlabeled search tasks that are existing in search logs as well as a small amount of labeled ones. Concretely, the Multi-view Active Semi-Supervised Search task Success Evaluation (MA4SE) approach is proposed, which exploits labeled data and unlabeled data by integrating the advantages of both semi-supervised learning and active learning with the multi view mechanism. In the semi-supervised learning part of MA4SE, we employ a multi-view semi supervised learning approach that utilizes different parameter configurations to achieve the disagreement between base classifiers. The base classifiers are trained separately from the pre-defined action and time views. In the active learning part of MA4SE, each classifier received from semi-supervised learning is applied to unlabeled search tasks, and the search tasks that need to be manually annotated are selected based on both the degree of disagreement between base classifiers and a regional density measurement. We evaluate the proposed approach on open datasets with two different definitions of search tasks success. The experimental results show that MA4SE outperforms the state-of-the-art semi-supervised search task success evaluation approach.
PKU-ICST at TRECVID2009: High Level Feature Extraction and Search

Yuxin Peng,Zhiguo Yang,Lei Cao,Jian Yi,Ning Wan,Yuan Feng,Xiaohua Zhai,En Shi,Hao Li

2009-01-01

Abstract:We participate in two tasks of TRECVID 2009: high-level feature extraction (HLFE) and search. This paper presents our approaches and results in the two tasks. In HLFE task, we mainly focus on exploring the effective feature representation, data imbalance learning and fusion between different data sets. In feature representation, we adopt five basic visual features and six keypoint-based BoW features, and combine them to represent each keyframe image. In imbalance learning, we propose two methods for this problem: OnUm and concept category. In the fusion between different data sets, we use three different training sets: (1) TRECVID 2009 training data set (Tv09), (2) TRECVID 2005 training data set (Tv05), and (3) Flickr images. In search task, we participate in two types of search tasks: automatic search and manual search. We explore multimodal feature representation, which includes visual-based features, concept-based feature, audio features and face features. Based on these features, two retrieval methods are jointly adopted for search task: pair-wise similarity measure and learning-based ranking. We achieve the good results in both tasks. In HLFE task, official evaluation shows that our team ranks 2 nd in type A and 1 st in types C, a and c. In Search task, official evaluations show that our team rank 2 nd in automatic search and 1 st in manual search.
At the web track of TREC 2003

Ji-Rong Wen,Ruihua Song,Deng Cai,Kaihua Zhu,Shipeng Yu,Shaozhi Ye,Wei-Ying Ma

2004-01-01

Abstract:This is the first year that our group participates in the Web track of the TREC conference. Here we report our system and methods on the topic distillation task and the home/named page finding task.All of our experiments are conducted on a Web search platform we designed and developed from scratch. We originally want to use an existing retrieval system such as Okapi or the full text search mechanism of SQL Server. But we soon find the limitations of such a strategy–these systems cannot fully support some important Web search functions such as link analysis and anchor text, and they also lack of the flexibility to arbitrarily adjust some parameters or add new ranking functions. So we decided to design and implement a research platform to let researchers to test various algorithms or new ideas easily and, also, to conduct the TREC experiments easily. We will introduce the framework of this system in the “Platform” section.
Fudan University at TRECVID 2007

Xiangyang Xue,Hui Yu,Hong Lu,Youguang Guo,Yuejie Zhang,Shile Zhang,Bin Li,Bolan Su,Yingbin Zheng,Wenjian Zhou,Lei Cen,Jie Zhang,Yu Jiang,Jiahui Qi,Jiaojiao Lu,Qian Diao,Zhenzhen Shi,Zichen Sun

2007-01-01

Abstract:In this notebook paper we describe our participation in the NIST TRECVID 2007 evaluation. We took part in two tasks of benchmark this year including high-level feature extraction and interactive search. For high-level feature extraction, we submitted 4 runs. FD_SVM: using SVM. FD_SVM_BN: using SVM and ontology. FD_MTL: using multi-task learning. FD_MTL_BN: using multi-task learning and ontology. Evaluation results illustrate that there are both advantages and disadvantages exist in all methods. For search, we submitted 5 interactive runs. Fudan_P: using multi-model and AP-based fusion. Fudan_R: using multi-model and MGR fusion. Fudan_C: cross system retrieval. Fudan_T: textual retrieval Fudan_I: image retrieval Evaluation results illustrate that the AP-based fusion method yields higher precision while the MGR fusion method finds more positive shots than other runs. We also experimented with simple cross system interactive retrieval to estimate the impact of manual browsing on the results.
Role Centralized Modeling for Expert Search in Enterprise Corporation

Jing Yao,Jun Xu,Cheng Jin,Junyu Niu

DOI: https://doi.org/10.1109/ALPIT.2007.26

2007-01-01

Abstract:Automatic expert finding systems aiming at identifying experts from a large set of document repository have attracted considerable interest in recent years. To better describe the relationship on expert-document and document-topic, we introduce the conception "Role" in our model and expand its use in expert search. We illustrate how the conception "Role" in our model helps to present an effective approach to model the probability of the candidate being an expert on the given topic with documents relevant to it. Consequently, we evaluate the effectiveness of our model by a series of experiments on TREC 2006 Enterprise Track expert search task. The results show that our role centralized model performs much better than general approaches overall. It is a big potential to dig in more deeply in the future.
Aliababa DAMO Academy at TREC Precision Medicine 2020: State-of-the-art Evidence Retriever for Precision Medicine with Expert-in-the-loop Active Learning

Qiao Jin,Chuanqi Tan,Mosha Chen,Ming Yan,Songfang Huang,Ningyu Zhang,Xiaozhong Liu

2020-01-01

Abstract:This paper describes the submissions of Alibaba DAMO Academy to the TREC Precision Medicine (PM) Track in 2020, which achieve state-of-the-art performance in the evidence quality assessment. The focus of the TREC PM Track is to retrieve academic papers that report critical clinical evidence for or against a given treatment in a population specified by its disease and gene mutation. We use a two-step approach that includes: 1) a baseline retriever using query expansion with Elasticsearch (ES) and 2) an automatic or expert-in-the-loop reranker: the automatic re-ranker uses features of the ES scores, pre-trained BioBERT scores, publication type scores and citation count scores; the expert-in-the-loop re-ranker uses expert annotations, fine-tuned BioBERT as well as features used in the automatic re-ranker. For the expert-in-the-loop re-ranker, we use a novel active learning annotation strategy that is sample-efficient: at each iteration of the annotation, 1) we fine-tune the BioBERT using all expert annotations of query-document relevance; 2) we let human experts annotate the actual relevance of the most relevant unannotated query-document pairs predicted by the fine-tuned BioBERT. Our submissions outperform the median topic-wise scores in the phase 1 assessment for general relevance and achieve state-of-the-art performance in the phase 2 assessment for evidence quality. Our analyses show that evidence quality is a distinct aspect than the general relevance, and thus additional modeling of it is necessary to assist IR for Evidence-based Precision Medicine
PKU-ICST at TRECVID 2012: Known-item Search Task

Yuxin Peng,Yunbo Peng,Xiaohua Zhai,Jian Zhang,Tianjun Xiao,Xin Huang,Kang Cai

2012-01-01

Abstract:We participate in all two types of known-item search task of TRECVID 2012: automatic search and interactive search. This paper presents our approaches and results. We adopt three kinds of text information, which are XML documents, ASR and OCR. And we index and search the three kinds of pre-processed text individually with Lucene. In addition, the results are combined and re-ranked by two re-ranking approaches. We achieve the good performances, and official evaluation shows that our team is ranked 1 st in both automatic search and interactive search.
PKU-ICST @TRECVID2012: Known-item Search Task.

Yuxin Peng,Yunbo Peng,Xiaohua Zhai,Jian Zhang,Tianjun Xiao,Xin Huang,Kang Cai

2012-01-01

Abstract:We participate in all two types of known-item search task of TRECVID 2012: automatic search and interactive search. This paper presents our approaches and results. We adopt three kinds of text information, which are XML documents, ASR and OCR. And we index and search the three kinds of pre-processed text individually with Lucene. In addition, the results are combined and re-ranked by two re-ranking approaches. We achieve the good performances, and official evaluation shows that our team is ranked 1 in both automatic search and interactive search.

Research on Expert Search at Enterprise Track of TREC 2005.

THUIR at TREC 2007: Enterprise Track

THUIR at TREC2007: Enterprise track 1

THUIR at TREC2008: Enterprise track

CNDS Expert Finding System for TREC 2005.

A Two-Stage Model for Expert Search

THUIR at TREC 2005: Enterprise Track.

TREC-10 Web Track Experiments at MSRA.

MSRA Att TRECVID 2008: High-Level Feature Extraction and Automatic Search.

WIM at TREC 2005.

TREC 2006 at Maryland: Blog, Enterprise, Legal and QA Tracks

Peking University at the TREC 2006 Terabyte Track

MSRA-USTC-SJTU at TRECVID 2007: High-Level Feature Extraction and Search.

Search Task Success Evaluation by Exploiting Multi-View Active Semi-Supervised Learning.

PKU-ICST at TRECVID2009: High Level Feature Extraction and Search

At the web track of TREC 2003

Fudan University at TRECVID 2007

Role Centralized Modeling for Expert Search in Enterprise Corporation

Aliababa DAMO Academy at TREC Precision Medicine 2020: State-of-the-art Evidence Retriever for Precision Medicine with Expert-in-the-loop Active Learning

PKU-ICST at TRECVID 2012: Known-item Search Task

PKU-ICST @TRECVID2012: Known-item Search Task.