THUIR@ NTCIR-‐12 IMine

Zeyang Liu,Ye Chen,Rongjie Cai,Jiaxin Mao,Chao Wang,Cheng Luo,Xin Li,Yiqun Liu,Min Zhang,Huanbo Luan,Shaoping Ma

2016-01-01

Abstract:• Background: Candidates from different resources are duplicated• Goal: Find diversified candidates• Cluster candidates with K-‐means algorithm• Query vector representation

What problem does this paper attempt to address?

THUIR at NTCIR-12 IMine Task.

Zeyang Liu,Ye Chen,Rongjie Cai,Jiaxin Mao,Chao Wang,Cheng Luo,Xin Li,Yiqun Liu,Min Zhang,Huan-Bo Luan,Shaoping Ma

2016-01-01

Abstract:In this paper, we describe our approaches in the NTCIR-12 IMine task, including Chinese Query Understanding and Chinese Vertical Incorporating. In Query Understanding subtask, we propose diﬀerent strategies to mine subtopic candidates from a wide range of resources and present a two-step method to predict the vertical intent for each subtopic. In Vertical Incorporating subtask, we adopt a probabilistic algorithm to rerank the result lists of Web documents and incorporate virtual verticals into the result lists based on the intent of subtopic behind the query.
THUIR at NTCIR-13 WWW Task.

Yukun Zheng,Cheng Luo,Weixuan Wu,Jia Chen,Yiqun Liu,Huanbo Luan,Min Zhang,Shaoping Ma

2017-01-01

Abstract:This paper describes our approaches and results in NTCIR13 WWW task. In English subtask, we adopt several advanced deep models, like DSSM and DRMM. In Chinese subtask, we additionally make a few changes in models to ensure them work well in the Chinese context and train the Duet model with the weak-supervised relevance labels generated by various click models. Meanwhile, we extract 3 types of features from corpus to train a learning to rank model.
THUIR at TREC 2004: Genomics Track.

Jiao Li,Xian Zhang,Min Zhang,Xiaoyan Zhu

DOI: https://doi.org/10.6028/nist.sp.500-261.genomics-tsinghua.ma

2004-01-01

Abstract:This is the first time that THUIR participates in TREC Genomics Track. We took part in both Ad hoc retrieval task and Categorization task. Based on our retrieval system TMiner, our research in the Ad hoc retrieval task focuses on: (1) Category of organism retrieval strategy; (2) Primary Feature Model; (3) Query Expansion (QE) technology; (4) Result fusion method. Five official runs have been submitted at triage task in the Categorization task. Unigrams are used as features in Vector Space Model, and the high dimension feature vectors are trained and classified by SVM classifier with RBFs as the kernel function. Three ways are taken to improve the classifier: (1) Perform feature selection to reduce the dimension of feature vectors; (2) Weight the important features; (3) Balance between the positive dataset and the negative dataset.
THUIR at NTCIR-9 INTENT Task.

Yufei Xue,Fei Chen,Tong Zhu,Chao Wang,Zhichao Li,Yiqun Liu,Min Zhang,Yijiang Jin,Shaoping Ma

2011-01-01

Abstract:This is the first year IR group of Tsinghua University (THUIR) participates in NTCIR. We register the INTENT task and focus on the Chinese topics of subtopic mining and document ranking subtask. In our experiments, we try to mine subtopics from different resources, namely query recommendation, Wikipedia and the query-URL bipartite graph which is constructed by clickthrough data. We also develop some methods to re-rank the subtopics and remove reduplicate ones with query log and search result snippets in search engines. In the document ranking task, methods applied to diversify English documents are used to validate their effectiveness on Chinese pages, such as HITS, Novelty-Result Selection and Documents Duplication Elimination. Based on the new metric, called D#-nDCG, we propose a DocumentDiversification algorithm to select the documents retrieved for subtopics mined in the subtopic mining task, and user browse logs are also leveraged to re-rank these selected results.
THUIR at TREC 2009 Web Track: Finding Relevant and Diverse Results for Large Scale Web Search

F Chen,Tong Zhu,Bin Zhou,Maomao Zhang

2009-01-01

Abstract:This is the 8th year that IR group of Tsinghua University (THUIR) participates in TREC. This year we focus on Web track, which contains two tasks, namely ad hoc and diversity. On ad hoc task, we improved the efficiency of our distributed retrieval system TMiner to handle terabytes of Web data. Then three studies have been done, namely page quality estimation, ranking feature analysis, and model comparison. On task, we proposed several new approaches on strategy, intention detection, and duplication elimination. To mine users intention, we proposed and compared two different strategies, namely + content-based diversity which is a kind of result clustering, and user based diverse intention prediction + searching which is in the branch of query expansion.
THUSAM at NTCIR-11 IMine Task

Cheng Luo,Xin Li,Alisher Khodzhaev,Fei Chen,Keyang Xu,Yujie Cao,Yiqun Liu,Min Zhang,Shaoping Ma

2014-01-01

Abstract:This paper describes our approaches and results in NTCIR11 IMine task. In 2014, we participate in subtasks for Chinese/English Subtopic Mining and Chinese Document Ranking. In Subtopic Mining subtask, we mine subtopic candidates from various resources and construct the subtopic hierarchy with several dierent strategies. In Document Ranking subtask, we rerank the result lists with HITS algorithm and then adopt a pruning exhaustive search algorithm to generate diversied result lists.
THUIR at NTCIR-10 INTENT-2 Task

Yufei Xue,Fei Chen,Aymeric Damien,Cheng Luo,Xin Li,Shuai Huo,Min Zhang,Yiqun Liu,Shaoping Ma

2013-01-01

Abstract:This paper describes our approaches and results in NTCIR10 INTENT-2 task. In this year, we participate in subtasks for both the Chinese and English topics. We extract subtopics from multiple resources for these topics, and several subtopic clustering and re-ranking methods are proposed in this work. In Document Ranking subtask, we redefine the novelty of a document and use the new definition to re-rank the retrieved documents. Based on the existing diversification methods, we also try to selectively diversify the search results for the given queries, according to the query types determined by our strategies.
THUIR at the NTCIR-14 WWW-2 Task.

Yukun Zheng,Zhumin Chu,Xiangsheng Li,Jiaxin Mao,Yiqun Liu,Min Zhang,Shaoping Ma

DOI: https://doi.org/10.1007/978-3-030-36805-0_13

2019-01-01

Abstract:The THUIR team participated in both Chinese and English subtasks of the NTCIR-14 We Want Web-2 (WWW-2) task. This paper describes our approaches and results in the WWW-2 task. In the Chinese subtask, we designed and trained two neural ranking models on the Sogou-QCL dataset. In the English subtask, we adopted learning to rank models by training them on MQ2007 and MQ2008 datasets. Our methods achieved the best performances in both Chinese and English subtasks. Through further analysis of results, we find that our neural models can achieve better performances in all navigational, informational and transactional queries in Chinese subtask. In the English subtask, the learning-to-rank methods have stronger modeling capabilities than BM25 by learning from effective hand-crafted features.
THUIR at TREC 2003: Novelty, Robust and Web.

Min Zhang,Chuan Lin,Yiqun Liu,Leo Zhao,Shaoping Ma

DOI: https://doi.org/10.6028/nist.sp.500-255.robust-tsinghuau.ma

2003-01-01

Abstract:This is the second time that Tsinghua University Information Retrieval Group (THUIR) participates in TREC. In this year, we took part in four tracks: novelty, robust, web and HARD, describing in following sections, respectively. A new IR system named TMiner has been built on which all experiments have been performed. In the system, Primary Feature Model (PFM) has been proposed and combined with BM2500 term weighting , which led to encouraging results. Word-pair searching has also been performed and improves system precision. Both approaches are described in robust experiments (section 2), and they were also used in web track experiments.
THUIR2 at NTCIR-16 Session Search (SS) Task

Weihang Su,Xiangsheng Li,Yiqun Liu,Min Zhang,Shaoping Ma

2023-07-01

Abstract:Our team(THUIR2) participated in both FOSS and POSS subtasks of the NTCIR-161 Session Search (SS) Task. This paper describes our approaches and results. In the FOSS subtask, we submit five runs using learning-to-rank and fine-tuned pre-trained language models. We fine-tuned the pre-trained language model with ad-hoc data and session information and assembled them by a learning-to-rank method. The assembled model achieves the best performance among all participants in the preliminary evaluation. In the POSS subtask, we used an assembled model which also achieves the best performance in the preliminary evaluation.

Information Retrieval,Artificial Intelligence
Subspace Clustering by Directly Solving Discriminative K-means

Chenhui Gao,Wenzhi Chen,Feiping Nie,Weizhong Yu,Feihu Yan

DOI: https://doi.org/10.1016/j.knosys.2022.109452

IF: 8.139

2022-01-01

Knowledge-Based Systems

Abstract:Applications in many domains such as text mining and natural language processing need to deal with high-dimensional data. High-dimensional data may present better clustering characteristics on a selected low-dimensional subspace. Subspace clustering is to project the data onto a low-dimensional subspace before clustering. Traditional subspace clustering methods employ eigenvalue decomposition to find the projection of the input data and perform K-means or kernel K-means to obtain the clustering matrix. This kind of methods is not only inefficient, but also adopts a two-step method to generate an approximate solution. Although Discriminative K-means (DisKmeans) integrates dimensionality reduction and clustering into a joint framework and solves the optimization problem by kernel K-means, such method needs to find the centroids in the kernel space and class labels iteratively and has a square time complexity. Accordingly, in this paper, we propose an algorithm, namely Fast DisKmeans (FDKM), to obtain the cluster indicator matrix in a direct way. Moreover, our proposed method has a linear time complexity, which is a significant reduction compared with the squared time complexity of DisKmeans. We also demonstrate that solving the object function of DisKmeans is equivalent to representing the cluster assignment matrix by a low-dimensional linear mapping of the data. Based on this observation, we propose the second algorithm, namely Iterative Fast DisKmeans (IFDKM), which also has a linear time complexity. A series of experiments were conducted on several datasets, and the experimental results showed the superior performance of FDKM and IFDKM.
Improved Feature Selection and Redundance Computing - THUIR at TREC 2004 Novelty Track.

Liyun Ru,Le Zhao,Min Zhang,Shaoping Ma

2004-01-01

Abstract:This is the third years that Tsinghua University Information Retrieval Group (THUIR) participates in Novelty task of TREC. Our research on this year’s novelty track mainly focused on four aspects: (1) text feature selection and reduction; (2) improved sentence classification in finding relevant information; (3)efficient sentence redundancy computing; (4) effective result filtering. All experiments have been performed on TMiner IR system, developed by THU IR group last year.
THUIR at TREC 2003: HARD Experiments.

Liang Ma,Wei Tan,Qunxiu Chen,Shaoping Ma,Shuicai Shi,Shibin Xiao,Hongwei Wang,Hongjun Wang

2003-01-01

Abstract:In this paper, we describe ideas and related experiments of Tsinghua University IR group in TREC-12 HARD Track. In this track, we focus on an automatic delivering mechanism, which combine the existing IR methods and can provide a quick retrieval solution for the practical environment. The final official evaluation show the old ways perform not well, but we think the experiment data will be helpful in evaluating the new ideas developed by other teams.
THUIR at TREC 2005: Enterprise Track.

Yupeng Fu,Wei Yu,Yize Li,Yiqun Liu,Min Zhang,Shaoping Ma

2008-01-01

Abstract:IR group of Tsinghua University participated in the expert finding task of TREC2005 enterprise track this year. We developed a novel method which is called document reorganization to solve the problem of locating expert for certain query topics. This method collects and combines related information from different media formats to organize a document which describes an expert candidate. This method proves both effective and efficient for expert finding task. Our submitted run (THUENT0505) obtains the best performance in all participants with evaluation metric MAP. The reorganized documents are also significantly smaller in size than the original corpus.
Multiple Kernel k -Means Clustering by Selecting Representative Kernels

Yaqiang Yao,Yang Li,Bingbing Jiang,Huanhuan Chen

DOI: https://doi.org/10.1109/tnnls.2020.3026532

IF: 14.255

2021-11-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:To cluster data that are not linearly separable in the original feature space, <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.211ex" height="2.176ex" style="vertical-align: -0.338ex;" viewBox="0 -791.3 521.5 936.9" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-6B" x="0" y="0"></use></g></svg></span> -means clustering was extended to the kernel version. However, the performance of kernel <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.211ex" height="2.176ex" style="vertical-align: -0.338ex;" viewBox="0 -791.3 521.5 936.9" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-6B" x="0" y="0"></use></g></svg></span> -means clustering largely depends on the choice of the kernel function. To mitigate this problem, multiple kernel learning has been introduced into the <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.211ex" height="2.176ex" style="vertical-align: -0.338ex;" viewBox="0 -791.3 521.5 936.9" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-6B" x="0" y="0"></use></g></svg></span> -means clustering to obtain an optimal kernel combination for clustering. Despite the success of multiple kernel <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.211ex" height="2.176ex" style="vertical-align: -0.338ex;" viewBox="0 -791.3 521.5 936.9" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-6B" x="0" y="0"></use></g></svg></span> -means clustering in various scenarios, few of the existing work update the combination coefficients based on the diversity of kernels, which leads to the result that the selected kernels contain high redundancy and would degrade the clustering performance and efficiency. We resolve this problem from the perspective of subset selection in this article. In particular, we first propose an effective strategy to select a diverse subset from the prespecified kernels as the representative kernels, and then incorporate the subset selection process into the framework of multiple <span class="mjpage"><svg xmlns:xlink="http://www.w3.org/1999/xlink" width="1.211ex" height="2.176ex" style="vertical-align: -0.338ex;" viewBox="0 -791.3 521.5 936.9" role="img" focusable="false" xmlns="http://www.w3.org/2000/svg"><g stroke="currentColor" fill="currentColor" stroke-width="0" transform="matrix(1 0 0 -1 0 0)"> <use xlink:href="#MJMATHI-6B" x="0" y="0"></use></g></svg></span> -means clustering. The representative kernels can be indicated as a significant combination weights. Due to the nonconvexity of the obtained objective function, we develop an alternating minimization method to optimize the combination coefficients of the selected kernels and the cluster membership alternatively. In particular, an efficient optimization method is developed to reduce the time complexity of optimizing the kernel combination weights. Finally, extensive experiments on benchmark and real-world data sets demonstrate the effectiveness and superiority of our approach in comparison with existin- methods.<svg xmlns="http://www.w3.org/2000/svg" style="display: none;"><defs id="MathJax_SVG_glyphs"><path stroke-width="1" id="MJMATHI-6B" d="M121 647Q121 657 125 670T137 683Q138 683 209 688T282 694Q294 694 294 686Q294 679 244 477Q194 279 194 272Q213 282 223 291Q247 309 292 354T362 415Q402 442 438 442Q468 442 485 423T503 369Q503 344 496 327T477 302T456 291T438 288Q418 288 406 299T394 328Q394 353 410 369T442 390L458 393Q446 405 434 405H430Q398 402 367 380T294 316T228 255Q230 254 243 252T267 246T293 238T320 224T342 206T359 180T365 147Q365 130 360 106T354 66Q354 26 381 26Q429 26 459 145Q461 153 479 153H483Q499 153 499 144Q499 139 496 130Q455 -11 378 -11Q333 -11 305 15T277 90Q277 108 280 121T283 145Q283 167 269 183T234 206T200 217T182 220H180Q168 178 159 139T145 81T136 44T129 20T122 7T111 -2Q98 -11 83 -11Q66 -11 57 -1T48 16Q48 26 85 176T158 471L195 616Q196 629 188 632T149 637H144Q134 637 131 637T124 640T121 647Z"></path></defs></svg>

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
THUIR at TREC 2008: Relevance Feedback Track

Bo Zhou,Qi Fang,Rongwei Cen,Min Zhang,Yiqun Liu,Shaoping Ma

DOI: https://doi.org/10.6028/nist.sp.500-277.relfdbk-thuir

2008-01-01

Abstract:Tsinghua University Information Retrieval Group (THUIR) has participated into the first Relevance Feedback Track of TREC2008. The TMiner search engine has been used as our text retrieval system, because the processing capability and flexibility of this system on large text data has been testified during many years’ Web Track and Terabyte Track. In the track, we studied two approaches: 1) query expansion, 2) search result re-ranking based on document relevance model. Query Expansion: Terms in the annotated documents (feedback) are used to expand the original query; the new born queries are sent to the search engine for further information retrieval; users get the documents retrieved by the expanded queries. Search Result Re-ranking: The relevance between the annotated documents and other documents are used to influence the search results; users finally get the re-ranked document list. In detail, we have experimented two different methods on which search result re-ranking based: a) Clustering; b) Documents Relevance Model. The rest of this paper is structured as follows. After the introduction of Query Expansion approach in Section 2, Search Result Re-ranking is discussed in Section 3. The evaluation results of the submitted runs are illustrated in Section 4. The last section contains summaries and outlines promising future work.
Overview of the NTCIR-12 IMine-2 Task

Takehiro Yamamoto,Yiqun Liu,Min Zhang,Zhicheng Dou,Ke Zhou,Ilya Markov,Makoto P. Kato,Hiroaki Ohshima,Sumio Fujita

2016-01-01

Abstract:In this paper, we provide an overview of the NTCIR-12 IMine-2 task, which is a core task of NTCIR-12 and also a succeeding work of IMine@NTCIR-11, INTENT-2@NTCIR-10, and INTENT@NTCIR-9 tasks. IMine-2 comprises the Query Understanding subtask and the Vertical Incorporating subtask. 23 groups from diverse countries including China, France, India, Portugal, Ireland, and Japan registered to the task. Finally, IMine2 attracted 9 participating teams; we received 42 runs for the Query Understanding subtask and 15 runs for the Vertical Incorporating subtask. We describe the subtasks, data, evaluation methods, and report the official results for each subtask.
Tensorized Incomplete Multi-view Kernel Subspace Clustering

Guang-Yu Zhang,Dong Huang,Chang-Dong Wang

DOI: https://doi.org/10.1016/j.neunet.2024.106529

Abstract:Recently considerable advances have been achieved in the incomplete multi-view clustering (IMC) research. However, the current IMC works are often faced with three challenging issues. First, they mostly lack the ability to recover the nonlinear subspace structures in the multiple kernel spaces. Second, they usually neglect the high-order relationship in multiple representations. Third, they often have two or even more hyper-parameters and may not be practical for some real-world applications. To tackle these issues, we present a Tensorized Incomplete Multi-view Kernel Subspace Clustering (TIMKSC) approach. Specifically, by incorporating the kernel learning technique into an incomplete subspace clustering framework, our approach can robustly explore the latent subspace structure hidden in multiple views. Furthermore, we impute the incomplete kernel matrices and learn the low-rank tensor representations in a mutual enhancement manner. Notably, our approach can discover the underlying relationship among the observed and missing samples while capturing the high-order correlation to assist subspace clustering. To solve the proposed optimization model, we design a three-step algorithm to efficiently minimize the unified objective function, which only involves one hyper-parameter that requires tuning. Experiments on various benchmark datasets demonstrate the superiority of our approach. The source code and datasets are available at: https://www.researchgate.net/publication/381828300_TIMKSC_20240629.
Similarity-Based Three-Way Clustering by Using Dimensionality Reduction

Anlong Li,Yiping Meng,Pingxin Wang

DOI: https://doi.org/10.3390/math12131951

IF: 2.4

2024-06-24

Mathematics

Abstract:Three-way clustering uses core region and fringe region to describe a cluster, which divide the dataset into three parts. The division helps identify the central core and outer sparse regions of a cluster. One of the main challenges in three-way clustering is the meaningful construction of the two sets. Aimed at handling high-dimensional data and improving the stability of clustering, this paper proposes a novel three-way clustering method. The proposed method uses dimensionality reduction techniques to reduce data dimensions and eliminate noise. Based on the reduced dataset, random sampling and feature extraction are performed multiple times to introduce randomness and diversity, enhancing the algorithm's robustness. Ensemble strategies are applied on these subsets, and the k-means algorithm is utilized to obtain multiple clustering results. Based on these results, we obtain co-association frequency between different samples and fused clustering result using the single-linkage method of hierarchical clustering. In order to describe the core region and fringe region of each cluster, the similar class of each sample is defined by co-association frequency. The lower and upper approximations of each cluster are obtained based on similar class. The samples in the lower approximation of each cluster belong to the core region of the cluster. The differences between lower and upper approximations of each cluster are defined as fringe region. Therefore, a three-way explanation of each cluster is naturally formed. By employing various UC Irvine Machine Learning Repository (UCI) datasets and comparing different clustering metrics such as Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and Accuracy (ACC), the experimental results show that the proposed strategy is effective in improving the structure of clustering results.

mathematics
Finding "abstract Fields" of Web Pages and Query Specific Retrieval - THUIR at TREC 2004 Web Track

Yiqun Liu,Canhui Wang,Min Zhang,Shaoping Ma

DOI: https://doi.org/10.6028/nist.sp.500-261.web-tsinghua.ma

2004-01-01

Abstract:In this year's TREC Web Track research, THUIR participated in the Mixed-query Task. This task involves a single query set comprising 3 kinds of queries (Homepage Finding, Named Page Finding and Topic distillation) which are mixed and unlabelled. Efforts have been made on two directions: to find a strong and robust unified approach which works well for all kinds of queries, and to build a query-specific retrieval strategy that classifies queries by types and perform specific approaches. The using of non-content information has been studied in both approaches. With topic distillation and navigational search tasks in the last year, we are able to build a training set with 150 topics and corresponding relevant qrels. This training set is used to evaluate effectiveness of different methods in mixed query search. Experiments in section 2, 3 and 4 are all based on this set.

THUIR@ NTCIR-‐12 IMine

THUIR at NTCIR-12 IMine Task.

THUIR at NTCIR-13 WWW Task.

THUIR at TREC 2004: Genomics Track.

THUIR at NTCIR-9 INTENT Task.

THUIR at TREC 2009 Web Track: Finding Relevant and Diverse Results for Large Scale Web Search

THUSAM at NTCIR-11 IMine Task

THUIR at NTCIR-10 INTENT-2 Task

THUIR at the NTCIR-14 WWW-2 Task.

THUIR at TREC 2003: Novelty, Robust and Web.

THUIR2 at NTCIR-16 Session Search (SS) Task

Subspace Clustering by Directly Solving Discriminative K-means

Improved Feature Selection and Redundance Computing - THUIR at TREC 2004 Novelty Track.

THUIR at TREC 2003: HARD Experiments.

THUIR at TREC 2005: Enterprise Track.

Multiple Kernel k -Means Clustering by Selecting Representative Kernels

THUIR at TREC 2008: Relevance Feedback Track

Overview of the NTCIR-12 IMine-2 Task

Tensorized Incomplete Multi-view Kernel Subspace Clustering

Similarity-Based Three-Way Clustering by Using Dimensionality Reduction

Finding "abstract Fields" of Web Pages and Query Specific Retrieval - THUIR at TREC 2004 Web Track