Multi-instance clustering with applications to multi-instance prediction

ZhangMin-Ling,ZhouZhi-Hua

IF: 5.3

2009-01-01

Applied Intelligence

Abstract:In the setting of multi-instance learning, each object is represented by a bag composed of multiple instances instead of by a single instance in a traditional learning setting. Previous works in th...

What problem does this paper attempt to address?

A Multi-Task Learning Strategy for Unsupervised Clustering Via Explicitly Separating the Commonality

Shu Kong,Donghui Wang

2012-01-01

Abstract:In this paper, we propose an unsupervised cluster method via a multi-task learning strategy, called MtCluster. Our MtCluster learns a cluster-specific dictionary for each cluster to represent its sample signals and a shared common pattern pool (the commonality) for the essentially complemental representation. By treating learning the cluster-specific dictionary as a single task, MtCluster works in a multi-task learning manner, in which all the tasks are connected by simultaneously learning the commonality. Actually, the learned cluster-specific dictionary spans the feature space of the corresponding cluster, and the commonality is just used for necessary complemental representation. To evaluate our method, we perform several experiments on public available datasets, and the promising results demonstrate the effectiveness of MtCluster.
Multi-View Clustering Via Simultaneously Learning Shared Subspace And Affinity Matrix

Nan Xu,Yanqing Guo,Jiujun Wang,Xiangyang Luo,Xiangwei Kong

DOI: https://doi.org/10.1177/1729881417745677

IF: 1.714

2017-01-01

International Journal of Advanced Robotic Systems

Abstract:Due to the existence of multiple views in many real-world data sets, multi-view clustering is increasingly popular. Many approaches have been investigated, among which the subspace clustering methods finding the underlying subspaces of data have been developed recently. Although the subspace-based multi-view methods can achieve promising performance, the shared subspace information has not been fully utilized. To address this problem, a novel multi-view clustering model by simultaneously learning shared subspace and affinity matrix is proposed. In our method, a shared subspace is learned to preserve the effective consensus information of all views. Then, a subspace-based affinity matrix with adaptive neighbors is learned to assign the most suitable cluster to each data point. An iterative strategy is developed for solving this problem. Moreover, experiments on four benchmark data sets demonstrate that our algorithm outperforms other state-of-the-art algorithms.
Multi-Instance Learning: A Survey

Zhi-Hua Zhou

2004-01-01

Abstract:In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. This paper provides a survey on this topic. At first, it introduces the origin of multi-instance learning. Then, developments on the study of learnability, learning algorithms, applications and extensions of multi-instance learning are reviewed. In particular, this paper employs a unified view to look into multi-instance learning algorithms. Some important issues to be addressed are also discussed.
Solving multi-instance problems with classifier ensemble based on constructive clustering

Zhi-Hua Zhou,Min-Ling Zhang

DOI: https://doi.org/10.1007/s10115-006-0029-3

IF: 2.7

2006-01-01

Knowledge and Information Systems

Abstract:In multi-instance learning, the training set is composed of labeled bags each consists of many unlabeled instances, that is, an object is represented by a set of feature vectors instead of only one feature vector. Most current multi-instance learning algorithms work through adapting single-instance learning algorithms to the multi-instance representation, while this paper proposes a new solution which goes at an opposite way, that is, adapting the multi-instance representation to single-instance learning algorithms. In detail, the instances of all the bags are collected together and clustered into d groups first. Each bag is then re-represented by d binary features, where the value of the i th feature is set to one if the concerned bag has instances falling into the i th group and zero otherwise. Thus, each bag is represented by one feature vector so that single-instance classifiers can be used to distinguish different classes of bags. Through repeating the above process with different values of d , many classifiers can be generated and then they can be combined into an ensemble for prediction. Experiments show that the proposed method works well on standard as well as generalized multi-instance problems.
Multi-Instance Learning from Supervised View

Zhi-Hua Zhou

DOI: https://doi.org/10.1007/s11390-006-0800-7

IF: 1.871

2006-01-01

Journal of Computer Science and Technology

Abstract:In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. This paper studies multi-instance learning from the view of supervised learning. First, by analyzing some representative learning algorithms, this paper shows that multi-instance learners can be derived from supervised learners by shifting their focuses from the discrimination on the instances to the discrimination on the bags. Second, considering that ensemble learning paradigms can effectively enhance supervised learners, this paper proposes to build multi-instance ensembles to solve multi-instance problems. Experiments on a real-world benchmark test show that ensemble learning paradigms can significantly enhance multi-instance learners.
Multi-task Clustering Through Instances Transfer

Xiaotong Zhang,Xianchao Zhang,Han Liu,Xinyue Liu

DOI: https://doi.org/10.1016/j.neucom.2017.04.029

IF: 6

2017-01-01

Neurocomputing

Abstract:Clustering is an essential issue in machine learning and data mining. As there are many related tasks in the real world, multi-task clustering, which improves the clustering performance of each task by transferring knowledge across the related tasks, receives increasing attention recently. Generally knowledge transfer can be accomplished in different ways. Nevertheless, besides transferring knowledge of feature representations, other knowledge transfer ways have seldom been adopted for multi-task clustering. In this paper, we propose a general multi-task clustering algorithm by transferring knowledge of instances. Our algorithm reweights the distance between samples in different tasks by learning a shared subspace, then selects the nearest neighbors for each sample from the other tasks in the learned shared subspace as the auxiliary data to aid the clustering process of each individual task. Experiments on real data sets in text mining and image mining demonstrate that our proposed algorithm outperforms the traditional single-task clustering methods and existing cross-domain multi-task clustering methods. (C) 2017 Elsevier B.V. All rights reserved.
Multi-instance learning by treating instances as non-I.I.D. samples

Zhi-Hua Zhou,Yu-Yin Sun,Yu-Feng Li

DOI: https://doi.org/10.1145/1553374.1553534

2009-01-01

Abstract:Previous studies on multi-instance learning typically treated instances in the bags as independently and identically distributed. The instances in a bag, however, are rarely independent in real tasks, and a better performance can be expected if the instances are treated in an non-i.i.d. way that exploits relations among instances. In this paper, we propose two simple yet effective methods. In the first method, we explicitly map every bag to an undirected graph and design a graph kernel for distinguishing the positive and negative bags. In the second method, we implicitly construct graphs by deriving affinity matrices and propose an efficient graph kernel considering the clique information. The effectiveness of the proposed methods are validated by experiments.
Clustering-based Multiple Instance Learning with Multi-View Feature

Chengkun He,Jie Shao,Jiasheng Zhang,Xiangmin Zhou

DOI: https://doi.org/10.1016/j.eswa.2019.113027

IF: 8.5

2019-01-01

Expert Systems with Applications

Abstract:Multi-instance learning (MIL) is a special kind of classification problem where samples (called "in-stances") are grouped into bags and labels are given only on bag level instead of instance level. From the expert and intelligent system perspective, MIL does not require full ground-truth labels which helps to reduce the cost of data labeling in real tasks. Many inviting problems such as image classification, video annotation and object detection can be formulated in MIL frameworks. In this paper, we propose a similarity-based method with clustering in a multi-view feature manner to solve MIL problems efficiently. The clustering is introduced as a novel strategy for instance selection. Considering unlabeled data, clustering methods fit well for obtaining the hidden structure information in feature space, and thus we utilize a clustering-based strategy to exploit such information on discovering positive instances. To fully use the original input data, we further develop our method in multi-view feature space, in order to make our model capture information from different feature space and provide support for the whole system. Experiments on benchmark datasets for classification have been conducted, and we also perform image retrieval to examine the extended multi-view model on a large image dataset, MIL-NUS-WIDE. The promising results demonstrate the effectiveness of our clustering-based MIL (CMIL) model. (C) 2019 Elsevier Ltd. All rights reserved.
Ensembles Of Multi-Instance Learners

Zh Zhou,Ml Zhang

DOI: https://doi.org/10.1007/978-3-540-39857-8_44

2003-01-01

Abstract:In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. Through analyzing two famous multi-instance learning algorithms, this paper shows that many supervised learning algorithms can be adapted to multi-instance learning, as long as their focuses are shifted from the discrimination on the instances to the discrimination on the bags. Moreover, considering that ensemble learning paradigms can effectively enhance supervised learners, this paper proposes to build ensembles of multi-instance learners to solve multi-instance problems. Experiments on a real-world benchmark test show that ensemble learning paradigms can significantly enhance multi-instance learners, and the result achieved by EM-DD ensemble exceeds the best result on the benchmark test reported in literature.
A review of multi-instance learning research

Yingjie TIAN,Dongkuan XU,Chunhua ZHANG

DOI: https://doi.org/10.15960/j.cnki.issn.1007-6093.2018.02.001

2018-01-01

Abstract:Multi-instance learning is a special kind of machine learning problem,has received extensive attention and been researched on in recent years.Many different types of multi-instance learning algorithms have been proposed to deal with practical problems in various fields.This paper reviews the algorithm research and application of multiinstance learning in detail,introduces various background assumptions,and introduces multi-instance learning from three aspects:instance level,bag level,and embedded space.Finally we provide the algorithm extensions and major applications in several areas.
Solving the Multi-Instance Problem with Neural Networks

Zhi-Hua Zhou,Min-Ling Zhang

2003-01-01

Abstract:Multi-instance learning was coined by Dietterich et al. recently in their investigation on drug activity prediction. In such a learning framework, the training examples are bags composed of instances, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. A bag is positive if it contains at least one positive instance, while it is negative if it contains no positive instance. However, the labels of the instances constituting the training bags are unknown. In this paper, the open problem of designing multi- instance modification for neural networks is addressed. In detail, a neural network algorithm named Backpropagation-MIP is presented, which is derived from the popular Backpropagation algorithm through employing a new error function capturing the nature of multi-instance learning, i.e. the labels of the training bags instead of that of the training instances are known. Experiments on both the real-world and artificial benchmark multi-instance data show that the performance of Backpropagation-MIP is comparable to that of some well-established multi-instance learning methods.
Clustering-Augmented Multi-instance Learning for Neural Relation Extraction

Qi Zhang,Siliang Tang,Jinquan Sun,Yu Wang,Lei Zhang

DOI: https://doi.org/10.1007/978-3-030-72240-1_47

2021-01-01

Abstract:Despite its efficiency in generating training data, distant supervision for sentential relation extraction assigns labels to instances in a context-agnostic manner—a process that may introduce false labels and confuse sentential model learning. In this paper, we propose to integrate instance clustering with distant training, and develop a novel clustering-augmented multi-instance training framework. Specifically, for sentences labeled with the same relation type, we jointly perform clustering based on their semantic representations, and treat each cluster as a training unit for multi-instance training. Comparing to existing bag-level attention models, our proposed method does not restrict the training unit to be sentences with the same entity pair, as it may cause the selective attention to focus on instances with simple sentence context, and thus fail to provide informative supervision. Experiments on two popular datasets demonstrate the effectiveness of augmenting multi-instance learning with clustering.
Neural Networks for Multi-Instance Learning

Zhi-Hua Zhou,Min-Ling Zhang

2002-01-01

Abstract:Multi-instance learning was coined by Dietterich et al. in their investigation on drug activity prediction. In such a learning framework, the training examples are bags composed of instances, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. A bag is positive if it contains at least one positive instance, while it is negative if it contains no positive instance. However, the labels of the instances constituting the training bags are unknown. In this paper, the open problem of designing multi-instance modification for neural networks is addressed. In detail, a neural network algorithm named BP-MIP is presented, which is derived from the popular BP algorithm through employing a new error function capturing the nature of multi-instance learning, i.e. the labels of the training bags instead of that of the training instances are known. Experiments on both the real-world and artificial benchmark multi-instance data show that the performance of BP-MIP is comparable to that of some well-established multi-instance learning methods.
Multi-Side Multi-Instance Algorithm

Shu Zhao,Chao Xu,Yanping Zhang,Jun Ma

DOI: https://doi.org/10.1109/cis.2011.109

2011-01-01

Abstract:Multi-instance learning is a new machine learning framework following supervised learning, unsupervised learning and reinforcement learning. In order to solve the complex computing problems of many dimensions and large amount of samples better in multi-instance learning, a multi-instance learning algorithm based on multi-side is put forward in this paper. The algorithm selects the feature attributes of sample, and proceeds multi-instance learning to sample from different attribute sides. This is similar to the process that people acknowledge problems from different sides, also offer a simple way to solve many dimensions samples of multi-instance problem. The experimental results on benchmark data sets show that this algorithm is efficient and effective.
Web Image Clustering Based on Multi-instance

Jing Lu,ShaoPing Ma,Min Zhang

DOI: https://doi.org/10.1109/WIIAT.2008.90

2008-01-01

Abstract:In image retrieval and annotation, Multi-Instance Learning has been studied actively. Most of the methods solve the MIL problem in a supervised way. In this paper, we proposed two unsupervised frameworks for clustering multi-instance objects based on Expectation Maximization (EM) and iterative heuristic optimization respectively. For each framework, we introduced three new algorithms of finding users' interests on specific web images without any manual labeled data. And comparative studies have shown the effectiveness of the proposed algorithms.
A Multi-Instance Multi-Label Scene Classification Method Based on Multi-Kernel Fusion

Chen Tong-tong,Liu Chan-juan,Zou Hai-lin,Zhou Shu-sen,Liu Ying,Ding Xin-miao

DOI: https://doi.org/10.1109/intellisys.2015.7361229

2015-01-01

Abstract:Multi-instance multi-label learning, an extension of multi-instance learning in multi-label classification, has been successfully used in image classification. In existing algorithms, the distribution of instances in bags is generally assumed to be independent of each other, which is difficult to be guaranteed in image classification. Considering instance correlations in a bag, in this paper a novel method of scene classification based on multi-kernel fusion and multi-instance multi-label learning is proposed. First, instance correlations are introduced by means of building graph. Then, different kernel matrices can be derived from kernel functions based on graphs in different scales. Finally, the multi-label can be predicted by the multi-kernel SVM classifier based on multiple-kernel fusion. Experimental results on scene data set and MSRC v2 data set show that the proposed method greatly improves the accuracy of the image scene classification compared with other methods.
Multi-Instance Multi-Label Learning with Application to Scene Classification

Zhi-hua Zhou,Min-ling Zhang

2006-01-01

Abstract:In this paper, we formalize multi-instance multi-label learning, where each train- ing example is associated with not only multiple instances but also multiple class labels. Such a problem can occur in many real-world tasks, e.g. an image usually contains multiple patches each of which can be described by a feature vector, and the image can belong to multiple categories since its semantics can be recognized in different ways. We analyze the relationship between multi-instance multi-label learning and the learning frameworks of traditional supervised learning, multi- instance learning and multi-label learning. Then, we propose the MIMLBOOST and MIMLSVM algorithms which achieve good performance in an application to scene classification.
Multi-instance Learning in the Presence of Positive and Unlabeled Bags

Min Yuan,Yitian Xu,Jie Zhao

DOI: https://doi.org/10.1016/j.eswa.2024.125551

2025-01-01

Abstract:Multi-instance learning is a classic algorithm within the weakly supervised learning framework. Most previous approaches to multi-instance learning have typically assumed that every bag comes with its own label. However, in real-world applications, obtaining fully-labeled bags can be a challenging and resource-intensive task, primarily due to the substantial time and labor required for labeling each instance. Fortunately, collecting bags with only positive and unlabeled instances is a more feasible option. In this paper, we aim to learn an efficient binary classifier using positive and unlabeled bags. To address the absence of negative bags, we adopt a novel strategy that combines label noise learning and multi-instance kernels. Our approach involves a comprehensive analysis of empirical risk minimization in the presence of label noise, allowing us to decompose the loss function for negative bags with false negative labels into two distinct parts. Importantly, only the second part of this loss function is affected by the presence of noisy labels. To mitigate the influence of noisy labels on the negative bags, we propose the Multi-Instance Kernel-based Centroid Estimation (MIKCE) technique. MIKCE is a versatile method that can be employed with various types of data, including linear and nonlinear cases. Additionally, we provide an upper bound on the generalization error, ensuring the convergence of the MIKCE algorithm. Finally, we conduct numerical experiments on 31 public datasets to empirically validate the effectiveness of the MIKCE approach.
Adaptive Instance-wise Multi-view Clustering

Shudong Huang,Hecheng Cai,Hao Dai,Wentao Feng,Jiancheng Lv

DOI: https://doi.org/10.1145/3664647.3681335

2024-01-01

Abstract:Multi-view clustering has garnered attention for its effectiveness in addressing heterogeneous data by unsupervisedly revealing underlying correlations between different views. As a mainstream method, multi-view graph clustering has attracted increasing attention in recent years. Despite its success, it still has some limitations. Notably, many methods construct the similarity graph without considering the local geometric structure and exploit coarse-grained complementary and consensus information from different views at the view level. To solve the shortcomings, we focus on local structure consistency and fine-grained representations across multiple views. Specifically, each view's local consistency similarity graph is obtained through the adaptive neighbor. Subsequently, the multi-view similarity tensor is rotated and sliced into fine-grained instance-wise slices. Finally, these slices are fused into the final similarity matrix. Consequently, cross-view consistency can be captured by exploring the intersections of multiple views in an instance-wise manner. We design a collaborative framework with the augmented Lagrangian method to refine all subtasks towards optimal solutions iteratively. Extensive experiments on several multi-view datasets confirm the significant enhancement in clustering accuracy achieved by our method.
A New Multiple Instance Algorithm Using Structural Information.

Xiaoyan Zhu,Ting Wang,Jiayin Wang,Ying Xu,Yuqian Liu

DOI: https://doi.org/10.1109/icdm51629.2021.00204

2021-01-01

Abstract:Multiple instance learning (MIL) is semisupervised learning that predicts the label of a bag with a wide diversity of instances. It has many applications and thus attracts increasingly more attention. In this paper, we propose a new MIL algorithm using the structural information of a bag to predict its label. In the proposed method, a bag is transformed into a graph, and spectral clustering is employed to divide the graph into several subgraphs. Then, the graph Fourier transform is utilized to extract the features of the subgraphs. Finally, an end-to-end neural network is used to predict the label of a bag with the extracted features. An empirical study with 25 datasets was conducted to validate the effectiveness of the proposed method. The experimental results show that the proposed method performs better than the 6 baseline methods on most datasets.

Multi-instance clustering with applications to multi-instance prediction

A Multi-Task Learning Strategy for Unsupervised Clustering Via Explicitly Separating the Commonality

Multi-View Clustering Via Simultaneously Learning Shared Subspace And Affinity Matrix

Multi-Instance Learning: A Survey

Solving multi-instance problems with classifier ensemble based on constructive clustering

Multi-Instance Learning from Supervised View

Multi-task Clustering Through Instances Transfer

Multi-instance learning by treating instances as non-I.I.D. samples

Clustering-based Multiple Instance Learning with Multi-View Feature

Ensembles Of Multi-Instance Learners

A review of multi-instance learning research

Solving the Multi-Instance Problem with Neural Networks

Clustering-Augmented Multi-instance Learning for Neural Relation Extraction

Neural Networks for Multi-Instance Learning

Multi-Side Multi-Instance Algorithm

Web Image Clustering Based on Multi-instance

A Multi-Instance Multi-Label Scene Classification Method Based on Multi-Kernel Fusion

Multi-Instance Multi-Label Learning with Application to Scene Classification

Multi-instance Learning in the Presence of Positive and Unlabeled Bags

Adaptive Instance-wise Multi-view Clustering

A New Multiple Instance Algorithm Using Structural Information.