Abstract:Multiple query criteria active learning (MQCAL) methods have a higher potential performance than conventional active learning methods in which only one criterion is deployed for sample selection. A central issue related to MQCAL methods concerns the development of an integration criteria strategy (ICS) that makes full use of all criteria. The conventional ICS adopted in relevant research all facilitate the desired effects, but several limitations still must be addressed. For instance, some of the strategies are not sufficiently scalable during the design process, and the number and type of criteria involved are dictated. Thus, it is challenging for the user to integrate other criteria into the original process unless modifications are made to the algorithm. Other strategies are too dependent on empirical parameters, which can only be acquired by experience or cross-validation and thus lack generality; additionally, these strategies are counter to the intention of active learning, as samples need to be labeled in the validation set before the active learning process can begin. To address these limitations, we propose a novel MQCAL method for classification tasks that employs a third strategy via weighted rank aggregation. The proposed method serves as a heuristic means to select high-value samples of high scalability and generality and is implemented through a three-step process: (1) the transformation of the sample selection to sample ranking and scoring, (2) the computation of the self-adaptive weights of each criterion, and (3) the weighted aggregation of each sample rank list. Ultimately, the sample at the top of the aggregated ranking list is the most comprehensively valuable and must be labeled. Several experiments generating 257 wins, 194 ties and 49 losses against other state-of-the-art MQCALs are conducted to verify that the proposed method can achieve superior results.

Famcdm: A Fusion Approach of Mcdm Methods to Rank Multiclass Classification Algorithms

Evaluation of Classification Algorithms Using MCDM and Rank Correlation.

User Preferences Based Software Defect Detection Algorithms Selection Using MCDM.

MFA-DMFS:new Multi-Classifier Fusion Method and Its Application

Evaluation of Feature Selection Methods for Text Classification with Small Datasets Using Multiple Criteria Decision-Making Methods

Multicriteria Decision Making Approach for Cluster Validation

A multicriteria decision making approach for estimating the number of clusters in a data set.

Health diagnostics using multi-attribute classification fusion.

Intuitionistic Fuzzy Reasoning for Multiple Two-Class Classifiers Fusion

Multiple Classifiers Fusion Based On Weighted Evidence Combination

A novel active learning framework for classification: using weighted rank aggregation to achieve multiple query criteria

Evaluation of clustering algorithms for financial risk analysis using MCDM methods

Trust and behavior analysis-based fusion method for heterogeneous multiple attribute group decision-making

Using Optimization-Based Classification Method for Massive Datasets.

Multiple Attribute Decision Making Based on Cross-Evaluation with Uncertain Decision Parameters

A MCDM-Based Evaluation Approach for Imbalanced Classification Methods in Financial Risk Prediction

An Adaptive Feature Selection Method for Multi-Class Classification.

Adaptive Fusion Based Method for Imbalanced Data Classification

A Novel Multi-class Classification Architecture Combining Population-based Sampling and Multi-expert Classifier for Imbalanced Data.

A new integrated MCDM approach for improving QFD based on DEMATEL and extended MULTIMOORA under uncertainty environment

A Study of Decision Process in MCDM Problems with Large Number of Criteria