Abstract:Ensembles, which employ a set of classifiers to enhance classification accuracy collectively, are crucial in the era of big data. However, although there is general agreement that the relation between ensemble size and its prediction accuracy, the exact nature of this relationship is still unknown. We introduce a novel perspective, rooted in the linear independence of classifier's votes, to analyze the interplay between ensemble size and prediction accuracy. This framework reveals a theoretical link, consequently proposing an ensemble size based on this relationship. Our study builds upon a geometric framework and develops a series of theorems. These theorems clarify the role of linear dependency in crafting ensembles. We present a method to determine the minimum ensemble size required to ensure a target probability of linearly independent votes among component classifiers. Incorporating real and synthetic datasets, our empirical results demonstrate a trend: increasing the number of classifiers enhances accuracy, as predicted by our theoretical insights. However, we also identify a point of diminishing returns, beyond which additional classifiers provide diminishing improvements in accuracy. Surprisingly, the calculated ideal ensemble size deviates from empirical results for certain datasets, emphasizing the influence of other factors. This study opens avenues for deeper investigations into the complex dynamics governing ensemble design and offers guidance for constructing efficient and effective ensembles in practical scenarios.

Classifiers Combination with Observational Learning

Classifier Combination Based on Active Learning

Classifier Ensemble Selection for Language Verification System

Multiple Classifiers Combination Based on Specialists' FIelds

Learning Classifier System Ensemble for Data Mining

Meta-learning Based Classifier Ensemble Strategy and Its Application

Combining Classifiers and Learning Mixture-of-Experts

A Classifier Ensemble Model and Its Applications

Voting Combinations-Based Ensemble: A Hybrid Approach

Ensemble Classification Based On Random Linear Base Classifiers

A novel ensemble classifier based on multiple diverse classification methods

An Attribute-Oriented Ensemble Classifier Based on Niche Gene Expression Programming

Evolutionary Classifier Ensembles For Semi-Supervised Learning

Ensemble learning: A survey

Ensemble learning based on multi-task class labels

Ensemble OS-ELM Based on Combination Weight for Data Stream Classification

Improving imbalance classification via ensemble learning based on two-stage learning

Classifier Selection and Ensemble Model for Multi-class Imbalance Learning in Education Grants Prediction

A Two-stage Heterogeneous Ensemble Learning Algorithm Based on MLP-SVR-LGBM

Corrective Classification: Classifier Ensembling with Corrective and Diverse Base Learners

Leveraging Linear Independence of Component Classifiers: Optimizing Size and Prediction Accuracy for Online Ensembles