Abstract:Class imbalance is a critical challenge in big data analytics, often leading to biased predictive models. This imbalance can lead to biased models that perform well on the majority class but poorly on the minority class. Many machine learning models tend to be biased towards the majority class because they aim to minimise overall error, often leading to poor performance on the minority class. This paper presents the balanced ensemble neural network, a novel solution to effectively address class imbalance in big data. Balanced ensemble neural network combines the robust capabilities of neural networks with the power of ensemble learning, incorporating class balancing strategies to ensure fair representation of minority classes. The methodology involves integrating multiple neural networks, each trained on balanced subsets of data using techniques like Synthetic Minority Over‐sampling Technique and Random Undersampling. This integration aims to leverage the strengths of individual networks while reducing their inherent biases. Our extensive experiments across various datasets reveal that BENN achieves an AUC‐ROC score of 0.94, surpassing other models such as random forest (0.88), support vector (0.84) and single neural net (0.80). It was also observed that BENN's performance is better compared to traditional neural network models and standard ensemble methods in key metrics like accuracy, precision, recall, F1‐score and AUC‐ROC. The results specifically highlight BENN's effectiveness in accurately classifying instances of minority classes, a notable challenge in many existing models. These findings underscore BENN's potential as a substantial advancement in handling class imbalance within big data environments, offering a promising direction for future research and application in machine learning.

An Enhanced Tree Ensemble for Classification in the Presence of Extreme Class Imbalance

Imbalanced Data Sets Classification Method Based on Over-Sampling Technique

Imbalanced Data Classification Algorithm Based on Integrated Sampling and Ensemble Learning.

BENN: Balanced Ensemble Neural Network for Handling Class Imbalance in Big Data

Equalization Ensemble for Large Scale Highly Imbalanced Data Classification

FORF-S: A Novel Classification Technique for Class Imbalance Problem

Class Imbalance Problem: A Wrapper-Based Approach using Under-Sampling with Ensemble Learning

Balancing the Scales: A Comprehensive Study on Tackling Class Imbalance in Binary Classification

Kd-Tree Based Efficient Ensemble Classification Algorithm for Imbalanced Learning

An ensemble imbalanced classification method based on model dynamic selection driven by data partition hybrid sampling

A New Sampling Approach for Classification of Imbalanced Data Sets with High Density.

Dbboost: Enhancing Imbalanced Classification by A Novel Ensemble Based Technique

Introducing DeepBalance: Random Deep Belief Network Ensembles to Address Class Imbalance

WOTBoost: Weighted Oversampling Technique in Boosting for imbalanced learning

Self-paced Ensemble for Highly Imbalanced Massive Data Classification

EHSBoost: Enhancing ensembles for imbalanced data-sets by evolutionary hybrid-sampling

A Novel Adaptive Minority Oversampling Technique for Improved Classification in Data Imbalanced Scenarios

Synthetic Over-sampling with the Minority and Majority Classes for Imbalance Problems

A cluster-based SMOTE both-sampling (CSBBoost) ensemble algorithm for classifying imbalanced data

A Diversity-Based Synthetic Oversampling Using Clustering for Handling Extreme Imbalance

Adaptive Subspace Optimization Ensemble Method for High-Dimensional Imbalanced Data Classification