Abstract:Dynamic streaming data is widespread in various real-world scenarios, and the distribution may change under unforeseen disturbances. The decrease in predicted performance caused by distribution change is known as concept drift. Many online learning algorithms utilize an ensemble of multiple classifiers to mitigate the impact but the generalization is constrained by the fitting capability of base classifiers. Deep neural networks have strong fitting ability and work well in static and offline learning environments while they may struggle in both dynamic adaptation and real-time learning. An Elastic Online Deep Learning (EODL) is proposed for dynamic streaming data with concept drift, which merges the fast adaptability of online ensemble methods and the fitting ability of deep neural networks. In the EODL model, each hidden layer is attached to a classifier, also called the intermediate classifier. These intermediate classifiers are regarded as base classifiers to obtain the ensemble prediction result. Then a depth adaption strategy is proposed to select different depth subnetworks for adapting the different distribution of new coming data through intermediate classifiers. With more in-distribution data arrived, deeper depth subnetworks will be selected through weights adjustment, and their training is based on the trained shallower subnetworks. When the data distribution changes, EODL will timely select shallower depth subnetworks to rapidly relearn the new data distribution. Moreover, a parameter adaptation strategy is proposed for mitigating the optimization conflict, which might occur in the backpropagation of hidden layers shared by intermediate classifiers when they have different predictions. The strategy works by setting an adaptive learning rate for each subnetwork located at different depths. Experimental results demonstrate that the EODL model can efficiently adapt to changes in dynamic streaming data and outperforms recent online learning models.

OEC: an Online Ensemble Classifier for Mining Data Streams with Noisy Labels

A learning algorithm for one-class data stream classification based on ensemble classifier

Classifying noisy data streams

Mining Concept-Drifting and Noisy Data Streams Using Ensemble Classifiers

An Ensemble Classifier Algorithm for Mining Data Streams Based on Concept Drift

An Aggregate Ensemble for Mining Concept Drifting Data Streams with Noise

Robust Ensemble Learning for Mining Noisy Data Streams.

Elastic online deep learning for dynamic streaming data

A Classifier Using Online Bagging Ensemble Method for Big Data Stream Learning

Online Boosting Adaptive Learning under Concept Drift for Multistream Classification

Selected Ensemble of Classifiers for Handling Concept-drifting Data Streams

An Ensemble Classifier for Mining Imbalanced Data Streams with Noise

Online Ensemble Using Adaptive Windowing for Data Streams with Concept Drift

A Method for Classifying Data Stream Based on Reverse Classifier

Online Active Learning with Drifted Data Streams Using Paired Ensemble Framework

Toward online node classification on streaming networks

Classification of High-Dimensional Evolving Data Streams Via a Resource-Efficient Online Ensemble

Learning Concept-Drifting Data Streams with Random Ensemble Decision Trees

Online Active Learning for Drifting Data Streams

Ensemble OS-ELM Based on Combination Weight for Data Stream Classification

Active Broad Learning with Multi-Objective Evolution for Data Stream Classification