Abstract:Dynamic streaming data is widespread in various real-world scenarios, and the distribution may change under unforeseen disturbances. The decrease in predicted performance caused by distribution change is known as concept drift. Many online learning algorithms utilize an ensemble of multiple classifiers to mitigate the impact but the generalization is constrained by the fitting capability of base classifiers. Deep neural networks have strong fitting ability and work well in static and offline learning environments while they may struggle in both dynamic adaptation and real-time learning. An Elastic Online Deep Learning (EODL) is proposed for dynamic streaming data with concept drift, which merges the fast adaptability of online ensemble methods and the fitting ability of deep neural networks. In the EODL model, each hidden layer is attached to a classifier, also called the intermediate classifier. These intermediate classifiers are regarded as base classifiers to obtain the ensemble prediction result. Then a depth adaption strategy is proposed to select different depth subnetworks for adapting the different distribution of new coming data through intermediate classifiers. With more in-distribution data arrived, deeper depth subnetworks will be selected through weights adjustment, and their training is based on the trained shallower subnetworks. When the data distribution changes, EODL will timely select shallower depth subnetworks to rapidly relearn the new data distribution. Moreover, a parameter adaptation strategy is proposed for mitigating the optimization conflict, which might occur in the backpropagation of hidden layers shared by intermediate classifiers when they have different predictions. The strategy works by setting an adaptive learning rate for each subnetwork located at different depths. Experimental results demonstrate that the EODL model can efficiently adapt to changes in dynamic streaming data and outperforms recent online learning models.

UGG-DA:Uncertainty-Guided Gradual Distribution Adaptation and Dynamic Prediction with Streaming Data

DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptation

SCD-CA: Stable Concept Drift Learning Based on Correlation Alignment for Streaming Data Forecasting.

Subspace Metric-Based Transfer Learning for Spindle Thermal Error Prediction under Time-Varying Conditions

Gradual Domain Adaptation: Theory and Algorithms

Elastic online deep learning for dynamic streaming data

T-SaS: Toward Shift-aware Dynamic Adaptation for Streaming Data

Theoretically Guaranteed Distribution Adaptable Learning

Transfer Learning with Dynamic Distribution Adaptation

A novel joint distinct subspace learning and dynamic distribution adaptation method for fault transfer diagnosis

Unsupervised Domain Adaptation via Domain-Adaptive Diffusion

Source-Free Domain Adaptation Via Target Prediction Distribution Searching

Deep transfer learning based on dynamic domain adaptation for remaining useful life prediction under different working conditions

Continual domain adaptation on aerial images under gradually degrading weather

Addressing Concept Shift in Online Time Series Forecasting: Detect-then-Adapt

A Unified Replay-based Continuous Learning Framework for Spatio-Temporal Prediction on Streaming Data

Domain adaptation with optimized feature distribution for streamer action recognition in live video

Act Now: A Novel Online Forecasting Framework for Large-Scale Streaming Data

Transfer learning for concept drifting data streams in heterogeneous environments

Streaming Active Deep Forest for Evolving Data Stream Classification