What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively train a neural network model in the context of Online Continual Learning (OCL) when labels are scarce. Specifically, the authors focus on the situations where the following three assumptions are usually not valid in real - world environments: 1. **Data is independently and identically distributed (A1)**: In reality, data streams are usually continuous and not independently and identically distributed. 2. **The model can see the training data multiple times (A2)**: With the infinite growth of data volume, it becomes difficult to ensure that the model accesses all data multiple times. 3. **The training data is fully labeled (A3)**: In reality, it is impossible to provide labels for every newly arriving data. To address these problems, the authors propose a new contrastive loss function, SemiCon (Semi - supervised Contrastive Loss), for partially - labeled data, which combines the advantages of supervised and self - supervised contrastive learning. This method aims to utilize labeled and unlabeled data by introducing a unified semi - supervised contrastive loss, thereby enabling effective online continual learning in the absence of labels. ### Main Contributions 1. **Propose a new contrastive loss function SemiCon**: It combines supervised contrastive loss and self - supervised contrastive loss and can effectively train the model when labels are scarce. 2. **Memory - based method**: Use limited labeled data and a large amount of unlabeled data streams. Each time the data added to the memory is labeled by an Oracle. 3. **Performance**: - On the Split - CIFAR10 dataset, it can achieve results similar to the fully - supervised method using only 2.6% of the labels. - On the Split - CIFAR100 dataset, it can also obtain similar performance using 10% of the labels. ### Method Overview - **Problem Definition**: In an incremental unlabeled data stream, selectively label specific data through an Oracle to form a partially - labeled dataset. - **SemiCon Loss Function**: It consists of two parts: - \( L_m \): Supervised contrastive loss, applicable to labeled memory data, treating unlabeled stream data as negative samples. - \( L_u \): Unsupervised contrastive loss, applicable to unlabeled stream data, treating labeled stream data as negative samples. The unified loss function is: \[ L_{\text{SemiCon}} = L_m+\alpha L_u \] where \(\alpha\) is a hyperparameter that controls the importance of unlabeled data. - **Training Process**: Combine batch data from the data stream and the memory bank, and update the model parameters through multi - view augmentation and SGD optimization. After each iteration, update the memory bank using reservoir sampling and label the newly added data through an Oracle. ### Experimental Results - Experiments on the Split - CIFAR10 and Split - CIFAR100 datasets show that this method outperforms existing semi - supervised OCL methods when labels are scarce and, in some cases, can be comparable to fully - supervised methods. Through these improvements, this paper demonstrates an effective solution to handling the label - scarcity problem in online continual learning.

Contrastive Learning for Online Semi-Supervised General Continual Learning

Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition

Continual Contrastive Self-supervised Learning for Image Classification.

A Unified Contrastive Loss for Self-Training

Interpolation-Based Contrastive Learning for Few-Label Semi-Supervised Learning

Semi-Supervised Graph Contrastive Learning with Virtual Adversarial Augmentation

Patch-Based Contrastive Learning and Memory Consolidation for Online Unsupervised Continual Learning

Semi-supervised Contrastive Learning Using Partial Label Information

ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning

Semi-Supervised Contrastive Learning of Musical Representations

A Comprehensive Empirical Evaluation on Online Continual Learning

Continual Contrastive Learning for Image Classification

Generalized Supervised Contrastive Learning

Hypernetworks for Continual Semi-Supervised Learning

Learning to Predict Gradients for Semi-Supervised Continual Learning

GBVSSL: Contrastive Semi-Supervised Learning Based on Generalized Bias-Variance Decomposition

Deep semi-supervised learning with contrastive learning and partial label propagation for image data

Contrastive Supervised Distillation for Continual Representation Learning

Online Continual Learning with Contrastive Vision Transformer

Class-Aware Contrastive Semi-Supervised Learning

Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation Distillation