Abstract:Different from human nature, it is still common practice today for vision tasks to train deep learning models only initially and on fixed datasets. A variety of approaches have recently addressed handling continual data streams. However, extending these methods to manage out-of-distribution (OOD) scenarios has not effectively been investigated. On the other hand, it has recently been shown that non-continual neural mesh models exhibit strong performance in generalizing to such OOD scenarios. To leverage this decisive property in a continual learning setting, we propose incremental neural mesh models that can be extended with new meshes over time. In addition, we present a latent space initialization strategy that enables us to allocate feature space for future unseen classes in advance and a positional regularization term that forces the features of the different classes to consistently stay in respective latent space regions. We demonstrate the effectiveness of our method through extensive experiments on the Pascal3D and ObjectNet3D datasets and show that our approach outperforms the baselines for classification by $2-6\%$ in the in-domain and by $6-50\%$ in the OOD setting. Our work also presents the first incremental learning approach for pose estimation. Our code and model can be found at <a class="link-external link-https" href="https://github.com/Fischer-Tom/iNeMo" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the challenges in class - incremental learning (CIL) and dealing with out - of - distribution (OOD) scenarios. Specifically, the paper focuses on the following two main problems: 1. **The problem of catastrophic forgetting in class - incremental learning**: - In traditional deep - learning models, the model is usually trained all at once on a fixed training set. However, in practical applications, data often arrives gradually, and the model is required to be able to keep learning new classes without forgetting the knowledge of old classes. In this case, the model is prone to catastrophic forgetting, that is, forgetting the previously learned tasks when learning new tasks. - The paper proposes an incremental Neural Mesh Models (iNeMo), which effectively alleviates the problem of catastrophic forgetting by combining techniques such as knowledge distillation and replay buffer. 2. **The ability to handle out - of - distribution samples**: - Most of the existing class - incremental learning methods focus on handling in - distribution data, and perform poorly on out - of - distribution (OOD) data. Out - of - distribution data refers to those samples whose distribution is different from that of the data in the training set, such as severely occluded, noisy or images from different domains. - By introducing Neural Mesh Models (NeMo), iNeMo can not only maintain good performance in class - incremental learning, but also significantly improve the robustness to OOD data. ### Specific implementation methods To achieve the above goals, the paper proposes the following key techniques: - **Incremental Neural Mesh Models (iNeMo)**: Based on the existing Neural Mesh Models (NeMo), the model is expanded by adding new meshes to adapt to the learning of new classes. - **Latent space initialization strategy**: Allocate the feature space in advance for future unseen classes to ensure that the features of new classes can be reasonably distributed in the latent space. - **Position regularization term**: By introducing a regularization term, the features of different classes are forced to always remain within their respective latent space regions, thus avoiding confusion between features. - **Knowledge distillation**: Use the old version of the model for knowledge distillation to ensure that the new model will not forget the knowledge of the old task when learning a new task. - **Contrastive loss function**: Optimize the feature representation through the contrastive loss function to ensure that the features of new and old classes can be effectively separated. ### Experimental results The paper verifies the effectiveness of iNeMo through extensive experiments. The experimental results show that in the classification tasks on the Pascal3D and ObjectNet3D datasets, iNeMo improves the accuracy rate by 2% - 6% compared with the baseline method; in the OOD scenario, the performance improvement is more significant, reaching 6% - 50%. In addition, iNeMo also realizes incremental pose estimation for the first time and outperforms other methods in most cases. In conclusion, by introducing the iNeMo model, this paper successfully solves the problem of catastrophic forgetting in class - incremental learning and significantly improves the robustness to out - of - distribution samples.

iNeMo: Incremental Neural Mesh Models for Robust Class-Incremental Learning

Incremental Learning of Object Detectors without Catastrophic Forgetting

InOR-Net: Incremental 3-D Object Recognition Network for Point Cloud Representation

General Incremental Learning with Domain-aware Categorical Representations

Peculiarities of polyelectrolyte multilayer assembly on patterned surfaces.

Towards Class-incremental Object Detection with Nearest Mean of Exemplars

I3DOL: Incremental 3D Object Learning without Catastrophic Forgetting

Incremental learning with neural networks for computer vision: a survey

Modeling the Background for Incremental Learning in Semantic Segmentation

Incremental Deep Neural Network for Visual Categories

IL-NeRF: Incremental Learning for Neural Radiance Fields with Camera Pose Alignment

Class-Incremental Learning: Survey and Performance Evaluation on Image Classification

Rethinking Task-Incremental Learning Baselines

A Bio-Inspired Incremental Learning Architecture for Applied Perceptual Problems

PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning

Neural Network Retraining for Model Serving

Look At Me, No Replay! SurpriseNet: Anomaly Detection Inspired Class Incremental Learning

Model Behavior Preserving for Class-Incremental Learning

FearNet: Brain-Inspired Model for Incremental Learning

RD-IOD: Two-Level Residual-Distillation-Based Triple-Network for Incremental Object Detection

Neural Architecture Search for Class-incremental Learning