Abstract:In numerous academic and practical settings, it is common to progressively add data features to existing models as data evolves due to new discoveries and ongoing data collection efforts. Incremental attribute learning focuses on incorporating these new feature dimensions efficiently without retraining the entire model. The key challenge for incremental attribute learning is adapting models to include new attributes dynamically while maintaining their performance on existing data formats. This research leverages strategies from existing incremental learning practices, applying them to incremental attribute learning through a new framework that utilizes knowledge distillation to prevent catastrophic forgetting. Additionally, this study proposes an embedding layer algorithm to manage the variability of input features in incremental learning effectively. Extensive experiments across 11 tabular datasets, covering binary classification, multi-class classification, and regression tasks, indicate that our approach surpasses traditional incremental attribute learning methods and conventional retraining in terms of accuracy and efficiency. Compared to existing incremental attribute learning models, our model requires only a fraction of the training time and achieves a relative improvement of 7.97% in classification tasks. Furthermore, our approach exhibits minimal forgetfulness of old format data, with a forgetting rate of approximately 1% per increment stage, while maintaining strong performance on new format data. Despite its advantages, our method is limited to tabular data and requires additional storage space. We encourage future research to delve deeper into incremental attribute learning and its broader applications, aiming to consistently enhance model performance as new attributes are introduced. Additionally, we suggest exploring methods for refining and selecting soft labels or intermediate vectors to optimize storage efficiency while effectively leveraging the original model's knowledge.

Feature Discriminability for Pattern Classification Based on Neural Incremental Attribute Learning

Statistical Discriminability Estimation for Pattern Classification Based on Neural Incremental Attribute Learning

OPTIMIZED NEURAL INCREMENTAL ATTRIBUTE LEARNING FOR CLASSIFICATION BASED ON STATISTICAL DISCRIMINABILITY

Feature Ordering for Neural Incremental Attribute Learning Based on Fisher's Linear Discriminant

Ordered Incremental Attribute Learning based on mRMR and Neural Networks

Evolving Linear Discriminant in a Continuously Growing Dimensional Space for Incremental Attribute Learning.

Linear Feature Sensibility for Output Partitioning in Ordered Neural Incremental Attribute Learning.

Neural Incremental Attribute Learning in Groups.

Feature selection in growing dimensional space for classification based on neural incremental attribute learning

CLDA: Feature Selection for Text Categorization Based on Constrained LDA

Pattern Classification with Ordered Features Using Mrmr and Neural Networks

Regression Based on Neural Incremental Attribute Learning with Correlation-Based Feature Ordering

Integrated Feature Preprocessing for Classification Based on Neural Incremental Attribute Learning.

Neural Incremental Attribute Learning Based on Principal Component Analysis

Output Effect Evaluation Based on Input Features in Neural Incremental Attribute Learning for Better Classification Performance

Incremental Subgraph Feature Selection for Graph Classification.

Integrated Low-Rank-Based Discriminative Feature Learning for Recognition

Hyper-feature aggregation and relaxed distillation for class incremental learning

Enhanced Local Gradient Order Features and Discriminant Analysis for Face Recognition

Incremental Attribute Learning by Knowledge distillation method

Incremental Classification of Remote Sensing Images with Feature Pyramid and Class Hierarchy