Abstract:In numerous academic and practical settings, it is common to progressively add data features to existing models as data evolves due to new discoveries and ongoing data collection efforts. Incremental attribute learning focuses on incorporating these new feature dimensions efficiently without retraining the entire model. The key challenge for incremental attribute learning is adapting models to include new attributes dynamically while maintaining their performance on existing data formats. This research leverages strategies from existing incremental learning practices, applying them to incremental attribute learning through a new framework that utilizes knowledge distillation to prevent catastrophic forgetting. Additionally, this study proposes an embedding layer algorithm to manage the variability of input features in incremental learning effectively. Extensive experiments across 11 tabular datasets, covering binary classification, multi-class classification, and regression tasks, indicate that our approach surpasses traditional incremental attribute learning methods and conventional retraining in terms of accuracy and efficiency. Compared to existing incremental attribute learning models, our model requires only a fraction of the training time and achieves a relative improvement of 7.97% in classification tasks. Furthermore, our approach exhibits minimal forgetfulness of old format data, with a forgetting rate of approximately 1% per increment stage, while maintaining strong performance on new format data. Despite its advantages, our method is limited to tabular data and requires additional storage space. We encourage future research to delve deeper into incremental attribute learning and its broader applications, aiming to consistently enhance model performance as new attributes are introduced. Additionally, we suggest exploring methods for refining and selecting soft labels or intermediate vectors to optimize storage efficiency while effectively leveraging the original model's knowledge.

Incremental Learning Of Bayesian Networks With Hidden Variables

Learning Bayesian Networks with Hidden Variables Using the Combination of EM and Evolutionary Algorithms

A Fast Incremental Method Based on Regularized Extreme Learning Machine

Hybrid Incremental Learning Algorithms for Bayesian Network Structures

Incremental Learning Bayesian Networks for Financial Data Modeling

Incremental learning Bayesian network structures efficiently

Learning Bayesian Networks From Data: An Efficient Approach Based On Extended Evolutionary Programming

Research on Incremental Learning of Bayesian Network Structure Based on Genetic Algorithms

Research On Learning Bayesian Network Structure With Hidden Variables Based On Genetic Algorithms

Incremental Learning Bayesian Networks for the Stock Return Prediction

Improved Population-Based Incremental Learning of Bayesian Networks with partly known structure and parallel computing

Evolutionary Lazy Learning for Naive Bayes Classification

Learning Bayesian Networks Structures Based on Extending Evolutionary Programming

A Real Coded Population-Based Incremental Learning for Inverse Problems in Continuous Space

Financial Data Modeling using a Hybrid Bayesian Network Structured Learning Algorithm

Learning Bayesian Networks Structures From Incomplete Data Based On Extending Evolutionary Programming

A New Method Of Learning Bayesian Networks Structures From Incomplete Data

An Improved Population-Based Incremental Learning Method For Inverse Problems

An Incremental Approach for Sparse Bayesian Network Structure Learning

A New Implementation of Population Based Incremental Learning Method for Optimizations in Electromagnetics

Incremental Attribute Learning by Knowledge distillation method