Abstract:In numerous academic and practical settings, it is common to progressively add data features to existing models as data evolves due to new discoveries and ongoing data collection efforts. Incremental attribute learning focuses on incorporating these new feature dimensions efficiently without retraining the entire model. The key challenge for incremental attribute learning is adapting models to include new attributes dynamically while maintaining their performance on existing data formats. This research leverages strategies from existing incremental learning practices, applying them to incremental attribute learning through a new framework that utilizes knowledge distillation to prevent catastrophic forgetting. Additionally, this study proposes an embedding layer algorithm to manage the variability of input features in incremental learning effectively. Extensive experiments across 11 tabular datasets, covering binary classification, multi-class classification, and regression tasks, indicate that our approach surpasses traditional incremental attribute learning methods and conventional retraining in terms of accuracy and efficiency. Compared to existing incremental attribute learning models, our model requires only a fraction of the training time and achieves a relative improvement of 7.97% in classification tasks. Furthermore, our approach exhibits minimal forgetfulness of old format data, with a forgetting rate of approximately 1% per increment stage, while maintaining strong performance on new format data. Despite its advantages, our method is limited to tabular data and requires additional storage space. We encourage future research to delve deeper into incremental attribute learning and its broader applications, aiming to consistently enhance model performance as new attributes are introduced. Additionally, we suggest exploring methods for refining and selecting soft labels or intermediate vectors to optimize storage efficiency while effectively leveraging the original model's knowledge.

Three-Fast-Inter Incremental Association Markov Blanket Learning Algorithm

Sparse Bayesian Approach to Fast Learning Network for Multiclassification.

Fast Inverse-Free Generalized Sparse Bayesian Iearning Algorithm

Improving the performance and explainability of knowledge tracing via Markov blanket

Mining Markov Blankets Without Causal Sufficiency

Inferring Gene Regulatory Networks Using the Improved Markov Blanket Discovery Algorithm

An Incremental Approach for Sparse Bayesian Network Structure Learning

Evaluation of the Performance of the Markov Blanket Bayesian Classifier Algorithm

Causal Structure Learning Based on Genetic Algorithm and Markov Blanket

Improved Iamb with Expanded Markov Blanket for High-Dimensional Time Series Prediction

Continuous Variable Based Bayesian Network Structure Learning from Financial Factors

Causal Structure Learning by Using Intersection of Markov Blankets

An Intelligent Multi-View Active Learning Method Based on a Double-Branch Network

Efficient heuristics for learning scalable Bayesian network classifier from labeled and unlabeled data

Exploring the Non-Trivial Knowledge Implicit in Test Instance to Fully Represent Unrestricted Bayesian Classifier.

Strong Approximate Markov Blanket and Its Application on Filter-Based Feature Selection

Fast Matrix Factorization with Non-Uniform Weights on Missing Data.

Incremental Learning Bayesian Networks for Financial Data Modeling

An Approximate Markov Blanket Feature Selection Algorithm

Active Incremental Learning for Health State Assessment of Dynamic Systems with Unknown Scenarios

Incremental Attribute Learning by Knowledge distillation method