Abstract:In numerous academic and practical settings, it is common to progressively add data features to existing models as data evolves due to new discoveries and ongoing data collection efforts. Incremental attribute learning focuses on incorporating these new feature dimensions efficiently without retraining the entire model. The key challenge for incremental attribute learning is adapting models to include new attributes dynamically while maintaining their performance on existing data formats. This research leverages strategies from existing incremental learning practices, applying them to incremental attribute learning through a new framework that utilizes knowledge distillation to prevent catastrophic forgetting. Additionally, this study proposes an embedding layer algorithm to manage the variability of input features in incremental learning effectively. Extensive experiments across 11 tabular datasets, covering binary classification, multi-class classification, and regression tasks, indicate that our approach surpasses traditional incremental attribute learning methods and conventional retraining in terms of accuracy and efficiency. Compared to existing incremental attribute learning models, our model requires only a fraction of the training time and achieves a relative improvement of 7.97% in classification tasks. Furthermore, our approach exhibits minimal forgetfulness of old format data, with a forgetting rate of approximately 1% per increment stage, while maintaining strong performance on new format data. Despite its advantages, our method is limited to tabular data and requires additional storage space. We encourage future research to delve deeper into incremental attribute learning and its broader applications, aiming to consistently enhance model performance as new attributes are introduced. Additionally, we suggest exploring methods for refining and selecting soft labels or intermediate vectors to optimize storage efficiency while effectively leveraging the original model's knowledge.

Discretization of Continuous-Valued Attributes and Instance-Based Learning

Discretisation in Lazy Learning Algorithms

A New Discretization Approach of Continuous Attributes

Discretization for Inductive Learning from Continuous Sample Data

An ICA-Based multivariate discretization algorithm

Method of Discretization of Continuous Attributes of Decision Table

A New Discretization Algorithm for Continuous Interval-valued Attributes

Instance-Ranking: A New Perspective to Consider the Instance Dependency for Classification

The Discretization of Continuous Attributes Based on Compatibility Rough Set and Genetic Algorithm

A Survey of Discretization Techniques: Taxonomy and Empirical Analysis in Supervised Learning

Discretizing Numerical Attributes: An Analysis of Human Perceptions

On the Effectiveness of Discretizing Quantitative Attributes in Linear Classifiers

Towards using a Single Uniform Metric in Instance-Based Learning

Attribute reduction and optimal decision rules acquisition for continuous valued information systems

Learning bayesian networks structure with continuous variables

A New Supervised Discretization Method Based on Bayes Measure

Induction of Hybrid Decision Tree Based on Post-Discretization Strategy*

Incremental Attribute Learning by Knowledge distillation method

Nonuniform Dynamic Discretization in Hybrid Networks

Knowledge Reduction of General Rough Sets Based on Instance Learning

Learning with continuous piecewise linear decision trees