Abstract:Feature transformation aims to reconstruct the feature space of raw features to enhance the performance of downstream models. However, the exponential growth in the combinations of features and operations poses a challenge, making it difficult for existing methods to efficiently explore a wide space. Additionally, their optimization is solely driven by the accuracy of downstream models in specific domains, neglecting the acquisition of general feature knowledge. To fill this research gap, we propose an evolutionary LLM framework for automated feature transformation. This framework consists of two parts: 1) constructing a multi-population database through an RL data collector while utilizing evolutionary algorithm strategies for database maintenance, and 2) utilizing the ability of Large Language Model (LLM) in sequence understanding, we employ few-shot prompts to guide LLM in generating superior samples based on feature transformation sequence distinction. Leveraging the multi-population database initially provides a wide search scope to discover excellent populations. Through culling and evolution, the high-quality populations are afforded greater opportunities, thereby furthering the pursuit of optimal individuals. Through the integration of LLMs with evolutionary algorithms, we achieve efficient exploration within a vast space, while harnessing feature knowledge to propel optimization, thus realizing a more adaptable search paradigm. Finally, we empirically demonstrate the effectiveness and generality of our proposed method.

Large Language Models for Transforming Categorical Data to Interpretable Feature Vectors

Implementing LLMs in industrial process modeling: Addressing Categorical Variables

Capture the Flag: Uncovering Data Insights with Large Language Models

Exploring Large Language Models for Feature Selection: A Data-centric Perspective

Evaluating Large Language Models on Time Series Feature Understanding: A Comprehensive Taxonomy and Benchmark

Large Language Models are Interpretable Learners

Evolutionary Large Language Model for Automated Feature Transformation

LLM-Select: Feature Selection with Large Language Models

Several categories of Large Language Models (LLMs): A Short Survey

Are Large Language Models the New Interface for Data Pipelines?

Large Language Models for Time Series: A Survey

Large Language Model in Medical Informatics: Direct Classification and Enhanced Text Representations for Automatic ICD Coding

Evaluating Large Language Models for Causal Modeling

Translating Expert Intuition into Quantifiable Features: Encode Investigator Domain Knowledge via LLM for Enhanced Predictive Analytics

Multilevel Large Language Models for Everyone

Can Large Language Models Transform Computational Social Science?

Incorporating LLM Priors into Tabular Learners

ELF-Gym: Evaluating Large Language Models Generated Features for Tabular Prediction

Rediscovering the Latent Dimensions of Personality with Large Language Models as Trait Descriptors

On Classification with Large Language Models in Cultural Analytics