Abstract:For reliability, machine learning models in some areas, e.g., finance and healthcare, require to be both accurate and globally interpretable. Among them, credit risk assessment is a major application of machine learning for financial institutions to evaluate credit of users and detect default or fraud. Simple white-box models, such as Logistic Regression (LR), are usually used for credit risk assessment, but not powerful enough to model complex nonlinear interactions among features. In contrast, complex black-box models are powerful at modeling, but lack of interpretability, especially global interpretability. Fortunately, automatic feature crossing is a promising way to find cross features to make simple classifiers to be more accurate without heavy handcrafted feature engineering. However, existing automatic feature crossing methods have problems in efficiency on credit risk assessment, for corresponding data usually contains hundreds of feature fields. In this work, we find local interpretations in Deep Neural Networks (DNNs) of a specific feature are usually inconsistent among different samples. We demonstrate this is caused by nonlinear feature interactions in the hidden layers of DNN. Thus, we can mine feature interactions in DNN, and use them as cross features in LR. This will result in mining cross features more efficiently. Accordingly, we propose a novel automatic feature crossing method called DNN2LR. The final model, which is a LR model empowered with cross features, generated by DNN2LR is a white-box model. We conduct experiments on both public and business datasets from real-world credit risk assessment applications, which show that, DNN2LR outperform both conventional models used for credit assessment and several feature crossing methods. Moreover, comparing with state-of-the-art feature crossing methods, i.e., AutoCross, the proposed DNN2LR method accelerates the speed by about 10 to 40 times on financial credit assessment datasets, which contain hundreds of feature fields.

AutoCross: Automatic Feature Crossing for Tabular Data in Real-World Applications

Cross-Reading by Leveraging a Hybrid Index of Heterogeneous Information.

Mining Cross Features for Financial Credit Risk Assessment

DNN2LR: Automatic Feature Crossing for Credit Scoring

Auto-FP: An Experimental Study of Automated Feature Preprocessing for Tabular Data

AutoGroup: Automatic Feature Grouping for Modelling Explicit High-Order Feature Interactions in CTR Prediction

CrossCat: A Fully Bayesian Nonparametric Method for Analyzing Heterogeneous, High Dimensional Data

Catch: Collaborative Feature Set Search for Automated Feature Engineering

XCrossNet: Feature Structure-Oriented Learning for Click-Through Rate Prediction

Automatic Large-Scale Data Acquisition via Crowdsourcing for Crosswalk Classification: A Deep Learning Approach

Automated Model Selection for Tabular Data

AutoFeature: Searching for Feature Interactions and Their Architectures for Click-through Rate Prediction

AutoKaggle: A Multi-Agent Framework for Autonomous Data Science Competitions

Crisscross optimization algorithm and its application

Self-optimizing Feature Generation via Categorical Hashing Representation and Hierarchical Reinforcement Crossing

scCross: a deep generative model for unifying single-cell multi-omics with seamless integration, cross-modal generation, and in silico exploration

Cross-Attention Regression Flow for Defect Detection

Enhancing Tabular Data Optimization with a Flexible Graph-based Reinforced Exploration Strategy

Cross-GCN: Enhancing Graph Convolutional Network with K-Order Feature Interactions

Automatic Financial Feature Construction

Feature Augmentation with Reinforcement Learning