Abstract:We study the problem of balancing effectiveness and efficiency in automated feature selection. Feature selection is to find an optimal feature subset from large feature space. After exploring many feature selection methods, we observe a computational dilemma: 1) traditional feature selection (e.g., mRMR) is mostly efficient, but difficult to identify the best subset; 2) the emerging reinforced feature selection automatically navigates feature space to search the best subset, but is usually inefficient. Are automation and efficiency always apart from each other? Can we bridge the gap between effectiveness and efficiency under automation? Motivated by this dilemma, we aim to develop a novel feature space navigation method. In our preliminary work, we leveraged interactive reinforcement learning to accelerate feature selection by external trainer-agent interaction. Our preliminary work can be significantly improved by modeling the structured knowledge of its downstream task (e.g., decision tree) as learning feedback. In this journal version, we propose a novel interactive and closed-loop architecture to simultaneously model interactive reinforcement learning (IRL) and decision tree feedback (DTF). Specifically, IRL is to create an interactive feature selection loop and DTF is to feed structured feature knowledge back to the loop. The DTF improves IRL from two aspects. First, the tree-structured feature hierarchy generated by decision tree is leveraged to improve state representation. In particular, we represent the selected feature subset as an undirected graph of feature-feature correlations and a directed tree of decision features. We propose a new embedding method capable of empowering Graph Convolutional Network (GCN) to jointly learn state representation from both the graph and the tree. Second, the tree-structured feature hierarchy is exploited to develop a new reward scheme. In particular, we personalize reward assignment of agents based on decision tree feature importance. In addition, observing agents’ actions can also be a feedback, we devise another new reward scheme, to weigh and assign reward based on the selected frequency ratio of each agent in historical action records. Finally, we present extensive experiments with real-world datasets to demonstrate the improved performances of our method.

ACR-Tree: Constructing R-Trees Using Deep Reinforcement Learning.

TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning

Efficient Tree Policy with Attention-Based State Representation for Interactive Recommendation

A Novel Tree-Based Method for Interpretable Reinforcement Learning

Large-Scale Interactive Recommendation with Tree-Structured Reinforcement Learning.

RW-Tree: A Learned Workload-aware Framework for R-tree Construction

RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Space

Large-scale Interactive Recommendation with Tree-structured Policy Gradient

Forest-based Deep Recommender

Learning Deep Tree-based Retriever for Efficient Recommendation: Theory and Method

Interactive Reinforcement Learning for Feature Selection with Decision Tree in the Loop

Learning Decision Trees and Forests with Algorithmic Recourse

A Reinforcement Learning Approach to Online Learning of Decision Trees

Deep Reinforcement Learning Integrated RRT Algorithm for Path Planning

CDT: Cascading Decision Trees for Explainable Reinforcement Learning

Putting the Iterative Training of Decision Trees to the Test on a Real-World Robotic Task

Deep Reinforcement Learning for Tenuous Subgraph Finding

Mixed Deep Reinforcement Learning-behavior Tree for Intelligent Agents Design

Efficient Second-Order TreeCRF for Neural Dependency Parsing.

SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks

Reinforcement Learning with Tree-LSTM for Join Order Selection