Abstract:We study the problem of balancing effectiveness and efficiency in automated feature selection. Feature selection is to find an optimal feature subset from large feature space. After exploring many feature selection methods, we observe a computational dilemma: 1) traditional feature selection (e.g., mRMR) is mostly efficient, but difficult to identify the best subset; 2) the emerging reinforced feature selection automatically navigates feature space to search the best subset, but is usually inefficient. Are automation and efficiency always apart from each other? Can we bridge the gap between effectiveness and efficiency under automation? Motivated by this dilemma, we aim to develop a novel feature space navigation method. In our preliminary work, we leveraged interactive reinforcement learning to accelerate feature selection by external trainer-agent interaction. Our preliminary work can be significantly improved by modeling the structured knowledge of its downstream task (e.g., decision tree) as learning feedback. In this journal version, we propose a novel interactive and closed-loop architecture to simultaneously model interactive reinforcement learning (IRL) and decision tree feedback (DTF). Specifically, IRL is to create an interactive feature selection loop and DTF is to feed structured feature knowledge back to the loop. The DTF improves IRL from two aspects. First, the tree-structured feature hierarchy generated by decision tree is leveraged to improve state representation. In particular, we represent the selected feature subset as an undirected graph of feature-feature correlations and a directed tree of decision features. We propose a new embedding method capable of empowering Graph Convolutional Network (GCN) to jointly learn state representation from both the graph and the tree. Second, the tree-structured feature hierarchy is exploited to develop a new reward scheme. In particular, we personalize reward assignment of agents based on decision tree feature importance. In addition, observing agents’ actions can also be a feedback, we devise another new reward scheme, to weigh and assign reward based on the selected frequency ratio of each agent in historical action records. Finally, we present extensive experiments with real-world datasets to demonstrate the improved performances of our method.

No-Fringe U-Tree: An Optimized Algorithm for Reinforcement Learning

An Observation Dimension Weight-Based U-Tree Algorithm

Tree Based Discretization for Continuous State Space Reinforcement Learning

Upside-Down Reinforcement Learning for More Interpretable Optimal Control

TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning

RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Space

Interactive Reinforcement Learning for Feature Selection with Decision Tree in the Loop

Online Reinforcement Learning for Real-Time Exploration in Continuous State and Action Markov Decision Processes

The tree reconstruction game: phylogenetic reconstruction using reinforcement learning

A novel reinforcement learning-based method for structure optimization

Mixed Deep Reinforcement Learning-behavior Tree for Intelligent Agents Design

Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning

An Optimal Computing Budget Allocation Tree Policy for Monte Carlo Tree Search

ACR-Tree: Constructing R-Trees Using Deep Reinforcement Learning.

Graph learning-based generation of abstractions for reinforcement learning

A Scalable Model-Free Recurrent Neural Network Framework for Solving POMDPs

Provably Efficient UCB-type Algorithms For Learning Predictive State Representations

A Scalable Derivative-free Exploration Approach for Reinforcement Learning

Planning spatial networks with Monte Carlo tree search

An Efficient Dynamic Sampling Policy for Monte Carlo Tree Search.

A* Tree Search for Portfolio Management