Abstract:In this paper, we investigate dynamic feature selection within multivariate time-series scenario, a common occurrence in clinical prediction monitoring where each feature corresponds to a bio-test result. Many existing feature selection methods fall short in effectively leveraging time-series information, primarily because they are designed for static data. Our approach addresses this limitation by enabling the selection of time-varying feature subsets for each patient. Specifically, we employ reinforcement learning to optimize a policy under maximum cost restrictions. The prediction model is subsequently updated using synthetic data generated by trained policy. Our method can seamlessly integrate with non-differentiable prediction models. We conducted experiments on a sizable clinical dataset encompassing regression and classification tasks. The results demonstrate that our approach outperforms strong feature selection baselines, particularly when subjected to stringent cost limitations. Code will be released once paper is accepted.

What problem does this paper attempt to address?

This paper discusses the problem of dynamic feature selection in medical prediction monitoring, especially in the context of multivariate time series. This scenario is commonly seen in clinical prediction monitoring, where each feature corresponds to a biological detection result. Existing feature selection methods often fail to effectively utilize the time series information because they are designed for static data. The researchers propose a new approach that optimizes the feature subset selection strategy for each patient's time-varying features using reinforcement learning, to satisfy maximum cost constraints. The prediction model is then updated using synthetic data generated by the training strategy. This approach seamlessly integrates with non-differentiable prediction models and has been experimented on large clinical datasets for regression and classification tasks, showing superiority over traditional feature selection baselines, especially under strict cost constraints. Specifically, the researchers first train predictors using multivariate time series data, and then construct an environment based on the predictor and the training dataset. At each stage, sequences are randomly sampled from the training set, and the agent selects a feature subset to update at each step. The feedback from the environment includes cost rewards and prediction rewards. After policy convergence, new training sets are created based on the agent's action records of feature updates. Finally, a new predictor is trained using the generated dataset to adapt to the new data distribution. The main contributions of this paper include: 1. Proposing a new method for feature selection in multivariate time series, which outperforms existing methods, is applicable to non-differentiable predictors, and can handle regression and classification tasks. 2. Providing an interpretability method to compute time-varying feature importance, offering a more comprehensive temporal insight than existing methods, which is beneficial for data collection strategies in bedside monitoring and clinical research in healthcare. In addition, the paper also discusses related works such as filter, wrapper, and embedded feature selection methods, as well as the application of reinforcement learning on static clinical features. However, these methods do not consider the repeated occurrence of the same feature in clinical sequences. The proposed dynamic feature selection method takes into account the multiple updates and temporal changes of features, selecting different feature subsets for each sample sequence.

Dynamic feature selection in medical predictive monitoring by reinforcement learning

Extracting Dynamic Information of Temporal Clinical Data to Predict the Outcome in Critically Ill Patients.

Reinforcement Learning in Clinical Medicine: a Method to Optimize Dynamic Treatment Regime over Time.

Feature selection integrating Shapley values and mutual information in reinforcement learning: An application in the prediction of post-operative outcomes in patients with end-stage renal disease

Multiview Deep Learning-based Efficient Medical Data Management for Survival Time Forecasting

Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data

Predictive analytics with gradient boosting in clinical medicine

Automated Feature Selection: A Reinforcement Learning Perspective

Deep Reinforcement Learning for Cost-Effective Medical Diagnosis

Towards Dynamic Feature Acquisition on Medical Time Series by Maximizing Conditional Mutual Information

Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation

Dynamic Measurement Scheduling for Adverse Event Forecasting using Deep RL

A reinforcement learning guided adaptive cost-sensitive feature acquisition method

Effective Medical Test Suggestions Using Deep Reinforcement Learning

Learning to Maximize Mutual Information for Dynamic Feature Selection

Longitudinal LASSO: Jointly Learning Features and Temporal Contingency for Outcome Prediction

Sparse-attentive meta temporal point process for clinical decision support

HIGH-DIMENSIONAL A-LEARNING FOR OPTIMAL DYNAMIC TREATMENT REGIMES

Adversarial reinforcement learning for dynamic treatment regimes

Dynamic logistic state space prediction model for clinical decision making

A matching-based machine learning approach to estimating optimal dynamic treatment regimes with time-to-event outcomes