Abstract:Abstract When making decisions, people often overlook critical information or are overly swayed by irrelevant information. A common approach to mitigate these biases is to provide decision-makers, especially professionals such as medical doctors, with decision aids, such as decision trees and flowcharts. Designing effective decision aids is a difficult problem. We propose that recently developed reinforcement learning methods for discovering clever heuristics for good decision-making can be partially leveraged to assist human experts in this design process. One of the biggest remaining obstacles to leveraging the aforementioned methods for improving human decision-making is that the policies they learn are opaque to people. To solve this problem, we introduce AI-Interpret: a general method for transforming idiosyncratic policies into simple and interpretable descriptions. Our algorithm combines recent advances in imitation learning and program induction with a new clustering method for identifying a large subset of demonstrations that can be accurately described by a simple, high-performing decision rule. We evaluate our new AI-Interpret algorithm and employ it to translate information-acquisition policies discovered through metalevel reinforcement learning. The results of three large behavioral experiments showed that providing the decision rules generated by AI-Interpret as flowcharts significantly improved people’s planning strategies and decisions across three different classes of sequential decision problems. Moreover, our fourth experiment revealed that this approach is significantly more effective at improving human decision-making than training people by giving them performance feedback. Finally, a series of ablation studies confirmed that our AI-Interpret algorithm was critical to the discovery of interpretable decision rules and that it is ready to be applied to other reinforcement learning problems. We conclude that the methods and findings presented in this article are an important step towards leveraging automatic strategy discovery to improve human decision-making. The code for our algorithm and the experiments is available at https://github.com/RationalityEnhancement/InterpretableStrategyDiscovery .

Experience-driven discovery of planning strategies

What are the mechanisms underlying metacognitive learning?

Improving Human Decision-Making by Discovering Efficient Strategies for Hierarchical Planning

Automatic discovery and description of human planning strategies

Analytics of Planning Behaviours in Self-Regulated Learning: Links with Strategy Use and Prior Knowledge

Automatic discovery of interpretable planning strategies

Intention as Hierarchical Constraints in Human Planning

An unsupervised adaptive strategy for constructing probabilistic roadmaps

Learning model-based planning from scratch

An intelligent tutor for planning in large partially observable environments

Leveraging automatic strategy discovery to teach people how to select better projects

Flexible and Efficient Long-Range Planning Through Curious Exploration

Curiosity-driven recommendation strategy for adaptive learning via deep reinforcement learning

Planning to Learn: A Novel Algorithm for Active Learning during Model-Based Planning

On Predictive planning and counterfactual learning in active inference

A recurrent network model of planning explains hippocampal replay and human behavior

Discovering Underlying Plans Based on Shallow Models

Learning Heuristic Selection with Dynamic Algorithm Configuration

When to Replan? An Adaptive Replanning Strategy for Autonomous Navigation using Deep Reinforcement Learning