Abstract:Time-inconsistency refers to a paradox in decision making where agents exhibit inconsistent behaviors over time. Examples are procrastination where agents tends to costly postpone easy tasks, and abandonments where agents start a plan and quit in the middle. These behaviors are undesirable in the sense that agents make clearly suboptimal decisions over optimal ones. To capture such behaviors and more importantly, to quantify inefficiency caused by such behaviors, [Kleinberg&Oren 2014] propose a graph model which is essentially same as the standard planning model except for the cost structure. Using this model, they initiate the study of several interesting problems: 1) cost ratio: the worst ratio between the actual cost of the agent and the optimal cost, over all graph instances; 2) motivating subgraph: how to motivate the agent to reach the goal by deleting nodes and edges; 3) Intermediate rewards: how to motivate agents to reach the goal by placing intermediate rewards. Kleinberg and Oren give partial answers to these questions, but the main problems are still open. In fact, they raise these problems explicitly as open problems in their paper. In this paper, we give answers to all three open problems in [Kleinberg&Oren 2014]. First, we show a tight upper bound of cost ratio for graphs without Akerlof's structure, thus confirm the conjecture by Kleinberg and Oren that Akerlof's structure is indeed the worst case for cost ratio. Second, we prove that finding a motivating subgraph is NP-hard, showing that it is generally inefficient to motivate agents by deleting nodes and edges in the graph. Last but not least, we show that computing a strategy to place minimum amount of total reward is also NP-hard. Therefore, it is computational inefficient to motivate agents by placing intermediate rewards. The techniques we use to prove these results are nontrivial and of independent interests.

Graph Planning with Expected Finite Horizon

Computational Issues in Time-Inconsistent Planning

Planning with General Objective Functions: Going Beyond Total Rewards

Distributed Algorithm for a Finite Time Horizon Resource Allocation over a Directed Network

Secure Route Planning Using Dynamic Games with Stopping States

Path Planning under Time-Dependent Uncertainty

Time-Inconsistent Planning: A Computational Problem in Behavioral Economics

Optimistic Planning by Regularized Dynamic Programming

Fast Planning in Stochastic Games

Horizon-Free Reinforcement Learning in Polynomial Time: the Power of Stationary Policies

Planning Against a Prophet: A Graph-Theoretic Framework for Making Sequential Decisions

Intractability of Optimal Multi-Robot Path Planning on Planar Graphs

Stochastic Processes with Expected Stopping Time

Structure and Intractability of Optimal Multi-Robot Path Planning on Graphs

How long is long enough? Finite-horizon approximation of energy storage scheduling problems

Variational Planning for Graph-based MDPs.

Settling the Open Problems in Kleinberg-Oren Time-Inconsistent Planning Models.

Time-optimal Point-to-point Motion Planning: A Two-stage Approach

The Optimal Stopping Problem under a Random Horizon

Partial and Conditional Expectations in Markov Decision Processes with Integer Weights