Abstract:Decision makers often face the need of performance guarantee with some sufficiently high probability. Such problems can be modelled using a discrete time Markov decision process (MDP) with a probability criterion for the first achieving target value. The objective is to find a policy that maximizes the probability of the total discounted reward exceeding a target value in the preceding stages. We show that our formulation cannot be described by former models with standard criteria. We provide the properties of the objective functions, optimal value functions and optimal policies. An algorithm for computing the optimal policies for the finite horizon case is given. In this stochastic stopping model, we prove that there exists an optimal deterministic and stationary policy and the optimality equation has a unique solution. Using perturbation analysis, we approximate general models and prove the existence of ε-optimal policy for finite state space. We give an example for the reliability of the satellite systems using the above theory. Finally, we extend these results to more general cases.

First Passage Models for Denumerable Semi-Markov Decision Processes with Nonnegative Discounted Costs

Optimal Risk Probability for First Passage Models in Semi-Markov Decision Processes

First Passage Optimality for Continuous-Time Markov Decision Processes with Varying Discount Factors and History-Dependent Policies

First Passage Markov Decision Processes with Constraints and Varying Discount Factors

Mean-variance optimality for semi-Markov decision processes under first passage criteria.

A Minimization Problem of the Risk Probability in First Passage Semi-Markov Decision Processes with Loss Rates

On the First Passage G-Mean-variance Optimality for Discounted Continuous-Time Markov Decision Processes.

Constrained Markov Decision Processes with First Passage Criteria

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

Performance Optimization of Semi-Markov Decision Processes with Discounted-cost Criteria.

First Passage Risk Probability Minimization for Piecewise Deterministic Markov Decision Processes

Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates.

Minimizing Risk Models in Denumerable Semi-Markov Decision Processes with A Target Set

Continuous Time Markov Decision Processes with Expected Discounted Total Rewards

First Passage Problems for Nonstationary Discrete-Time Stochastic Control Systems

The Risk Probability Criterion for Discounted Continuous-Time Markov Decision Processes

Optimal Stopping Time on Discounted Semi-Markov Processes

Discrete-time Zero-Sum Markov Games with First Passage Criteria

Mixed Markov Decision Processes in a Semi-Markov Environment with Discounted Criterion

Optimal models with maximizing probability of first achieving target value in the preceding stages

Risk-sensitive Semi-Markov Decision Processes with General Utilities and Multiple Criteria