Abstract:This paper deals with a risk probability minimization problem for finite horizon partially observable semi-Markov decision processes, which are the fairly most general models for stochastic dynamic systems. In contrast to the expected discounted and average criteria, the optimality investigated in this paper is to minimize the probability that the accumulated rewards do not reach a prescribed profit level at the finite terminal stage. First, the state space is augmented as the joint conditional distribution of the current unobserved state and the remaining profit goal. We introduce a class of policies depending on observable histories and a class of Markov policies including observable process with the joint conditional distribution. Then under mild assumptions, we prove that the value function is the unique solution to the optimality equation for the probability criterion by using iteration techniques. The existence of (& varepsilon;-)optimal Markov policy for this problem is established. Finally, we use a bandit problem with the probability criterion to demonstrate our main results in which an effective algorithm and the corresponding numerical calculation are given for the semi-Markov model. Moreover, for the case of reduction to the discrete-time Markov model, we derive a concise solution.

Mixed Markov Decision Processes in a Semi-Markov Environment with Discounted Criterion

Performance Optimization of Semi-Markov Decision Processes with Discounted-cost Criteria.

Continuous Time Markov Decision Processes with Discounted Moment Criterion

A unified approach for semi-Markov decision processes with discounted and average reward criteria

A Survey on Semi-Markov Decision Processes

Continuous Time Markov Decision Processes with Expected Discounted Total Rewards

Performance Optimization for Countable Semi-Markov Decision Processes with Discounted-cost

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

Markov Decision Problems with Unbounded Transition Rates under Discounted-Cost Performance Criteria

Relations Between Discounted Models and Average Models for Semi-Markov Decision Processes

Optimal Replacement of a System According Toa Semi-Markov Decision Process in a Semi-Markov Environment

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion

Semi-Markov Decision Processes with Variance Minimization Criterion

Constrained Semi-Markov Decision Processes with Ratio and Time Expected Average Criteria in Polish Spaces

Risk-sensitive Semi-Markov Decision Processes with General Utilities and Multiple Criteria

Finite Horizon Partially Observable Semi-Markov Decision Processes under Risk Probability Criteria

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

First Passage Models for Denumerable Semi-Markov Decision Processes with Nonnegative Discounted Costs

Towards Analysis Of Semi-Markov Decision Processes

A Mean–variance Optimization Problem for Discounted Markov Decision Processes

Markov Decision Processes with Incomplete Information and Semi-Uniform Feller Transition Probabilities