Abstract:This paper deals with a risk probability minimization problem for finite horizon partially observable semi-Markov decision processes, which are the fairly most general models for stochastic dynamic systems. In contrast to the expected discounted and average criteria, the optimality investigated in this paper is to minimize the probability that the accumulated rewards do not reach a prescribed profit level at the finite terminal stage. First, the state space is augmented as the joint conditional distribution of the current unobserved state and the remaining profit goal. We introduce a class of policies depending on observable histories and a class of Markov policies including observable process with the joint conditional distribution. Then under mild assumptions, we prove that the value function is the unique solution to the optimality equation for the probability criterion by using iteration techniques. The existence of (& varepsilon;-)optimal Markov policy for this problem is established. Finally, we use a bandit problem with the probability criterion to demonstrate our main results in which an effective algorithm and the corresponding numerical calculation are given for the semi-Markov model. Moreover, for the case of reduction to the discrete-time Markov model, we derive a concise solution.

Risk-sensitive Semi-Markov Decision Processes with General Utilities and Multiple Criteria

Finite Horizon Partially Observable Semi-Markov Decision Processes under Risk Probability Criteria

Risk-sensitive Continuous-Time Markov Decision Processes with Unbounded Rates and Borel Spaces

Minimum Risk Probability for Finite Horizon Semi-Markov Decision Processes

Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates.

Finite Horizon Semi-Markov Decision Processes with Application to Maintenance Systems.

Optimal Risk Probability for First Passage Models in Semi-Markov Decision Processes

Risk-sensitive Infinite-Horizon Discounted Piecewise Deterministic Markov Decision Processes

Minimizing Risk Models in Denumerable Semi-Markov Decision Processes with A Target Set

Finite Horizon Semi-Markov Decision Processes with Multiple Constraints

A Risk Minimization Problem for Finite Horizon Semi-Markov Decision Processes with Loss Rates

Semi-Markov Decision Processes with Variance Minimization Criterion

Risk-Sensitive Average Markov Decision Processes in General Spaces

Finite Horizon Risk-Sensitive Continuous-Time Markov Decision Processes with Unbounded Transition and Cost Rates

A Survey on Semi-Markov Decision Processes

Constrained Semi-Markov Decision Processes with Ratio and Time Expected Average Criteria in Polish Spaces

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

Markov Decision Processes with Risk-Sensitive Criteria: An Overview

Risk-sensitive Average Continuous-Time Markov Decision Processes with Unbounded Transition and Cost Rates.