Abstract:This paper deals with a risk probability minimization problem for finite horizon partially observable semi-Markov decision processes, which are the fairly most general models for stochastic dynamic systems. In contrast to the expected discounted and average criteria, the optimality investigated in this paper is to minimize the probability that the accumulated rewards do not reach a prescribed profit level at the finite terminal stage. First, the state space is augmented as the joint conditional distribution of the current unobserved state and the remaining profit goal. We introduce a class of policies depending on observable histories and a class of Markov policies including observable process with the joint conditional distribution. Then under mild assumptions, we prove that the value function is the unique solution to the optimality equation for the probability criterion by using iteration techniques. The existence of (& varepsilon;-)optimal Markov policy for this problem is established. Finally, we use a bandit problem with the probability criterion to demonstrate our main results in which an effective algorithm and the corresponding numerical calculation are given for the semi-Markov model. Moreover, for the case of reduction to the discrete-time Markov model, we derive a concise solution.

Semi-Markov Decision Processes with Variance Minimization Criterion

Markov Decision Processes with Variance Minimization: A New Condition and Approach

Mean-variance optimality for semi-Markov decision processes under first passage criteria.

Finite Horizon Partially Observable Semi-Markov Decision Processes under Risk Probability Criteria

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

A Mean–variance Optimization Problem for Discounted Markov Decision Processes

Optimization of Markov Decision Processes under the Variance Criterion

Mean-Variance Problems for Finite Horizon Semi-Markov Decision Processes

Finite Horizon Continuous-Time Markov Decision Processes with Mean and Variance Criteria

Nonstationary Denumerable State Markov Decision Processes – with Average Variance Criterion

Variance Minimization of Parameterized Markov Decision Processes

Mean-variance optimization of discrete time discounted Markov decision processes.

Optimization Of Parametric Policies Of Markov Decision Processes Under A Variance Criterion

Performance Optimization of Semi-Markov Decision Processes with Discounted-cost Criteria.

Risk-sensitive Semi-Markov Decision Processes with General Utilities and Multiple Criteria

New Average Optimality Conditions for Semi-Markov Decision Processes in Borel Spaces.

First Passage Markov Decision Processes with Constraints and Varying Discount Factors

Continuous Time Markov Decision Processes with Expected Discounted Total Rewards

Mixed Markov Decision Processes in a Semi-Markov Environment with Discounted Criterion

Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

Zero-Sum Semi-Markov Games with State-Action-Dependent Discount Factors